o ZhJ@s2dZddlZddlZddlmZddlmZddlmZm Z m Z m Z ddl m Z mZmZddlmZddlmZdd lmZmZdd lmZdd lmZmZdd lmZd efddZee Z!ej"ddd e#fddZ$d efddZ%Gddde&eZ'GdddeZ(Gddde(eZ)Gddde(eZ*dS)z@A chain for comparing the output of two models using embeddings.N)Enum)util)AnyDictListOptional)AsyncCallbackManagerForChainRunCallbackManagerForChainRun Callbacks) Embeddings)pre_init) ConfigDictField)Chain)PairwiseStringEvaluatorStringEvaluatorRUN_KEYreturnc Cs2zddl}|WSty}ztd|d}~ww)Nrz@Could not import numpy, please install with `pip install numpy`.)numpy ImportError)nperc/var/www/html/lang_env/lib/python3.10/site-packages/langchain/evaluation/embedding_distance/base.py _import_numpysr)maxsizecCs ttdr dStddS)NrTaNumPy not found in the current Python environment. langchain will use a pure Python implementation for embedding distance operations, which may significantly impact performance, especially for large datasets. For optimal speed and efficiency, consider installing NumPy: pip install numpyF)boolr find_specloggerwarningrrrr _check_numpy%s r"c CsRz ddlm}W|Sty(z ddlm}WY|Sty'tdww)zaCreate an Embeddings object. Returns: Embeddings: The created Embeddings object. rOpenAIEmbeddingstCould not import OpenAIEmbeddings. Please install the OpenAIEmbeddings package using `pip install langchain-openai`.)langchain_openair$r%langchain_community.embeddings.openair#rrr_embedding_factory3s  r(c@s$eZdZdZdZdZdZdZdZdS)EmbeddingDistancea Embedding Distance Metric. Attributes: COSINE: Cosine distance metric. EUCLIDEAN: Euclidean distance metric. MANHATTAN: Manhattan distance metric. CHEBYSHEV: Chebyshev distance metric. HAMMING: Hamming distance metric. ZcosineZ euclideanZ manhattanZ chebyshevZhammingN) __name__ __module__ __qualname____doc__COSINE EUCLIDEAN MANHATTAN CHEBYSHEVHAMMINGrrrrr)Js r)c@s6eZdZUdZeedZeed<ee j dZ e ed<e de eefde eeffdd Zed d Zedeefd d ZdedefddZde defddZedededefddZedededefddZedededefddZedededefddZedededefddZd edefd!d"Z d#S)$_EmbeddingDistanceChainMixina0Shared functionality for embedding distance evaluators. Attributes: embeddings (Embeddings): The embedding objects to vectorize the outputs. distance_metric (EmbeddingDistance): The distance metric to use for comparing the embeddings. )default_factory embeddings)defaultdistance_metricvaluesrcCs|d}g}z ddlm}||Wn tyYnwz ddlm}||Wn ty4Ynw|s;tdt|t|rUzddl}W|StyTtdw|S)zValidate that the TikTok library is installed. Args: values (Dict[str, Any]): The values to validate. Returns: Dict[str, Any]: The validated values. r5rr#r%NzThe tiktoken library is required to use the default OpenAI embeddings with embedding distance evaluators. Please either manually select a different Embeddings object or install tiktoken using `pip install tiktoken`.) getr&r$appendrr' isinstancetupletiktoken)clsr8r5types_r$r=rrr_validate_tiktoken_installedhs8       z9_EmbeddingDistanceChainMixin._validate_tiktoken_installedT)Zarbitrary_types_allowedcCsdgS)zgReturn the output keys of the chain. Returns: List[str]: The output keys. scorerselfrrr output_keyssz(_EmbeddingDistanceChainMixin.output_keysresultcCs$d|di}t|vr|t|t<|S)NrAr)rCrEparsedrrr_prepare_outputs  z,_EmbeddingDistanceChainMixin._prepare_outputmetricc CsJtj|jtj|jtj|jtj|jtj |j i}||vr||St d|)zGet the metric function for the given metric name. Args: metric (EmbeddingDistance): The metric name. Returns: Any: The metric function. zInvalid metric: ) r)r._cosine_distancer/_euclidean_distancer0_manhattan_distancer1_chebyshev_distancer2_hamming_distance ValueError)rCrHZmetricsrrr _get_metrics z(_EmbeddingDistanceChainMixin._get_metricabcCs6zddlm}Wn tytdwd|||S)zCompute the cosine distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.ndarray: The cosine distance. r)cosine_similarityzThe cosine_similarity function is required to compute cosine distance. Please install the langchain-community package using `pip install langchain-community`.g?)Zlangchain_community.utils.mathrRr)rPrQrRrrrrIs  z-_EmbeddingDistanceChainMixin._cosine_distancecCs:trddl}|j||Stddt||DdS)zCompute the Euclidean distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Euclidean distance. rNcss$|] \}}||||VqdSNr.0xyrrr s"zC_EmbeddingDistanceChainMixin._euclidean_distance..g?)r"rZlinalgZnormsumziprPrQrrrrrJs z0_EmbeddingDistanceChainMixin._euclidean_distancecC8trt}||||Stddt||DS)zCompute the Manhattan distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Manhattan distance. cs |] \}}t||VqdSrSabsrTrrrrXzC_EmbeddingDistanceChainMixin._manhattan_distance..)r"rrYr_rZr[rrrrK z0_EmbeddingDistanceChainMixin._manhattan_distancecCr\)zCompute the Chebyshev distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Chebyshev distance. csr]rSr^rTrrrrXr`zC_EmbeddingDistanceChainMixin._chebyshev_distance..)r"rmaxr_rZr[rrrrLraz0_EmbeddingDistanceChainMixin._chebyshev_distancecCs:tr t}|||kStddt||Dt|S)zCompute the Hamming distance between two vectors. Args: a (np.ndarray): The first vector. b (np.ndarray): The second vector. Returns: np.floating: The Hamming distance. css |] \}}||krdVqdS)rNrrTrrrrXr`zA_EmbeddingDistanceChainMixin._hamming_distance..)r"rmeanrYrZlenr[rrrrMs  z._EmbeddingDistanceChainMixin._hamming_distancevectorscCsh||j}tr't|tjr'||ddd|ddd}t|S||d|d}t|S)zCompute the score based on the distance metric. Args: vectors (np.ndarray): The input vectors. Returns: float: The computed score. rr) rOr7r"r;rZndarrayZreshapeitemfloat)rCrerHrArrr_compute_scores &z+_EmbeddingDistanceChainMixin._compute_scoreN)!r*r+r,r-rr(r5r __annotations__r)r.r7r rstrrr@r Z model_configpropertyrrDdictrGrO staticmethodrIrJrKrLrMrhrirrrrr3\s0 $-r3c@sFeZdZdZedefddZedefddZede efddZ dd e ee fd e ede ee ffd d Z dd e ee fd e ede ee ffddZd d d d dddede edede e ede e ee fdede defddZd d d d dddede edede e ede e ee fdede defddZd S)EmbeddingDistanceEvalChaina"Use embedding distances to score semantic difference between a prediction and reference. Examples: >>> chain = EmbeddingDistanceEvalChain() >>> result = chain.evaluate_strings(prediction="Hello", reference="Hi") >>> print(result) {'score': 0.5} rcCsdS)zReturn whether the chain requires a reference. Returns: bool: True if a reference is required, False otherwise. TrrBrrrrequires_reference4sz-EmbeddingDistanceEvalChain.requires_referencecCd|jjdS)NZ embedding_ _distancer7valuerBrrrevaluation_name=z*EmbeddingDistanceEvalChain.evaluation_namecCddgS)eReturn the input keys of the chain. Returns: List[str]: The input keys. prediction referencerrBrrr input_keysAz%EmbeddingDistanceEvalChain.input_keysNinputs run_managercC@|j|d|dg}trt}||}||}d|iS)a0Compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (Optional[CallbackManagerForChainRun], optional): The callback manager. Returns: Dict[str, Any]: The computed score. ryrzrAr5Zembed_documentsr"rarrayrirCr}r~rerrArrr_callJs  z EmbeddingDistanceEvalChain._callcH|j|d|dgIdH}trt}||}||}d|iS)a:Asynchronously compute the score for a prediction and reference. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. ryrzNrAr5Zaembed_documentsr"rrrirrrr_acallb   z!EmbeddingDistanceEvalChain._acallF)rz callbackstagsmetadatainclude_run_inforyrzrrrrkwargsc K"|||d||||d}||S)aEvaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The reference string (required) callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. ryrzr}rrrrrG rCryrzrrrrrrErrr_evaluate_strings}s z,EmbeddingDistanceEvalChain._evaluate_stringsc ,|j||d||||dIdH}||S)aAsynchronously evaluate the embedding distance between a prediction and reference. Args: prediction (str): The output string from the first model. reference (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNZacallrGrrrr_aevaluate_stringss z-EmbeddingDistanceEvalChain._aevaluate_stringsrS)r*r+r,r-rlrrprkrurr{rrrr rrrr rmrrrrrrro)s           &   roc@s(eZdZdZedeefddZedefddZ dde ee fd e e de ee ffd d Z dde ee fd e ede ee ffd d Zddddddededede eede e ee fdede defddZddddddededede eede e ee fdede defddZdS)"PairwiseEmbeddingDistanceEvalChainaUse embedding distances to score semantic difference between two predictions. Examples: >>> chain = PairwiseEmbeddingDistanceEvalChain() >>> result = chain.evaluate_string_pairs(prediction="Hello", prediction_b="Hi") >>> print(result) {'score': 0.5} rcCrw)rxry prediction_brrBrrrr{r|z-PairwiseEmbeddingDistanceEvalChain.input_keyscCrq)NZpairwise_embedding_rrrsrBrrrrurvz2PairwiseEmbeddingDistanceEvalChain.evaluation_nameNr}r~cCr)aCompute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (CallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. ryrrArrrrrrs  z(PairwiseEmbeddingDistanceEvalChain._callcr)a/Asynchronously compute the score for two predictions. Args: inputs (Dict[str, Any]): The input data. run_manager (AsyncCallbackManagerForChainRun, optional): The callback manager. Returns: Dict[str, Any]: The computed score. ryrNrArrrrrrrz)PairwiseEmbeddingDistanceEvalChain._acallF)rrrrryrrrrrrc Kr)aEvaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. ryrrr rCryrrrrrrrErrr_evaluate_string_pairss z9PairwiseEmbeddingDistanceEvalChain._evaluate_string_pairsc r)aAsynchronously evaluate the embedding distance between two predictions. Args: prediction (str): The output string from the first model. prediction_b (str): The output string from the second model. callbacks (Callbacks, optional): The callbacks to use. tags (List[str], optional): Tags to apply to traces metadata (Dict[str, Any], optional): metadata to apply to traces **kwargs (Any): Additional keyword arguments. Returns: dict: A dictionary containing: - score: The embedding distance between the two predictions. rrNrrrrr_aevaluate_string_pairs4s z:PairwiseEmbeddingDistanceEvalChain._aevaluate_string_pairsrS)r*r+r,r-rlrrkr{rurrrr rrrr rrmrrrrrrrs           (   r)+r- functoolsloggingenumr importlibrtypingrrrrZ langchain_core.callbacks.managerrr r Zlangchain_core.embeddingsr Zlangchain_core.utilsr Zpydanticr rZlangchain.chains.baserZlangchain.evaluation.schemarrZlangchain.schemarr getLoggerr*r lru_cacherr"r(rkr)r3rorrrrrs4        N