o .if%@sdZddlZddlmZmZmZmZddlmZddl m Z ddl m Z ddl mZmZddlmZdd lmZmZdd lmZmZdd lmZmZGd d d eZdS)zp Ensemble retriever that ensemble the results of multiple retrievers by using weighted Reciprocal Rank Fusion N)AnyDictListOptional)Document)dumpd)root_validator) BaseRetriever RetrieverLike)RunnableConfig) ensure_config patch_config)ConfigurableFieldSpecget_unique_config_specs)#AsyncCallbackManagerForRetrieverRunCallbackManagerForRetrieverRunc @sfeZdZUdZeeed<eeed<dZe ed<e dee fddZ e d d d eeefdeeeffd d Z d$dedeededeefddZ d$dedeededeefddZdededeefddZdededeefddZdddededeedeefddZdddededeedeefdd Zd!eeedeefd"d#ZdS)%EnsembleRetrieveraRetriever that ensembles the multiple retrievers. It uses a rank fusion. Args: retrievers: A list of retrievers to ensemble. weights: A list of weights corresponding to the retrievers. Defaults to equal weighting for all retrievers. c: A constant added to the rank, controlling the balance between the importance of high-ranked items and the consideration given to lower-ranked items. Default is 60. retrieversweights<creturncCstdd|jDS)z+List configurable fields for this runnable.css |] }|jD]}|VqqdSN) config_specs).0 retrieverspecrV/var/www/html/corbot_env/lib/python3.10/site-packages/langchain/retrievers/ensemble.py .s  z1EnsembleRetriever.config_specs..)rr)selfrrrr+szEnsembleRetriever.config_specsT)prevaluescCs,|dst|d}d|g||d<|S)Nrr)getlen)clsr" n_retrieversrrr set_weights2s  zEnsembleRetriever.set_weightsNinputconfigkwargsc Ksddlm}t|}|j|dd|dd|dg|j|di|jd}|jt||fd |d i|}z |j |||d }Wnt yU}z| ||d}~ww|j |fi||S) Nr)CallbackManager callbacksverboseFtagsmetadatar.inheritable_tags local_tagsinheritable_metadatalocal_metadatanamerun_name run_managerr*) langchain_core.callbacks.managerr,r configurer$r/r0on_retriever_startr rank_fusion Exceptionon_retriever_erroron_retriever_end) r r)r*r+r,callback_managerr9resulterrrinvoke9s@      zEnsembleRetriever.invokec sddlm}t|}|j|dd|dd|dg|j|di|jd}|jt||fd |d i|IdH}z |j |||d IdH}Wnt y_}z | |IdH|d}~ww|j |fi|IdH|S) Nr)AsyncCallbackManagerr-r.Fr/r0r1r6r7r8) r:rEr r;r$r/r0r<r arank_fusionr>r?r@) r r)r*r+rErAr9rBrCrrrainvokeZsF       zEnsembleRetriever.ainvokequeryr9cCs|||}|S)z Get the relevant documents for a given query. Args: query: The query to search for. Returns: A list of reranked documents. )r=r rHr9fused_documentsrrr_get_relevant_documents}s z)EnsembleRetriever._get_relevant_documentscs|||IdH}|S)z Asynchronously get the relevant documents for a given query. Args: query: The query to search for. Returns: A list of reranked documents. N)rFrIrrr_aget_relevant_documentssz*EnsembleRetriever._aget_relevant_documents)r*csRfddt|jD}tt|D] }dd||D||<q||}|S)z Retrieve the results of the retrievers and use rank_fusion_func to get the final result. Args: query: The query to search for. Returns: A list of reranked documents. c 6g|]\}}|tjd|dddqS retriever_r#)tag)r-)rDr get_childrirr*rHr9rr z1EnsembleRetriever.rank_fusion..cS$g|]}t|tst|dn|qS page_content isinstancerrdocrrrrU) enumeraterranger%weighted_reciprocal_rankr rHr9r*retriever_docsrSrJrrTrr=s  zEnsembleRetriever.rank_fusioncs`tjfddt|jDIdH}tt|D] }dd||D||<q||}|S)z Asynchronously retrieve the results of the retrievers and use rank_fusion_func to get the final result. Args: query: The query to search for. Returns: A list of reranked documents. c rMrN)rGr rQrRrTrrrUrVz2EnsembleRetriever.arank_fusion..NcSrWrXr[r]rrrrUr_)asynciogatherr`rrar%rbrcrrTrrFs   zEnsembleRetriever.arank_fusion doc_listsc st|t|jkr tdt}|D] }|D]}||jqqdd|Dt||jD]!\}}t|ddD]\}}|d||j}|j|7<q7q-t  fdddd }d d|Dfd d |D} | S) a Perform weighted Reciprocal Rank Fusion on multiple rank lists. You can find more details about RRF here: https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf Args: doc_lists: A list of rank lists, where each rank list contains unique items. Returns: list: The final aggregated list of items sorted by their weighted RRF scores in descending order. zsz>EnsembleRetriever.weighted_reciprocal_rank..r#)startcs|Srr)x) rrf_score_dicrr sz.T)keyreversecSsi|] }|D]}|j|qqSrrY)rdoc_listr^rrrrh$s csg|]}|qSrr)rrZ)page_content_to_doc_maprrrU'sz>EnsembleRetriever.weighted_reciprocal_rank..) r%r ValueErrorsetaddrZzipr`rsortedkeys) r rg all_documentsror^weightrank rrf_scoresorted_documents sorted_docsr)rprkrrbs2 z*EnsembleRetriever.weighted_reciprocal_rankr)__name__ __module__ __qualname____doc__rr __annotations__floatrintpropertyrrrrstrrr(rr rrDrGrrKrrLr=rFrbrrrrrs   $ " #   . + r)rretypingrrrrlangchain_core.documentsrlangchain_core.load.dumprlangchain_core.pydantic_v1rlangchain_core.retrieversr r langchain_core.runnablesr langchain_core.runnables.configr r langchain_core.runnables.utilsrrlangchain.callbacks.managerrrrrrrrs