o Zh3@sddlmZddlZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddlmZddlmZddlmZeeZGdd d eZdS) ) annotationsN)AnyIterableListOptionalTuple)Document) Embeddings) VectorStore)maximal_marginal_relevancec @seZdZdZdddddddddFddZedGddZ    dHdId(d)Z *  dJdKd2d3Z *  dJdLd5d6Z * 7 8 dMdNd=d>Z * 7 8 dMdOd?d@Z e dddddddAgdddf dPdBdCZ dQdRdDdEZdS)SDingoax`Dingo` vector store. To use, you should have the ``dingodb`` python package installed. Example: .. code-block:: python from langchain_community.vectorstores import Dingo from langchain_community.embeddings.openai import OpenAIEmbeddings embeddings = OpenAIEmbeddings() dingo = Dingo(embeddings, "text") NirootZ123123F)client index_name dimensionhostuserpasswordself_id embeddingr text_keystrrrr Optional[str]rintrOptional[List[str]]rrrboolc Cszddl} Wn tytdw|dur|ndg}|dur"|} nz | |||} Wnty>} ztd| d} ~ ww||_| |_|durk|| vrk|| vrk| durd| j||ddn| j||d ||_ ||_ dS) zInitialize with Dingo client.rNzSCould not import dingo python package. Please install it with `pip install dingodb.172.20.31.10:13000Dingo failed to connect: TFrZauto_idr) dingodb ImportErrorDingoDB ValueError _text_key_client get_indexupper create_index _index_name _embedding) selfrrrrrrrrrr dingo_clienter.]/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/vectorstores/dingo.py__init__ s8    zDingo.__init__returnOptional[Embeddings]cCs|jSN)r*r+r.r.r/ embeddingsTszDingo.embeddingstexttexts Iterable[str] metadatasOptional[List[dict]]ids batch_sizekwargs List[str]c Ks|pdd|D}g}t|}|j|}t|D]\} } |r#|| ni} | | |j<|| qtdtt||D]"} | |} |j |j || | || | || | } | s\t dq:|S)aRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. ids: Optional list of ids to associate with the texts. Returns: List of ids from adding the texts into the vectorstore. cS"g|] }ttjddqSN ruuiduuid4r.0_r.r.r/ n"z#Dingo.add_texts..rvector add fail) listr*embed_documents enumerater$appendrangelenr% vector_addr) Exception)r+r8r:r<rr=r>metadatas_listembedsir6metadatajadd_resr.r.r/ add_textsXs"   "zDingo.add_textsqueryk search_paramsOptional[dict]timeout Optional[int]List[Document]cKs&|j|f||d|}dd|DS)vReturn Dingo documents most similar to query, along with scores. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. search_params: Dictionary of argument(s) to filter on metadata Returns: List of Documents most similar to the query and score for each )r]r^cSsg|]\}}|qSr.r.)rGdocrHr.r.r/rIsz+Dingo.similarity_search..)similarity_search_with_score)r+r\r]r^r`r>Zdocs_and_scoresr.r.r/similarity_searchszDingo.similarity_searchList[Tuple[Document, float]]cKsg}|j|}|jj|j|||d}|sgS|ddD]P} | d} d|vr6|ddur6| |dkr6q| d} | d} | |jd dd } | | | d }| D]}| |d dd ||<qS|t | |d | fq|S) rc)xqtop_kr^rvectorWithDistancesZdistanceZscore_thresholdN scalarDataidfieldsdata)rlr6scoreZ page_contentrW) r* embed_queryr% vector_searchr)getr$keysrOr)r+r\r]r^r`r>docsZ query_objresultsresror:rlr6rWZmeta_keyr.r.r/res*    z"Dingo.similarity_search_with_score? List[float]fetch_k lambda_multfloatc  sjjj|g||d}ttj|gtjddd|ddD||d}g} |D])} i} |dd| dD]\}} | t || d dd iq8| | q(fd d| DS) aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents selected by maximal marginal relevance. )r^ri)ZdtypecSsg|]}|ddqS)ZvectorZ floatValuesr.)rGitemr.r.r/rIs zADingo.max_marginal_relevance_search_by_vector..rrj)r]r|rkrmrncs g|] }t|j|dqS)rp)rpopr$)rGrWr4r.r/rIs) r%rrr)r nparrayZfloat32itemsupdaterrO) r+rr]r{r|r^r>rvZ mmr_selectedselectedrVZ meta_datavr.r4r/'max_marginal_relevance_search_by_vectors(       z-Dingo.max_marginal_relevance_search_by_vectorcKs|j|}||||||S)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. Returns: List of Documents selected by maximal marginal relevance. )r*rqr)r+r\r]r{r|r^r>rr.r.r/max_marginal_relevance_searchs  z#Dingo.max_marginal_relevance_searchrc  Kszddl}Wn tytdw|dur|}nz || | | }Wnty5}ztd|d}~ww| dur\| ddur\|dur[||vr[||vr[|j||ddn|duru||vru||vru|j||d |p}d d |D}g}t|}| |}t |D]\}}|r||ni}|||<| |qt dt t|| D] }|| }|||||||||||}|std q|||||d S)a=Construct Dingo wrapper from raw documents. This is a user friendly interface that: 1. Embeds documents. 2. Adds the documents to a provided Dingo index This is intended to be a quick way to get started. Example: .. code-block:: python from langchain_community.vectorstores import Dingo from langchain_community.embeddings import OpenAIEmbeddings import dingodb sss embeddings = OpenAIEmbeddings() dingo = Dingo.from_texts( texts, embeddings, index_name="langchain-demo" ) rNzTCould not import dingo python package. Please install it with `pip install dingodb`.rrTFrrcSr@rArCrFr.r.r/rI^rJz$Dingo.from_texts..rK)rr)r r!r"r#rsr&r'r(rLrMrNrOrPrQrRrS)clsr8rr:r<rrrrrrrr=r>r r,r-rTrUrVr6rWrXrYr.r.r/ from_textssX'       zDingo.from_textscKs"|durtd|jj|j|dS)z^Delete by vector IDs or filter. Args: ids: List of ids to delete. NzNo ids provided to delete.)r<)r#r%Z vector_deleter))r+r<r>r.r.r/deleteqs z Dingo.delete)rr rrrrrrrrrrrrrrrr)r1r2)NNr6r7)r8r9r:r;r<rrrr=rr>rr1r?)r[NN) r\rr]rr^r_r`rar>rr1rb) r\rr]rr^r_r`rar>rr1rg)r[rxryN)rrzr]rr{rr|r}r^r_r>rr1rb)r\rr]rr{rr|r}r^r_r>rr1rb)r8r?rr r:r;r<rrrrrrrrrrr?rrrrr=rr>rr1r r3)r<rr>rr1r)__name__ __module__ __qualname____doc__r0propertyr5rZrfrerr classmethodrrr.r.r.r/r sb4  ,  0 4 ^r ) __future__rloggingrDtypingrrrrrnumpyrZlangchain_core.documentsrZlangchain_core.embeddingsr Zlangchain_core.vectorstoresr Z&langchain_community.vectorstores.utilsr getLoggerrloggerr r.r.r.r/s