o ZhR@sddlmZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z ddlZddlmZddlmZddlmZddlmZerFddlZeZdZGd d d eZdS) ) annotationsN) TYPE_CHECKINGAnyDictIterableListOptionalSetTupleType)Document) Embeddings) VectorStore)maximal_marginal_relevancec@sZeZdZUdZdZded<edddfdZddZed[ddZ  d\d]ddZ d^dd Z e ddfd_d(d)Z e ddfd`d+d,Z -dadbd.d/Zde dddfdcd3d4Z - 5 6  ddded:d;Z - 5 6  dddfd=d>Z     dgdhdEdFZ didjdGdHZ didkdIdJZd^dKdLZd^dMdNZdldOdPZdmdQdRZeddeddfdndUdVZededdfdodXdYZdS)pAwaDBz`AwaDB` vector store.Zlangchain_awadbstr_DEFAULT_TABLE_NAMEN table_name embeddingOptional[Embeddings]log_and_data_dir Optional[str]clientOptional[awadb.Client]kwargsrreturnNonecKszddl}Wn tytdw|dur||_n|dur%|||_n||_||jkr@|d7}|ttdd7}|j |i|_ |durR||j |<||_ dS)a@Initialize with AwaDB client. If table_name is not specified, a random table name of `_DEFAULT_TABLE_NAME + last segment of uuid` would be created automatically. Args: table_name: Name of the table created, default _DEFAULT_TABLE_NAME. embedding: Optional Embeddings initially set. log_and_data_dir: Optional the root directory of log and data. client: Optional AwaDB client. kwargs: Any possible extend parameters in the future. Returns: None. rNzRCould not import awadb python package. Please install it with `pip install awadb`._-) awadb ImportError awadb_clientZClientrruuiduuid4splitCreatetable2embeddingsusing_table_name)selfrrrrrr!r+]/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/vectorstores/awadb.py__init__s(       zAwaDB.__init__cCs|j|jvr |j|jSdSN)r)r()r*r+r+r, embeddingsKs  zAwaDB.embeddingstexts Iterable[str] metadatasOptional[List[dict]]is_duplicate_textsOptional[bool] List[str]cKsN|jdur tdd}|j|jvr|j|jt|}|jdd||||S)aRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadatas associated with the texts. is_duplicate_texts: Optional whether to duplicate texts. Defaults to True. kwargs: any possible extend parameters in the future. Returns: List of ids from adding the texts into the vectorstore. NAwaDB client is None!!!embedding_texttext_embedding)r# ValueErrorr)r(Zembed_documentslistZAddTexts)r*r0r2r4rr/r+r+r, add_textsQs   zAwaDB.add_textsboolcKs|jdur td|j|S)zLoad the local specified table. Args: table_name: Table name kwargs: Any possible extend parameters in the future. Returns: Success or failure of loading the local specified table Nr7)r#r:Load)r*rrr+r+r, load_localts  zAwaDB.load_localquerykinttext_in_page_content meta_filterOptional[dict]List[Document]c Ksj|jdur tdd}|j|jvr|j|j|}n ddlm}||}hd}|j|||||dS)a8Return docs most similar to query. Args: query: Text query. k: The maximum number of documents to return. text_in_page_content: Filter by the text in page_content of Document. meta_filter (Optional[dict]): Filter by metadata. Defaults to None. E.g. `{"color" : "red", "price": 4.20}`. Optional. E.g. `{"max_price" : 15.66, "min_price": 4.20}` `price` is the metadata field, means range filter(4.20<'price'<15.66). E.g. `{"maxe_price" : 15.66, "mine_price": 4.20}` `price` is the metadata field, means range filter(4.20<='price'<=15.66). kwargs: Any possible extend parameters in the future. Returns: Returns the k most similar documents to the specified text query. Nr7r AwaEmbedding>scorer9_idrCrDnot_include_fields_in_metadata) r#r:r)r( embed_queryr!rH Embeddingsimilarity_search_by_vector) r*r@rArCrDrrrHnot_include_fieldsr+r+r,similarity_searchs    zAwaDB.similarity_searchList[Tuple[Document, float]]cKs|jdur tdd}|j|jvr|j|j|}n ddlm}||}g}ddh} |j||||| d} | D]} | j d} | j d=| | f} | | q9|S) aThe most k similar documents and scores of the specified query. Args: query: Text query. k: The k most similar documents to the text query. text_in_page_content: Filter by the text in page_content of Document. meta_filter: Filter by metadata. Defaults to None. kwargs: Any possible extend parameters in the future. Returns: The k most similar documents to the specified text query. 0 is dissimilar, 1 is the most similar. Nr7rrGr9rJrKrI) r#r:r)r(rMr!rHrNrOmetadataappend)r*r@rArCrDrrrHresultsrPZretrieval_docsdocrIZ doc_tupler+r+r,similarity_search_with_scores,      z"AwaDB.similarity_search_with_scorercKs|j||fi|Sr.)rW)r*r@rArr+r+r,(_similarity_search_with_relevance_scoressz.AwaDB._similarity_search_with_relevance_scoresOptional[List[float]]rLOptional[Set[str]]c Ks|jdur tdg}|dur|S|jj|||||d}|dkr$|S|ddD],} d} i} | D]} | dkr=| | } q2|durF| |vrFq2| | | | <q2|t| | dq*|S) a Return docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. text_in_page_content: Filter by the text in page_content of Document. meta_filter: Filter by metadata. Defaults to None. not_incude_fields_in_metadata: Not include meta fields of each document. Returns: List of Documents which are the most similar to the query vector. Nr7)rCrDrPrZ ResultItemsr8 page_contentrS)r#r:ZSearch__len__rTr ) r*rrArCrDrLrrUZ show_resultsZ item_detailcontentZ meta_dataZitem_keyr+r+r,rOs6  z!AwaDB.similarity_search_by_vector?fetch_k lambda_multfloatc Ksx|jdur tdg}|j|jvr|j|j|}n ddlm} | |}|dkr/gS|j ||||||d} | S)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. text_in_page_content: Filter by the text in page_content of Document. meta_filter (Optional[dict]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. Nr7rrG)rcrCrD) r#r:r)r(rMr!rHrNr^'max_marginal_relevance_search_by_vector) r*r@rArbrcrCrDrrrHrUr+r+r,max_marginal_relevance_search)s$     z#AwaDB.max_marginal_relevance_search List[float]cKs|jdur tdg}|dur|Sddh} |j||||| d} g} | D] } | | jdq#ttj|tjd| d} | D]}d| |jvrR| |jd=|| |q<|S) aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. text_in_page_content: Filter by the text in page_content of Document. meta_filter (Optional[dict]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. Nr7rJrIrKr9)Zdtype)Zembedding_list) r#r:rOrTrSrnparrayZfloat32)r*rrArbrcrCrDrrUrPZretrieved_docsZtop_embeddingsrVZ selected_docsZs_idr+r+r,re^s2  z-AwaDB.max_marginal_relevance_search_by_vectoridsOptional[List[str]]rPlimit Optional[int]Dict[str, Document]cKs|jdur td|jj|||||d}i}|D]/} d} i} | D]} | dkr+| | } q | dks3| dkr4q | | | | <q t| | d} | || d<q|S) aReturn docs according ids. Args: ids: The ids of the embedding vectors. text_in_page_content: Filter by the text in page_content of Document. meta_filter: Filter by any metadata of the document. not_include_fields: Not pack the specified fields of each document. limit: The number of documents to return. Defaults to 5. Optional. Returns: Documents which satisfy the input conditions. Nr7)rjrCrDrPrlr[r8r9rJr\)r#r:ZGetr )r*rjrCrDrPrlrZ docs_detailrUZ doc_detailr_Z meta_infofieldrVr+r+r,gets.  z AwaDB.getcKs>|jdur tdd}|dus|dkr|S|j|}|S)aJDelete the documents which have the specified ids. Args: ids: The ids of the embedding vectors. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful. False otherwise, None if not implemented. Nr7r)r#r:r^Delete)r*rjrretr+r+r,deletes  z AwaDB.deletecKs&|jdur td|jj|d||dS)a@Update the documents which have the specified ids. Args: ids: The id list of the updating embedding vector. texts: The texts of the updating documents. metadatas: The metadatas of the updating documents. Returns: the ids of the updated documents. Nr7r8)rjZtext_field_namer0r2)r#r:Z UpdateTexts)r*rjr0r2rr+r+r,updates z AwaDB.updatecK(|jdurdS|j|}|r||_|S)zCreate a new table.NF)r#r'r)r*rrrrr+r+r, create_tables  zAwaDB.create_tablecKru)zJUse the specified table. Don't know the tables, please invoke list_tables.NF)r#ZUser)rvr+r+r,use s  z AwaDB.usecKs|jdurgS|jS)z*List all the tables created by the client.N)r#Z ListAllTablesr*rr+r+r, list_tabless  zAwaDB.list_tablescKs|jS)zGet the current table.)r)ryr+r+r,get_current_table(szAwaDB.get_current_tablecls Type[AwaDB]c Ks"|||||d}|j||d|S)a3Create an AwaDB vectorstore from a raw documents. Args: texts (List[str]): List of texts to add to the table. embedding (Optional[Embeddings]): Embedding function. Defaults to None. metadatas (Optional[List[dict]]): List of metadatas. Defaults to None. table_name (str): Name of the table to create. log_and_data_dir (Optional[str]): Directory of logging and persistence. client (Optional[awadb.Client]): AwaDB client Returns: AwaDB: AwaDB vectorstore. )rrrr)r0r2)r<) r|r0rr2rrrrr#r+r+r, from_texts0szAwaDB.from_texts documentsc Ks2dd|D}dd|D}|j||||||dS)avCreate an AwaDB vectorstore from a list of documents. If a log_and_data_dir specified, the table will be persisted there. Args: documents (List[Document]): List of documents to add to the vectorstore. embedding (Optional[Embeddings]): Embedding function. Defaults to None. table_name (str): Name of the table to create. log_and_data_dir (Optional[str]): Directory to persist the table. client (Optional[awadb.Client]): AwaDB client. Any: Any possible parameters in the future Returns: AwaDB: AwaDB vectorstore. cSg|]}|jqSr+)r].0rVr+r+r, jz(AwaDB.from_documents..cSrr+)rSrr+r+r,rkr)r0rr2rrr)r~) r|rrrrrrr0r2r+r+r,from_documentsQszAwaDB.from_documents) rrrrrrrrrrrr)rr)NN) r0r1r2r3r4r5rrrr6)rrrrrr=) r@rrArBrCrrDrErrrrF) r@rrArBrCrrDrErrrrR)r)r@rrArBrrrrR)rrYrArBrCrrDrErLrZrrrrF)rr`raNN)r@rrArBrbrBrcrdrCrrDrErrrrF)rrgrArBrbrBrcrdrCrrDrErrrrF)NNNNN)rjrkrCrrDrErPrZrlrmrrrrnr.)rjrkrrrr5) rjr6r0r1r2r3rrrr6)rrrr6)rrrr)r|r}r0r6rrr2r3rrrrrrrrrr)r|r}rrFrrrrrrrrrrrr)__name__ __module__ __qualname____doc__r__annotations__r-propertyr/r<r? DEFAULT_TOPNrQrWrXrOrfrerprsrtrwrxrzr{ classmethodr~rr+r+r+r,rs   1  # 1 7   : 8 @ 4      r) __future__rloggingr$typingrrrrrrr r r numpyrhZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.vectorstoresrZ&langchain_community.vectorstores.utilsrr! getLoggerloggerrrr+r+r+r,s ,