o Zh]@sxddlZddlmZmZmZmZmZmZddlZ ddl m Z ddl m Z ddlmZddlmZdZGdd d eZdS) N)AnyDictIterableListOptionalTuple)Document) Embeddings) VectorStore)maximal_marginal_relevancec@seZdZdZdededdfddZd?dd Zede efd d Z d@d e e d e e edede e fddZ d@d e e d e e edede e fddZ d@de e e dede efddZ d@de e e dede efddZefddde edede ee e fdede eeeeff ddZefddde dede ee e fdede eeeff ddZefddde dede ee e fdede eeeff d d!Zefddde dede ee e fdede eeeff d"d#Zefddde dede ee e fdede eeeff d$d%Zefddde edede ee e fdede ef d&d'Zefddde edede ee e fdede ef d(d)Z efddde dede ee e fdede ef d*d+Z!efddde dede ee e fdede ef d,d-Z"ed.d/fddde eded0ed1ede ee e fdede efd2d3Z#ed.d/fddde eded0ed1ede ee e fdede efd4d5Z$ 6 . /dAddde ded0ed1ede ee e fdede efd7d8Z%ed.d/fddde ded0ed1ede ee e fdede efd9d:Z&e' d@d e e ded e e ededdf d;d<Z(e' d@d e e ded e e ededdf d=d>Z)dS)BSurrealDBStorea/ SurrealDB as Vector Store. To use, you should have the ``surrealdb`` python package installed. Args: embedding_function: Embedding function to use. dburl: SurrealDB connection url ns: surrealdb namespace for the vector store. (default: "langchain") db: surrealdb database for the vector store. (default: "database") collection: surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Example: .. code-block:: python from langchain_community.vectorstores.surrealdb import SurrealDBStore from langchain_community.embeddings import HuggingFaceEmbeddings model_name = "sentence-transformers/all-mpnet-base-v2" embedding_function = HuggingFaceEmbeddings(model_name=model_name) dburl = "ws://localhost:8000/rpc" ns = "langchain" db = "docstore" collection = "documents" db_user = "root" db_pass = "root" sdb = SurrealDBStore.from_texts( texts=texts, embedding=embedding_function, dburl, ns, db, collection, db_user=db_user, db_pass=db_pass) embedding_functionkwargsreturnNc Kszddlm}Wnty}ztd|d}~ww|dd|_|jdddkr1||j|_ntd|d d |_|d d |_|d d|_ ||_ ||_ dS)Nr)SurrealzZCannot import from surrealdb. please install with `pip install surrealdb`.dburlzws://localhost:8000/rpcwsz6Only websocket connections are supported at this time.nsZ langchaindbZdatabase collectionZ documents) Z surrealdbr ImportErrorpoprsdb ValueErrorrrrrr)selfrrrera/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/vectorstores/surrealdb.py__init__5s& zSurrealDBStore.__init__csr|jIdHd|jvr+d|jvr+|jd}|jd}|j||dIdH|j|j|jIdHdS)zr Initialize connection to surrealdb database and authenticate if credentials are provided NZdb_userZdb_pass)userpass)rconnectrgetZsigninZuserr)rr!passwordrrr initializeOs  zSurrealDBStore.initializecCst|jtr |jSdSN) isinstancerr rrrr embeddings[s zSurrealDBStore.embeddingstexts metadatasc s|jt|}g}t|D]4\}}|||d}|dur+|t|kr+|||d<ng|d<|j|j|IdH} || ddq|S)zAdd list of text along with embeddings to the vector store asynchronously Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents )text embeddingNmetadatarid) rZembed_documentslist enumeratelenrcreaterappend) rr+r,rr*idsidxr-datarecordrrr aadd_textscs zSurrealDBStore.aadd_textsc sJ ddttdtttdtdttffdd }t|||fi|S) zAdd list of text along with embeddings to the vector store Args: texts (Iterable[str]): collection of text to add to the database Returns: List of ids for the newly inserted documents Nr+r,rrcs*IdHj||fi|IdHSr'r&r:)r+r,rr)rr _add_textssz,SurrealDBStore.add_texts.._add_textsr')rstrrrdictrasynciorun)rr+r,rr<rr)r add_textss zSurrealDBStore.add_textsr6csx|durjjIdHdSt|tr!j|IdHdSt|tr:t|dkr:fdd|DIdH}dSdS)aDelete by document ID asynchronously. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. NTrcs g|] }j|IdHqSr')rdelete).0r0r)rr sz*SurrealDBStore.adelete..F)rrBrr(r=r1r3)rr6r_rr)radeletes zSurrealDBStore.adeletec s<dtttdtdttffdd }t||fi|S)a Delete by document ID. Args: ids: List of ids to delete. **kwargs: Other keyword arguments that subclasses might use. Returns: Optional[bool]: True if deletion is successful, False otherwise. r6rrcs*IdHjdd|i|IdHS)Nr6r)r&rF)r6rr)rr_deletesz&SurrealDBStore.delete.._delete)rrr=rboolr?r@)rr6rrGrr)rrBs&zSurrealDBStore.delete)filterr.krIcs|j|||ddd}d}|r9|D]%}t||ttfvr(d||d}n||}|d|d|d7}qd |d d |d } |j| |Id H} t| dkrVgS| d} | ddkrpddlm } | dd} | | dd| dDS)aRun similarity search for query embedding asynchronously and return documents and scores Args: embedding (List[float]): Query embedding. k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with scores score_thresholdr)rr.rJrK'z and metadata.z =  u select id, text, metadata, embedding, vector::similarity::cosine(embedding, $embedding) as similarity from ⟨rub⟩ where vector::similarity::cosine(embedding, $embedding) >= $score_threshold z4 order by similarity desc LIMIT $k; NstatusOK)SurrealExceptionresultz Unknown ErrorcSs@g|]}t|dd|di|dpid|d|dfqS)r-r0r/)Z page_contentr/ similarityr.)rr$)rCdocrrrrD s zKSurrealDBStore._asimilarity_search_by_vector_with_score..) rr$typer=rHrqueryr3Z surrealdb.wsrQ)rr.rJrIrargsZ custom_filterkeyZ filter_valuerVresultsrRrQerrrrr(_asimilarity_search_by_vector_with_scores:        z7SurrealDBStore._asimilarity_search_by_vector_with_scorerVc6|j|}dd|j||fd|i|IdHDS)afRun similarity search asynchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores cSg|] \}}}||fqSrrrCdocumentrSrErrrrD*zKSurrealDBStore.asimilarity_search_with_relevance_scores..rINr embed_queryr[rrVrJrIrZquery_embeddingrrr(asimilarity_search_with_relevance_scores  z7SurrealDBStore.asimilarity_search_with_relevance_scoresc 2dttttfffdd }t|S)aeRun similarity search synchronously and return relevance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance scores rc.IdHjfdiIdHSNrI)r&rdrrIrJrrVrrr(_similarity_search_with_relevance_scoresFs zhSurrealDBStore.similarity_search_with_relevance_scores.._similarity_search_with_relevance_scoresrrrfloatr?r@)rrVrJrIrrjrrir'similarity_search_with_relevance_scores3s  z6SurrealDBStore.similarity_search_with_relevance_scorescr\)anRun similarity search asynchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores cSr]rrr^rrrrDcr`z@SurrealDBStore.asimilarity_search_with_score..rINrarcrrrasimilarity_search_with_scorePrez,SurrealDBStore.asimilarity_search_with_scorec rf)amRun similarity search synchronously and return distance scores Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar along with relevance distance scores rcrgrh)r&rnrrirr_similarity_search_with_score zRSurrealDBStore.similarity_search_with_score.._similarity_search_with_scorerk)rrVrJrIrrorrirsimilarity_search_with_scorels& z+SurrealDBStore.similarity_search_with_scorecs*dd|j||fd|i|IdHDS)adRun similarity search on query embedding asynchronously Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query cSsg|]\}}}|qSrr)rCr_rErrrrDsz?SurrealDBStore.asimilarity_search_by_vector..rIN)r[)rr.rJrIrrrrasimilarity_search_by_vectors z+SurrealDBStore.asimilarity_search_by_vectorc *dttffdd }t|S)aURun similarity search on query embedding Args: embedding (List[float]): Query embedding k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rcs.IdHjfdiIdHSrh)r&rrrr.rIrJrrrr_similarity_search_by_vectorrpzPSurrealDBStore.similarity_search_by_vector.._similarity_search_by_vectorrrr?r@)rr.rJrIrrurrtrsimilarity_search_by_vectors z*SurrealDBStore.similarity_search_by_vectorcs,|j|}|j||fd|i|IdHS)aDRun similarity search on query asynchronously Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rIN)rrbrrrcrrrasimilarity_searchs  z!SurrealDBStore.asimilarity_searchc rs)a5Run similarity search on query Args: query (str): Query k (int): Number of results to return. Defaults to 4. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents most similar to the query rcrgrh)r&rxrrirr_similarity_searchsz._similarity_searchrv)rrVrJrIrryrrirsimilarity_searchs z SurrealDBStore.similarity_search?fetch_k lambda_multc sj|j||fd|i|IdH}dd|Ddd|D}ttj|tjd|||d} fdd| DS) aHReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rINcSg|]}|dqS)rrrCsubrrrrD zKSurrealDBStore.amax_marginal_relevance_search_by_vector..cSr)rrrrrrDr)Zdtype)rJr~csg|]}|qSrr)rCidocsrrrDr)r[r nparrayZfloat32) rr.rJr}r~rIrrRr*Z mmr_selectedrrr(amax_marginal_relevance_search_by_vectors" z7SurrealDBStore.amax_marginal_relevance_search_by_vectorc .dttffdd }t|S)aIReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rcs2IdHjfdiIdHSrh)r&rrr.r}rIrJrr~rrr(_max_marginal_relevance_search_by_vector6 zhSurrealDBStore.max_marginal_relevance_search_by_vector.._max_marginal_relevance_search_by_vectorrv)rr.rJr}r~rIrrrrr'max_marginal_relevance_search_by_vectors" z6SurrealDBStore.max_marginal_relevance_search_by_vectorr c s4|j|}|j||||fd|i|IdH}|S)a@Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rIN)rrbr) rrVrJr}r~rIrr.rrrramax_marginal_relevance_search>s  z-SurrealDBStore.amax_marginal_relevance_searchc r)a?Return docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. filter (Optional[Dict[str, str]]): Filter by metadata. Defaults to None. Returns: List of Documents selected by maximal marginal relevance. rcs2IdHjfdiIdHSrh)r&rrr}rIrJrr~rVrrr_max_marginal_relevance_search}rzTSurrealDBStore.max_marginal_relevance_search.._max_marginal_relevance_searchrv)rrVrJr}r~rIrrrrrmax_marginal_relevance_searchas" z,SurrealDBStore.max_marginal_relevance_searchcs>||fi|}|IdH|j||fi|IdH|S)aCreate SurrealDBStore from list of text asynchronously Args: texts (List[str]): list of text to vectorize and store embedding (Optional[Embeddings]): Embedding function. dburl (str): SurrealDB connection url (default: "ws://localhost:8000/rpc") ns (str): surrealdb namespace for the vector store. (default: "langchain") db (str): surrealdb database for the vector store. (default: "database") collection (str): surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Returns: SurrealDBStore object initialized and ready for use.Nr;clsr+r.r,rrrrr afrom_textss zSurrealDBStore.afrom_textscKs t|j|||fi|}|S)aCreate SurrealDBStore from list of text Args: texts (List[str]): list of text to vectorize and store embedding (Optional[Embeddings]): Embedding function. dburl (str): SurrealDB connection url ns (str): surrealdb namespace for the vector store. (default: "langchain") db (str): surrealdb database for the vector store. (default: "database") collection (str): surrealdb collection for the vector store. (default: "documents") (optional) db_user and db_pass: surrealdb credentials Returns: SurrealDBStore object initialized and ready for use.)r?r@rrrrr from_textsszSurrealDBStore.from_texts)rNr')r r{r|)*__name__ __module__ __qualname____doc__r rr r&propertyrr*rr=rr>r:rArHrFrB DEFAULT_Krlintrrrr[rdrmrnrqrrrwrxrzrrrr classmethodrrrrrrr s0&             N          1 ( & $   r )r?typingrrrrrrnumpyrZlangchain_core.documentsrZlangchain_core.embeddingsr Zlangchain_core.vectorstoresr Z&langchain_community.vectorstores.utilsr rr rrrrs