o ZhR@sddlmZddlZddlZddlZddlmZmZmZm Z m Z m Z m Z m Z mZddlZddlmZddlmZddlmZddlmZmZerVddlmZdd lmZmZee Z!dd d Z"e dddZ#GdddeZ$dS)) annotationsN) TYPE_CHECKINGAnyCallableIterableListOptionalTupleTypeVarUnion)Document) Embeddings) VectorStore)DistanceStrategymaximal_marginal_relevanceClient)NeighborVectorDistanceMetricreturnrc Cs6z ddlm}W|Sty}ztd|d}~ww)NrrzoCould not import aerospike_vector_search python package. Please install it with `pip install aerospike_vector`.)aerospike_vector_searchr ImportError)rera/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/vectorstores/aerospike.py_import_aerospike#srAVST Aerospike)boundc@s,eZdZdZdddddejfd]ddZed^ddZd_ddZ d`d!d"Z e dad%d&Z    '  (dbdcd4d5Z  ddded7d8Z 9  dfdgd>d?Z 9  dfdhd@dAZ 9  dfdidCdDZ 9  dfdjdEdFZdkdHdIZe dldLdMZ 9 N O  dmdndRdSZ 9 N O  dmdodTdUZe   V   ' dpdqd[d\ZdS)rrzu`Aerospike` vector store. To use, you should have the ``aerospike_vector_search`` python package installed. NZ_vector_textZ_idclientr embeddingUnion[Embeddings, Callable] namespacestr index_name Optional[str] vector_keytext_keyid_keyset_namedistance_strategy7Optional[Union[DistanceStrategy, VectorDistanceMetric]]c Csvt} t|ts tdt|| stdt|||_||_||_ ||_ ||_ ||_ ||_ ||_|| |_dS)aInitialize with Aerospike client. Args: client: Aerospike client. embedding: Embeddings object or Callable (deprecated) to embed text. namespace: Namespace to use for storing vectors. This should match index_name: Name of the index previously created in Aerospike. This vector_key: Key to use for vector in metadata. This should match the key used during index creation. text_key: Key to use for text in metadata. id_key: Key to use for id in metadata. set_name: Default set name to use for storing vectors. distance_strategy: Distance strategy to use for similarity search This should match the distance strategy used during index creation. z`Passing in `embedding` as a Callable is deprecated. Please pass in an Embeddings object instead.zDclient should be an instance of aerospike_vector_search.Client, got N)r isinstancer warningswarn ValueErrortype_client _embedding _text_key _vector_key_id_key _index_name _namespace _set_nameconvert_distance_strategy_distance_strategy) selfr r!r#r%r'r(r)r*r+ aerospikerrr__init__7s(  zAerospike.__init__rOptional[Embeddings]cCst|jtr |jSdS)z/Access the query embedding object if available.N)r-r3r r<rrr embeddingsms zAerospike.embeddingstexts Iterable[str]List[List[float]]cs.tjtrjt|Sfdd|DS)zEmbed search docs.csg|]}|qSr)r3).0tr@rr xsz.Aerospike._embed_documents..)r-r3r Zembed_documentslist)r<rBrr@r_embed_documentsts zAerospike._embed_documentstext List[float]cCs"t|jtr |j|S||S)zEmbed query text.)r-r3r Z embed_query)r<rJrrr _embed_queryzs   zAerospike._embed_query-Union[VectorDistanceMetric, DistanceStrategy]rcCsRddlm}t|tr |S||jkrtjS||jkrtjS||jkr%tjStd)z Convert Aerospikes distance strategy to langchains DistanceStrategy enum. This is a convenience method to allow users to pass in the same distance metric used to create the index. r)rDUnknown distance strategy, must be cosine, dot_product, or euclidean) aerospike_vector_search.typesrr-rCOSINE DOT_PRODUCTZSQUARED_EUCLIDEANEUCLIDEAN_DISTANCEr0)r+rrrrr:s    z#Aerospike.convert_distance_strategyT metadatasOptional[List[dict]]idsOptional[List[str]]embedding_chunk_sizeintwait_for_indexboolkwargsr List[str]c KsB|dur|j}|dur|j}|r|durtdt|}|p$dd|D}|r/dd|D}n |p7dd|D}tdt||D]S} || | |} || | |} || | |} || } t| | | D]\}}}|||j<|||j <qet| | D]\}}|||j <|j j d |j |||d|qzq@|r|j j|j |d |S) aRun more texts through the embeddings and add to the vectorstore. Args: texts: Iterable of strings to add to the vectorstore. metadatas: Optional list of metadata associated with the texts. ids: Optional list of ids to associate with the texts. set_name: Optional aerospike set name to add the texts to. batch_size: Batch size to use when adding the texts to the vectorstore. embedding_chunk_size: Chunk size to use when embedding the texts. index_name: Optional aerospike index name used for waiting for index completion. If not provided, the default index_name will be used. wait_for_index: If True, wait for the all the texts to be indexed before returning. Requires index_name to be provided. Defaults to True. kwargs: Additional keyword arguments to pass to the client upsert call. Returns: List of ids from adding the texts into the vectorstore. Nz6if wait_for_index is True, index_name must be providedcSsg|]}ttqSr)r$uuiduuid4rE_rrrrGz'Aerospike.add_texts..cSsg|]}|qSr)copy)rEmrrrrGcSsg|]}iqSrrr`rrrrGsr)r#keyr*Z record_data)r#namer)r9r7r0rHrangelenrIzipr5r4r6r2Zupsertr8Zwait_for_index_completion)r<rBrTrVr*rXr%rZr\iZ chunk_textsZ chunk_idsZchunk_metadatasrAmetadatar!rJidrrr add_textssL       zAerospike.add_textsOptional[bool]c KsTddlm}|r(|D]}z|jjd|j||d|Wq |y'YdSwdS)a7Delete by vector ID or other criteria. Args: ids: List of ids to delete. **kwargs: Other keyword arguments to pass to client delete call. Returns: Optional[bool]: True if deletion is successful, False otherwise, None if not implemented. r)AVSServerError)r#rfr*FTNr)rrpr2deleter8)r<rVr*r\rprmrrrrqs  zAerospike.deletequeryk metadata_keysList[Tuple[Document, float]]cKs |j||f|||d|S)aReturn aerospike documents most similar to query, along with scores. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Name of the index to search. Overrides the default index_name. kwargs: Additional keyword arguments to pass to the search method. Returns: List of Documents most similar to the query and associated scores. rtrur%)&similarity_search_by_vector_with_scorerL)r<rsrtrur%r\rrrsimilarity_search_with_score sz&Aerospike.similarity_search_with_scorec Ksg}|r|j|vr|jg|}|dur|j}|durtd|jjd||j|||d|}|D])}|j} |j| vrO| |j} |j} | t | | d| fq0t d|jdq0|S)aReturn aerospike documents most similar to embedding, along with scores. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Name of the index to search. Overrides the default index_name. kwargs: Additional keyword arguments to pass to the client vector_search method. Returns: List of Documents most similar to the query and associated scores. Nzindex_name must be provided)r%r#rslimit field_names)Z page_contentrlzFound document with no `z` key. Skipping.r) r4r7r0r2Z vector_searchr8fieldspopZdistanceappendr loggerwarning) r<r!rtrur%r\docsresultsresultrlrJscorerrrrx*s6     z0Aerospike.similarity_search_by_vector_with_scoreList[Document]cKs$dd|j|f|||d|DS)akReturn docs most similar to embedding vector. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Name of the index to search. Overrides the default index_name. kwargs: Additional keyword arguments to pass to the search method. Returns: List of Documents most similar to the query vector. cSg|]\}}|qSrrrEdocrarrrrG}sz9Aerospike.similarity_search_by_vector..rw)rx)r<r!rtrur%r\rrrsimilarity_search_by_vectorfsz%Aerospike.similarity_search_by_vectorcKs(|j|f|||d|}dd|DS)a*Return aerospike documents most similar to query. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Optional name of the index to search. Overrides the default index_name. Returns: List of Documents most similar to the query and score for each rwcSrrrrrrrrGrez/Aerospike.similarity_search..)ry)r<rsrtrur%r\Zdocs_and_scoresrrrsimilarity_searchszAerospike.similarity_searchCallable[[float], float]cCs>|jtjkr |jS|jtjkr|jS|jtjkr|jStd)a The 'correct' relevance function may differ depending on a few things, including: - the distance / similarity metric used by the VectorStore - the scale of your embeddings (OpenAI's are unit normed. Many others are not!) - embedding dimensionality - etc. 0 is dissimilar, 1 is similar. Aerospike's relevance_fn assume euclidean and dot product embeddings are normalized to unit norm. rN) r;rrP_cosine_relevance_score_fnrQZ%_max_inner_product_relevance_score_fnrRZ_euclidean_relevance_score_fnr0r@rrr_select_relevance_score_fns   z$Aerospike._select_relevance_score_fnrfloatcCs d|dS)zgAerospike returns cosine distance scores between [0,2] 0 is dissimilar, 1 is similar. r)rrrrrs z$Aerospike._cosine_relevance_score_fn?fetch_k lambda_multc  s|r j|vr jg|}j|f|||d|ttj|gtjdfddD||d}|rEj|vrE|D] } | jjq9fdd|DS)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: embedding: Embedding to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. metadata_keys: List of metadata keys to return with the documents. If None, all metadata keys will be returned. Defaults to None. index_name: Optional name of the index to search. Overrides the default index_name. Returns: List of Documents selected by maximal marginal relevance. rw)Zdtypecsg|]}|jjqSr)rlr5)rErr@rrrGrbzEAerospike.max_marginal_relevance_search_by_vector..)rtrcsg|]}|qSrr)rErk)rrrrGre)r5rrnparrayZfloat32rlr}) r<r!rtrrrur%r\Z mmr_selectedrkr)rr<r'max_marginal_relevance_search_by_vectors* z1Aerospike.max_marginal_relevance_search_by_vectorc Ks(||}|j||||f||d|S)aReturn docs selected using the maximal marginal relevance. Maximal marginal relevance optimizes for similarity to query AND diversity among selected documents. Args: query: Text to look up documents similar to. k: Number of Documents to return. Defaults to 4. fetch_k: Number of Documents to fetch to pass to MMR algorithm. lambda_mult: Number between 0 and 1 that determines the degree of diversity among the results with 0 corresponding to maximum diversity and 1 to minimum diversity. Defaults to 0.5. index_name: Name of the index to search. Returns: List of Documents selected by maximal marginal relevance. )rur%)rLr) r<rsrtrrrur%r\r!rrrmax_marginal_relevance_searchs z'Aerospike.max_marginal_relevance_searchtestr embeddings_chunk_size client_kwargsOptional[dict]c Ks8||||fi| } | j|f||||d| pi| S)a This is a user friendly interface that: 1. Embeds text. 2. Converts the texts into documents. 3. Adds the documents to a provided Aerospike index This is intended to be a quick way to get started. Example: .. code-block:: python from langchain_community.vectorstores import Aerospike from langchain_openai import OpenAIEmbeddings from aerospike_vector_search import Client, HostPort client = Client(seeds=HostPort(host="localhost", port=5000)) aerospike = Aerospike.from_texts( ["foo", "bar", "baz"], embedder, client, "namespace", index_name="index", vector_key="vector", distance_strategy=MODEL_DISTANCE_CALC, ) )rTrVr%rX)rn) clsrBr!rTr r#r%rVrrr\r=rrr from_textss$(zAerospike.from_texts)r rr!r"r#r$r%r&r'r$r(r$r)r$r*r&r+r,)rr?)rBrCrrD)rJr$rrK)r+rMrr)NNNrSNT)rBrCrTrUrVrWr*r&rXrYr%r&rZr[r\rrr])NN)rVrWr*r&r\rrro)rrNN) rsr$rtrYrurWr%r&r\rrrv) r!rKrtrYrurWr%r&r\rrrv) r!rKrtrYrurWr%r&r\rrr) rsr$rtrYrurWr%r&r\rrr)rr)rrrr)rrrrNN)r!rKrtrYrrYrrrurWr%r&r\rrr)rsr$rtrYrrYrrrurWr%r&r\rrr)NNrNNrSN)rBr]r!r rTrUr rr#r$r%r&rVrWrrYrrr\rrr)__name__ __module__ __qualname____doc__rrRr>propertyrArIrL staticmethodr:rnrqryrxrrrrrr classmethodrrrrrr1s  6     R # " ? %    8 &)rr)% __future__rloggingr^r.typingrrrrrrr r r numpyrZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.vectorstoresrZ&langchain_community.vectorstores.utilsrrrrrOrr getLoggerrrrrrrrrrs" ,