o Zhm/@sddlmZddlZddlZddlmZmZmZmZm Z ddl Z ddl m Z ddlmZddlmZeeZGdddeZdS) ) annotationsN)AnyIterableListOptionalTypeDocument) Embeddings) VectorStorec@seZdZUdZdZded<     d8d9ddZed:ddZ   d;dd?d+d,Z e         d@dAd2d3Z e         d@dBd6d7ZdS)CAtlasDBa`Atlas` vector store. Atlas is the `Nomic's` neural database and `rhizomatic` instrument. To use, you should have the ``nomic`` python package installed. Example: .. code-block:: python from langchain_community.vectorstores import AtlasDB from langchain_community.embeddings.openai import OpenAIEmbeddings embeddings = OpenAIEmbeddings() vectorstore = AtlasDB("my_project", embeddings.embed_query) Zatlas_idstr_ATLAS_DEFAULT_ID_FIELDNA description for your projectTFnameembedding_functionOptional[Embeddings]api_key Optional[str] description is_publicboolreset_project_if_existsreturnNonec Csz ddl}ddlm}Wn tytdw|dur td||||_d} |jdur1d} |||| ||tjd|_|j dS) a  Initialize the Atlas Client Args: name (str): The name of your project. If the project already exists, it will be loaded. embedding_function (Optional[Embeddings]): An optional function used for embedding your data. If None, data will be embedded with Nomic's embed model. api_key (str): Your nomic API key description (str): A description for your project. is_public (bool): Whether your project is publicly accessible. True by default. reset_project_if_exists (bool): Whether to reset this project if it already exists. Default False. Generally useful during development and testing. rN) AtlasProjectzRCould not import nomic python package. Please install it with `pip install nomic`.z/No API key provided. Sign up at atlas.nomic.ai!text embedding)rrmodalityrrZunique_id_field) nomicr ImportError ValueErrorlogin_embedding_functionr rprojectZ_latest_project_state) selfrrrrrrrrrr&]/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/vectorstores/atlas.py__init__"s0   zAtlasDB.__init__cCs|jS)N)r#)r%r&r&r' embeddingsYszAtlasDB.embeddingstexts Iterable[str] metadatasOptional[List[dict]]idsOptional[List[str]]refreshkwargsr List[str]c  s|durt|dkrd|dvrtdtdur%ddD|jdur|j}t|}|durFfddtD}nt t|D]} | || t j <| || d<qL|}|j j gt j |d|j |j j||d Wdn1swYnO|durfd dtD}ntD]\} } || d<| || t j <q|}|j j gt j |d|j |j |Wdn1swY|rt|j jdkr|j |j WdS1swYS) aRun more texts through the embeddings and add to the vectorstore. Args: texts (Iterable[str]): Texts to add to the vectorstore. metadatas (Optional[List[dict]], optional): Optional list of metadatas. ids (Optional[List[str]]): An optional list of ids. refresh(bool): Whether or not to refresh indices with the updated data. Default True. Returns: List[str]: List of IDs of the added texts. Nrrz#Cannot accept key text in metadata!cSsg|]}ttqSr&)r uuiduuid4).0_r&r&r' zsz%AtlasDB.add_texts..cs&g|]\}}tj|d|iqSrr r)r5ir6r.r*r&r'r7s)Zid_fielddata)r)r<cs"g|] \}}d|tj|iqSr8r9)r5r:rr.r&r'r7s)lenkeysr!listr#embed_documentsnpstack enumerateranger rr$Z_validate_map_data_inputswait_for_project_lockZadd_embeddingsadd_textindicesZ rebuild_maps) r%r*r,r.r0r1Z _embeddingsr)r<r:rr&r;r' add_texts]s^            zAtlasDB.add_textscKs@|j|jjdi|WdS1swYdS)zCreates an index in your project. See https://docs.nomic.ai/atlas_api.html#nomic.project.AtlasProject.create_index for full detail. Nr&)r$rF create_index)r%r1r&r&r'rJs $zAtlasDB.create_indexquerykintList[Document]c  s|jdur td|j|gd}t|dd}|j|jjdj ||d\}}|jj |ddWdn1sAwYfdd t |D}|S) aRun similarity search with AtlasDB Args: query (str): Query text to search for. k (int): Number of results to return. Defaults to 4. Returns: List[Document]: List of documents most similar to the query text. NzBAtlasDB requires an embedding_function for text similarity search!r)ZqueriesrMr=cs(g|]\}}t|d|dqS)r) page_contentmetadatar)r5r:Zneighborr<r&r'r7sz-AtlasDB.similarity_search..) r#NotImplementedErrorrArBarrayZreshaper$rFZ projectionsZ vector_searchget_datarD) r%rLrMr1Z _embeddingrZ neighborsr6docsr&rTr'similarity_searchs     zAtlasDB.similarity_searchcls Type[AtlasDB]r index_kwargsOptional[dict]c Ks|dus|dur td|ddd} | dur$| D]\} }|| | <q||||d|| d}|j|j|||d|jd i| Wd|S1sOwY|S) a^Create an AtlasDB vectorstore from a raw documents. Args: texts (List[str]): The list of texts to ingest. name (str): Name of the project to create. api_key (str): Your nomic API key, embedding (Optional[Embeddings]): Embedding function. Defaults to None. metadatas (Optional[List[dict]]): List of metadatas. Defaults to None. ids (Optional[List[str]]): Optional list of document IDs. If None, ids will be auto created description (str): A description for your project. is_public (bool): Whether your project is publicly accessible. True by default. reset_project_if_exists (bool): Whether to reset this project if it already exists. Default False. Generally useful during development and testing. index_kwargs (Optional[dict]): Dict of kwargs for index creation. See https://docs.nomic.ai/atlas_api.html Returns: AtlasDB: Nomic's neural database and finest rhizomatic instrument N$`name` and `api_key` cannot be None._indexr)rZ indexed_fieldr)rrrrr)r*r,r.r&)r!itemsr$rFrIrJ)rZr*rr,r.rrrrrr\r1Zall_index_kwargsrMvZatlasDBr&r&r' from_textss*%   zAtlasDB.from_texts documentspersist_directoryc  KsR|dus|dur tddd|D} dd|D} |j||| || |||| | d S)a+Create an AtlasDB vectorstore from a list of documents. Args: name (str): Name of the collection to create. api_key (str): Your nomic API key, documents (List[Document]): List of documents to add to the vectorstore. embedding (Optional[Embeddings]): Embedding function. Defaults to None. ids (Optional[List[str]]): Optional list of document IDs. If None, ids will be auto created description (str): A description for your project. is_public (bool): Whether your project is publicly accessible. True by default. reset_project_if_exists (bool): Whether to reset this project if it already exists. Default False. Generally useful during development and testing. index_kwargs (Optional[dict]): Dict of kwargs for index creation. See https://docs.nomic.ai/atlas_api.html Returns: AtlasDB: Nomic's neural database and finest rhizomatic instrument Nr^cSg|]}|jqSr&)rRr5docr&r&r'r79z*AtlasDB.from_documents..cSrer&)rSrfr&r&r'r7:rh) rrr*rr,r.rrrr\)r!rb)rZrcrr.rrrdrrrr\r1r*r,r&r&r'from_documentss $zAtlasDB.from_documents)NNrTF)rr rrrrrr rrrrrr)rr)NNT) r*r+r,r-r.r/r0rr1rrr2)r1rrr)rK)rLr rMrNr1rrrO) NNNNNrTFN)rZr[r*r2rrr,r-r.r/rrrrrr rrrrr\r]r1rrr )rZr[rcrOrrr.r/rrrrrdrrr rrrrr\r]r1rrr )__name__ __module__ __qualname____doc__r__annotations__r(propertyr)rIrJrY classmethodrbrir&r&r&r'r sP   7  N ";r ) __future__rloggingr3typingrrrrrnumpyrBZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.vectorstoresr getLoggerrjloggerr r&r&r&r's