o Zh7@sdZddlmZddlZddlZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZer:ddlmZeZGd d d eZdS) z'Wrapper around Epsilla vector database.) annotationsN) TYPE_CHECKINGAnyIterableListOptionalTypeDocument) Embeddings) VectorStore)vectordbc@seZdZUdZdZded<dZded<dZded<eefdBddZe dCddZ dDddZ dEdDddZ dFdGd!d"Z dHdId'd(Z   )dJdKd1d2Z dLdMd8d9Zeddeeed)fdNd=d>Zedeeed)fdOd@dAZdS)PEpsillaar Wrapper around Epsilla vector database. As a prerequisite, you need to install ``pyepsilla`` package and have a running Epsilla vector database (for example, through our docker image) See the following documentation for how to run an Epsilla vector database: https://epsilla-inc.gitbook.io/epsilladb/quick-start Args: client (Any): Epsilla client to connect to. embeddings (Embeddings): Function used to embed the texts. db_path (Optional[str]): The path where the database will be persisted. Defaults to "/tmp/langchain-epsilla". db_name (Optional[str]): Give a name to the loaded database. Defaults to "langchain_store". Example: .. code-block:: python from langchain_community.vectorstores import Epsilla from pyepsilla import vectordb client = vectordb.Client() embeddings = OpenAIEmbeddings() db_path = "/tmp/vectorstore" db_name = "langchain_store" epsilla = Epsilla(client, embeddings, db_path, db_name) Zlangchain_storestr_LANGCHAIN_DEFAULT_DB_NAMEz/tmp/langchain-epsilla_LANGCHAIN_DEFAULT_DB_PATHZlangchain_collection_LANGCHAIN_DEFAULT_TABLE_NAMEclientr embeddingsr db_path Optional[str]db_namec Cszddl}Wnty}ztd|d}~wwt||jj|jjjfs-tdt |||_ ||_ ||_ t j|_|j j||d|j j|ddS)z%Initialize with necessary components.rNziCould not import pyepsilla python package. Please install pyepsilla package with `pip install pyepsilla`.zbclient should be an instance of pyepsilla.vectordb.Client or pyepsilla.cloud.client.Vectordb, got )rrr) pyepsilla ImportError isinstancer ZClientcloudrZVectordb TypeErrortype_clientZ_db_name _embeddingsrr_collection_nameZload_dbZuse_db)selfrrrrrer$_/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/vectorstores/epsilla.py__init__4s0 zEpsilla.__init__returnOptional[Embeddings]cCs|jSN)r r"r$r$r%rSszEpsilla.embeddingscollection_nameNonecCs ||_dS)z~ Set default collection to use. Args: collection_name (str): The name of the collection. N)r!r"r+r$r$r%use_collectionWs zEpsilla.use_collectioncCs|s|j}|j|dS)z Clear data in a collection. Args: collection_name (Optional[str]): The name of the collection. If not provided, the default collection will be used. N)r!rZ drop_tabler-r$r$r% clear_data`szEpsilla.clear_dataNresponse_fieldsOptional[List[str]] List[dict]cCsT|s|j}|jj||d\}}|dkr&td|dtd|d|dS)aGet the collection. Args: collection_name (Optional[str]): The name of the collection to retrieve data from. If not provided, the default collection will be used. response_fields (Optional[List[str]]): List of field names in the result. If not specified, all available fields will be responded. Returns: A list of the retrieved data. ) table_namer1zFailed to get records: message Error: {}.result)r!rgetloggererror Exceptionformat)r"r+r1 status_coderesponser$r$r%r9ls z Epsilla.getr4list metadatasOptional[list[dict]]c CsF|stdt|d}dddddddd |d g}|durnd d |D}|D]E}|D]>\}} ||vr7q.t| tr?d} n t| trGd} nt| trOd } nt| trWd} ntd|d||| d||q.q(|j j ||d\} } | dkr| dkrt d|ddSt d|d| dtd| ddS)NzEmbeddings list is empty.ridINT)namedataTypetextSTRINGrZ VECTOR_FLOAT)rErF dimensionscSsg|]}|dqS)rEr$).0fieldr$r$r% z.Epsilla._create_collection..FLOATZBOOLzUnsupported data type for .)Z table_fieldsr5iz#Continuing with the existing table zFailed to create collection : r6r7) ValueErrorlenitemsrrintfloatboolappendrZ create_tabler:infor;r<r=) r"r4rrAdimfields field_namesmetadatakeyvalueZd_typer>r?r$r$r%_create_collectionsH        zEpsilla._create_collectionFtexts Iterable[str]Optional[List[dict]]drop_oldOptional[bool]kwargs List[str]c sN|sj}n|_|rjj|dt|}zj|}Wnty0fdd|D}Ynwt|dkr>t dgSj |||ddd|D}g}t |D](\} } | || || d} |d urv||  } | D]\} }|| | <qm| | qSjj||d \}}|d krtd |d |dtd|ddd|DS)a Embed texts and add them to the database. Args: texts (Iterable[str]): The texts to embed. metadatas (Optional[List[dict]]): Metadata dicts attached to each of the texts. Defaults to None. collection_name (Optional[str]): Which collection to use. Defaults to "langchain_collection". If provided, default collection name will be set as well. drop_old (Optional[bool]): Whether to drop the previous collection and create a new one. Defaults to False. Returns: List of ids of the added texts. rcsg|]}j|qSr$)r embed_query)rJxr*r$r%rLz%Epsilla.add_texts..rzNothing to insert, skipping.)r4rrAcSsg|]}ttqSr$)hashuuiduuid4)rJ_r$r$r%rLrirCrGrN)r4recordsr5zFailed to add records to rPr6r7cSsg|]}t|qSr$)r)rJrCr$r$r%rLrM)r!rZdrop_dbr@r Zembed_documentsNotImplementedErrorrRr:debugr_ enumeraterSrWinsertr;r<r=)r"r`rAr+rcrerZidsroindexrCrecordr\r]r^r>r?r$r*r% add_textssL        zEpsilla.add_textsquerykrTList[Document]c s|s|j}|j|}|jj|d||d\}}|dkr/td|ddtd|dgdt t fd d |d S) a Return the documents that are semantically most relevant to the query. Args: query (str): String to query the vectorstore with. k (Optional[int]): Number of documents to return. Defaults to 4. collection_name (Optional[str]): Collection to use. Defaults to "langchain_store" or the one provided before. Returns: List of documents that are semantically most relevant to the query r)r4Z query_field query_vectorlimitr5zSearch failed: r6rOr7rncs tdfddDdS)NrGcsi|] }|vr||qSr$r$)rJr]) exclude_keysitemr$r% sz?Epsilla.similarity_search....) page_contentr\r r~r}rr%s  z+Epsilla.similarity_search..r8) r!r rgrrxr:r;r<r=r@map)r"rxryr+rer{r>r?r$rr%similarity_searchs&   zEpsilla.similarity_searchcls Type[Epsilla] embeddingc Ks.t||||d} | j|f|||d| | S)aCreate an Epsilla vectorstore from raw documents. Args: texts (List[str]): List of text data to be inserted. embeddings (Embeddings): Embedding function. client (pyepsilla.vectordb.Client): Epsilla client to connect to. metadatas (Optional[List[dict]]): Metadata for each text. Defaults to None. db_path (Optional[str]): The path where the database will be persisted. Defaults to "/tmp/langchain-epsilla". db_name (Optional[str]): Give a name to the loaded database. Defaults to "langchain_store". collection_name (Optional[str]): Which collection to use. Defaults to "langchain_collection". If provided, default collection name will be set as well. drop_old (Optional[bool]): Whether to drop the previous collection and create a new one. Defaults to False. Returns: Epsilla: Epsilla vector store. )rr)rAr+rc)rrv) rr`rrArrrr+rcreinstancer$r$r% from_textss"zEpsilla.from_texts documentsc Ks>dd|D} dd|D} |j| |f| |||||d|S)a"Create an Epsilla vectorstore from a list of documents. Args: texts (List[str]): List of text data to be inserted. embeddings (Embeddings): Embedding function. client (pyepsilla.vectordb.Client): Epsilla client to connect to. metadatas (Optional[List[dict]]): Metadata for each text. Defaults to None. db_path (Optional[str]): The path where the database will be persisted. Defaults to "/tmp/langchain-epsilla". db_name (Optional[str]): Give a name to the loaded database. Defaults to "langchain_store". collection_name (Optional[str]): Which collection to use. Defaults to "langchain_collection". If provided, default collection name will be set as well. drop_old (Optional[bool]): Whether to drop the previous collection and create a new one. Defaults to False. Returns: Epsilla: Epsilla vector store. cSg|]}|jqSr$)rrJdocr$r$r%rLmz*Epsilla.from_documents..cSrr$)r\rr$r$r%rLnr)rArrrr+rc)r) rrrrrrr+rcrer`rAr$r$r%from_documentsLs! zEpsilla.from_documents)rrrr rrrr)r'r()r+rr'r,)r/)r/N)r+rr1r2r'r3r))r4rrr@rArBr'r,)Nr/F) r`rarArbr+rrcrdrerr'rf)rwr/) rxrryrTr+rrerr'rz)rrr`rfrr rArbrrrrrrr+rrcrdrerr'r)rrrrzrr rrrrrrr+rrcrdrerr'r)__name__ __module__ __qualname____doc__r__annotations__rrr&propertyrr.r0r9r_rvr classmethodrrr$r$r$r%rsJ          / G (,r)r __future__rloggingrktypingrrrrrrZlangchain_core.documentsr Zlangchain_core.embeddingsr Zlangchain_core.vectorstoresr rr getLoggerr:rr$r$r$r%s