o .if_@sdZddlmZddlZddlZddlZddlmZddlm Z m Z m Z m Z m Z ddlmZddlmZmZddlmZejd d Zd#ddZd$ddZd%ddZd&ddZd'dd ZGd!d"d"eZdS)(a!Module contains code for a cache backed embedder. The cache backed embedder is a wrapper around an embedder that caches embeddings in a key-value store. The cache is used to avoid recomputing embeddings for the same text. The text is hashed and the hash is used as the key in the cache. ) annotationsN)partial)CallableListSequenceUnioncast) Embeddings) BaseStore ByteStore)EncoderBackedStorei)int input_stringstrreturn uuid.UUIDcCs t|d}tt|S)z1Hash a string and returns the corresponding UUID.zutf-8)hashlibsha1encode hexdigestuuiduuid5NAMESPACE_UUID)r hash_valuerS/var/www/html/corbot_env/lib/python3.10/site-packages/langchain/embeddings/cache.py_hash_string_to_uuids rkey namespacecCs|tt|S)z Encode a key.)rr)rrrrr _key_encodersrCallable[[str], str]cCs tt|dS)zCreate an encoder for a key.r)rrr!rrr_create_key_encoder$s r"valueSequence[float]bytescCst|S)zSerialize a value.)jsondumpsr)r#rrr_value_serializer)sr(serialized_value List[float]cCstttt|S)zDeserialize a value.)rrfloatr&loadsdecode)r)rrr_value_deserializer.sr.csJeZdZdZdfdd Zdd dZdddZedddddZZ S)CacheBackedEmbeddingsavInterface for caching results from embedding models. The interface allows works with any store that implements the abstract store interface accepting keys of type str and values of list of floats. If need be, the interface can be extended to accept other implementations of the value serializer and deserializer, as well as the key encoder. Examples: .. code-block: python from langchain.embeddings import CacheBackedEmbeddings from langchain.storage import LocalFileStore from langchain_community.embeddings import OpenAIEmbeddings store = LocalFileStore('./my_cache') underlying_embedder = OpenAIEmbeddings() embedder = CacheBackedEmbeddings.from_bytes_store( underlying_embedder, store, namespace=underlying_embedder.model ) # Embedding is computed and cached embeddings = embedder.embed_documents(["hello", "goodbye"]) # Embeddings are retrieved from the cache, no computation is done embeddings = embedder.embed_documents(["hello", "goodbye"]) underlying_embeddingsr document_embedding_storeBaseStore[str, List[float]]rNonecst||_||_dS)zInitialize the embedder. Args: underlying_embeddings: the embedder to use for computing embeddings. document_embedding_store: The store to use for caching document embeddings. N)super__init__r1r0)selfr0r1 __class__rrr5Ss  zCacheBackedEmbeddings.__init__texts List[str]List[List[float]]cs|j}ddt|D}fdd|D}|r9|j|}|jtt||t||D]\}}|||<q0tt t t |S)awEmbed a list of texts. The method first checks the cache for the embeddings. If the embeddings are not found, the method uses the underlying embedder to embed the documents and stores the results in the cache. Args: texts: A list of texts to embed. Returns: A list of embeddings for the given texts. cSsg|] \}}|dur|qS)Nr).0ivectorrrr rsz9CacheBackedEmbeddings.embed_documents..csg|]}|qSrr)r<r=r9rrr?us) r1mget enumerater0embed_documentsmsetlistziprrr+)r6r9vectorsmissing_indices missing_textsmissing_vectorsindexupdated_vectorrr@rrCbs"     z%CacheBackedEmbeddings.embed_documentstextrr*cCs |j|S)a<Embed query text. This method does not support caching at the moment. Support for caching queries is easily to implement, but might make sense to hold off to see the most common patterns. If the cache has an eviction policy, we may need to be a bit more careful about sharing the cache between documents and queries. Generally, one is OK evicting query caches, but document caches should be kept. Args: text: The text to embed. Returns: The embedding for the given text. )r0 embed_query)r6rMrrrrNs z!CacheBackedEmbeddings.embed_queryr!document_embedding_cacher rcCs0|}t|}ttttf||tt}|||S)aOn-ramp that adds the necessary serialization and encoding to the store. Args: underlying_embeddings: The embedder to use for embedding. document_embedding_cache: The cache to use for storing document embeddings. *, namespace: The namespace to use for document cache. This namespace is used to avoid collisions with other caches. For example, set it to the name of the embedding model used. )r"r rrr+r(r.)clsr0rPr key_encoderencoder_backed_storerrrfrom_bytes_stores z&CacheBackedEmbeddings.from_bytes_store)r0r r1r2rr3)r9r:rr;)rMrrr*)r0r rPr rrrr/) __name__ __module__ __qualname____doc__r5rCrN classmethodrT __classcell__rrr7rr/3s  !r/)rrrr)rrrrrr)rrrr )r#r$rr%)r)r%rr*)rX __future__rrr&r functoolsrtypingrrrrrlangchain_core.embeddingsr langchain_core.storesr r langchain.storage.encoder_backedr UUIDrrrr"r(r.r/rrrrs"