o Zh0&@sddlZddlZddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z ddlmZddlmZddlZddlZddlmZddlmZdd lmZdd lmZmZmZmZee Z!Gd d d Z"Gd ddeeZ#Gddde$Z%dS)N)datetime)mktime)AnyDictListLiteralOptional) urlencode)format_date_time) Embeddings)secret_from_env)ndarray) BaseModel ConfigDictField SecretStrc@s*eZdZdZdedededdfddZdS) UrlzURL class for parsing the URL.hostpathschemareturnNcCs||_||_||_dSN)rrr)selfrrrr^/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/embeddings/sparkllm.py__init__%sz Url.__init____name__ __module__ __qualname____doc__strrrrrrr"src @s|eZdZUdZededdZeed< ededdd dZ e eed < ed ed dd dZ e eed < edd Z e ed< edd Zeded< eddZdee de de eeefddZdee de eeefddZde de eefddZe d-d!e d"e de d e de f d#d$Zed!e defd%d&Zd'e dedee effd(d)Zed*e de efd+d,Z dS).SparkLLMTextEmbeddingsuSparkLLM embedding model integration. Setup: To use, you should have the environment variable "SPARK_APP_ID","SPARK_API_KEY" and "SPARK_API_SECRET" set your APP_ID, API_KEY and API_SECRET or pass it as a name parameter to the constructor. .. code-block:: bash export SPARK_APP_ID="your-api-id" export SPARK_API_KEY="your-api-key" export SPARK_API_SECRET="your-api-secret" Key init args — completion params: api_key: Optional[str] Automatically inferred from env var `SPARK_API_KEY` if not provided. app_id: Optional[str] Automatically inferred from env var `SPARK_APP_ID` if not provided. api_secret: Optional[str] Automatically inferred from env var `SPARK_API_SECRET` if not provided. base_url: Optional[str] Base URL path for API requests. See full list of supported init args and their descriptions in the params section. Instantiate: .. code-block:: python from langchain_community.embeddings import SparkLLMTextEmbeddings embed = SparkLLMTextEmbeddings( api_key="...", app_id="...", api_secret="...", # other ) Embed single text: .. code-block:: python input_text = "The meaning of life is 42" embed.embed_query(input_text) .. code-block:: python [-0.4912109375, 0.60595703125, 0.658203125, 0.3037109375, 0.6591796875, 0.60302734375, ...] Embed multiple text: .. code-block:: python input_texts = ["This is a test query1.", "This is a test query2."] embed.embed_documents(input_texts) .. code-block:: python [ [-0.1962890625, 0.94677734375, 0.7998046875, -0.1971435546875, 0.445556640625, 0.54638671875, ...], [ -0.44970703125, 0.06585693359375, 0.7421875, -0.474609375, 0.62353515625, 1.0478515625, ...], ] app_idZ SPARK_APP_ID)aliasdefault_factory spark_app_idapi_keyZ SPARK_API_KEYN)default spark_api_key api_secretZSPARK_API_SECRETspark_api_secretz#https://emb-cn-huabei-1.xf-yun.com/base_urlpara)r-querydomainT)Zpopulate_by_nametextsrrc Csd}d}d}|jr|j}|jr|j}|jr|j}|j|d||d}g}|D]1}d|ddgi} ||| } tj|| ddid j} | | } | d urW| | q+| d q+|S) aDInternal method to call Spark Embedding API and return embeddings. Args: texts: A list of texts to embed. host: Base URL path for API requests Returns: A list of list of floats representing the embeddings, or list with value None if an error occurs. POST) request_urlmethodr'r*messagesuser)contentZrolez content-typezapplication/json)jsonheadersN) r&Zget_secret_valuer)r+_assemble_ws_auth_url _get_bodyrequestsposttext_parser_messageappendtolist) rr0rr#r'r*urlZ embed_resultr>Z query_contextr7responseZres_arrrrr_embeds:        zSparkLLMTextEmbeddings._embedcCs|||jS)zPublic method to get embeddings for a list of documents. Args: texts: The list of texts to embed. Returns: A list of embeddings, one for each text, or None if an error occurs. rDr,)rr0rrrembed_documentss z&SparkLLMTextEmbeddings.embed_documentsr>cCs$||g|j}|dur|dSdS)zPublic method to get embedding for a single query text. Args: text: The text to embed. Returns: Embeddings for the text, or None if an error occurs. NrrE)rr>resultrrr embed_querys z"SparkLLMTextEmbeddings.embed_queryGETr1r3r4cCst|}|j}|j}t}tt|}d ||||} t j | d| dt jd} t| jdd} d|dd| f} t| djdd} ||| d}|d t|S) Nz host: {} date: {} {} {} HTTP/1.1utf-8) digestmod)encodingz:api_key="%s", algorithm="%s", headers="%s", signature="%s"z hmac-sha256zhost date request-line)rdate authorization?)r" _parse_urlrrrnowr r timetupleformathmacnewencodehashlibsha256digestbase64 b64encodedecoder )r3r4r'r*urrrQrMZsignature_originZ signature_shaZsignature_sha_strZauthorization_originrNvaluesrrrr:s2   z,SparkLLMTextEmbeddings._assemble_ws_auth_urlcCsp|d}||dd}|d|d}|d}|dkr$td|||d}|d|}t|||}|S)Nz:///rzinvalid request url:)indexAssembleHeaderExceptionr)r3ZstidxrrZedidxrr]rrrrPs      z!SparkLLMTextEmbeddings._parse_urlappidc CsF|dddd|jddididd tt|d iid }|S) NZ 39769795890r_)r#uidstatusZembrLutf8)r/featurer5r>rJ)headerZ parameterpayload)r/rZr[r8dumpsrVr\)rrcr>bodyrrrr;s  z SparkLLMTextEmbeddings._get_bodymessagecCst|}|dd}|dkrtd|d|dS|ddd}t|}ttj}| d }tj ||d }t |d krI|dd }|S|}|S) NrhcoderzRequest error: z, rirgr><)dtypei ) r8loadsloggerwarningrZ b64decodenproZfloat32Z newbyteorderZ frombufferlen)rldatarmZ text_baseZ text_datadtr>arrayrrrr?s       z&SparkLLMTextEmbeddings._parser_message)rIr1r1)!rrrr rr r&r__annotations__r)rr+r,r!r/rrZ model_configrfloatrDrFrH staticmethodr:rrPdictrrr;r r?rrrrr",s^ >  &"(    r"c@s"eZdZdZdeddfddZdS)rbz3Exception raised for errors in the header assembly.msgrNcCs ||_dSr)rl)rr}rrrrs z AssembleHeaderException.__init__rrrrrrbsrb)&rZrWrTr8loggingrtimertypingrrrrr urllib.parser Zwsgiref.handlersr numpyrtr<Zlangchain_core.embeddingsr Zlangchain_core.utilsr r Zpydanticrrrr getLoggerrrqrr" Exceptionrbrrrrs*         e