o Zh"@sddlmZddlmZmZmZmZmZmZm Z ddlmZddl m Z ddlmZddl m Z mZddlmZddlmZddlmZddlmZddlmZddlmZGdd d eZd S) )pre_init)Any AsyncIteratorDictIteratorListOptionalUnion)root_validator)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)enforce_stop_tokens)GenerationChunkc @sneZdZUdZeed<eed< dZee eefed< dZ e dee fed< dZ e ed< ed e eeffd d Zed efd d Zede d e fddZ  ddedeeedeeded ef ddZ  ddedeeedeeded ef ddZ  ddedeeedeeded eef ddZ  ddedeeedeeded eef ddZdS) DeepSparseaHNeural Magic DeepSparse LLM interface. To use, you should have the ``deepsparse`` or ``deepsparse-nightly`` python package installed. See https://github.com/neuralmagic/deepsparse This interface let's you deploy optimized LLMs straight from the [SparseZoo](https://sparsezoo.neuralmagic.com/?useCase=text_generation) Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") pipelinemodelNmodel_configurationgeneration_configF streamingreturncCs|j|j|j|jdS)zGet the identifying parameters.)r model_configrr)rrrrselfrZ/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/llms/deepsparse.py_identifying_params1s zDeepSparse._identifying_paramscCsdS)zReturn type of llm. deepsparserrrrr _llm_type;szDeepSparse._llm_typevaluescCsVzddlm}Wn tytdw|dpi}|jd d|dd||d<|S) z2Validate that ``deepsparse`` package is installed.r)Pipelinez[Could not import `deepsparse` package. Please install it with `pip install deepsparse[llm]`rZtext_generationr)taskZ model_pathrNr)rr ImportErrorcreate)clsrr rrrrvalidate_environment@s   zDeepSparse.validate_environmentpromptstop run_managerkwargscKsl|jrd}|jd|||d|D]}||j7}q|}n|jdd|i|jjdj}|dur4t||}|S)Generate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse(model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none") llm.invoke("Tell me a joke.") r&r'r( sequencesrNr)r_streamtextrr generationsrrr&r'r(r)Zcombined_outputchunkr/rrr_callTs"  zDeepSparse._callcsx|jr"d}|jd|||d|2z 3dHW}||j7}q6|}n|jdd|i|jjdj}|dur:t||}|S)r*r+r,Nr-rr)r_astreamr/rrr0rr1rrr_acallzs&  zDeepSparse._acallcksR|jd|dd|j}|D]}t|jdjd}|r#|j|jd|VqdSaYields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompt to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like object containing a string token. Example: .. code-block:: python from langchain_community.llms import DeepSparse llm = DeepSparse( model="zoo:nlg/text_generation/codegen_mono-350m/pytorch/huggingface/bigpython_bigquery_thepile/base_quant-none", streaming=True ) for chunk in llm.stream("Tell me a joke", stop=["'"," "]): print(chunk, end='', flush=True) # noqa: T201 T)r-rr)r/)tokenNrrrrr0r/Zon_llm_new_tokenrr&r'r(r)Z inferencer7r2rrrr.szDeepSparse._streamcKsX|jd|dd|j}|D]}t|jdjd}|r&|j|jdIdH|VqdSr6r8r9rrrr4szDeepSparse._astream)NN)__name__ __module__ __qualname____doc__r__annotations__strrrrrr rboolpropertyrrrr%rr r3r r5rrr.rr4rrrrrs      )  )  ) rN)Zlangchain_core.utilsrtypingrrrrrrr Zpydanticr Zlangchain_core.callbacksr r Z#langchain_core.language_models.llmsr Zlangchain_community.llms.utilsrZlangchain_core.outputsrrrrrrs $