o Zh}@sdddlmZddlmZmZmZmZmZddlm Z m Z ddl m Z ddl mZGddde ZdS) )partial)AnyDictListOptionalSequence)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LLM)pre_initc @seZdZUdZeed<eed< dZeeed< dZ eeed< dZ ee eefed< dZ eeed< e d e eeffd d Ze d efd d Zede d e fddZ  ddedeeedeeded ef ddZ  ddedeeedeeded ef ddZdS) CTransformersa_C Transformers LLM models. To use, you should have the ``ctransformers`` python package installed. See https://github.com/marella/ctransformers Example: .. code-block:: python from langchain_community.llms import CTransformers llm = CTransformers(model="/path/to/ggml-gpt-2.bin", model_type="gpt2") clientmodelN model_type model_fileconfiglibreturncCs|j|j|j|jdS)zGet the identifying parameters.rrrrrselfr]/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/llms/ctransformers.py_identifying_params-s z!CTransformers._identifying_paramscCsdS)zReturn type of llm. ctransformersrrrrr _llm_type7szCTransformers._llm_typevaluescCsfzddlm}Wn tytdw|dpi}|j|df|d|d|dd ||d <|S) z5Validate that ``ctransformers`` package is installed.r)AutoModelForCausalLMz\Could not import `ctransformers` package. Please install it with `pip install ctransformers`rrrrr)rrrr )rr ImportErrorZfrom_pretrained)clsrrrrrrvalidate_environment<s$   z"CTransformers.validate_environmentpromptstop run_managerkwargscKsJg}|pt}|j||ddD]}|||j||jdqd|S)aVGenerate text from a prompt. Args: prompt: The prompt to generate text from. stop: A list of sequences to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python response = llm.invoke("Tell me a joke.") Tr"streamverbose)r Zget_noop_managerr appendon_llm_new_tokenr(join)rr!r"r#r$textZ _run_managerchunkrrr_callQs    zCTransformers._callcsRd}|r t|j|jd}d}|j||ddD]}|r"||IdH||7}q|S)aAsynchronous Call out to CTransformers generate method. Very helpful when streaming (like with websockets!) Args: prompt: The prompt to pass into the model. stop: A list of strings to stop generation when encountered. Returns: The string generated by the model. Example: .. code-block:: python response = llm.invoke("Once upon a time, ") Nr'r)Tr%)rr+r(r )rr!r"r#r$Z text_callbackr-tokenrrr_acallms zCTransformers._acall)NN)__name__ __module__ __qualname____doc__r__annotations__strrrrrrrpropertyrrr r rr r/rrr1rrrrr sZ      r N) functoolsrtypingrrrrrZlangchain_core.callbacksrr Z#langchain_core.language_models.llmsr Zlangchain_core.utilsr r rrrrs