o Zhf0@sddlmZddlZddlmZddlmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZmZdd lmZdd lmZmZeeZGd d d eZdS) ) annotationsN)Path)AnyDictIteratorListOptionalUnion)CallbackManagerForLLMRun)LLM)GenerationChunk)get_pydantic_field_namespre_init)_build_model_kwargs)Fieldmodel_validatorc@seZdZUdZdZded<ded< dZded< dZded < ed d d Z d ed < eddd Z d ed< eddd Z d ed< eddd Z ded< eddd Z ded< eddd Zded< eddd Zded< eddd Zded< eddd Zded< eddd Zded< edZded< dZded< d Zd!ed"< d#Zd!ed$< edZded%< dZd&ed'< gZd(ed)< d*Zd!ed+< d,Zded-< d.Zded/< dZd&ed0< d1Zd2ed3< d4Zd2ed5< ee d6Z!d7ed8< dZ"ded9< dZ#d:ed;< dZ$d< e&d\dBdCZ'e(dDdEe)d]dFdGZ*e+d^dHdIZ,e+d^dJdKZ-e+d_dLdMZ.d`dadNdOZ/  dbdcdTdUZ0  dbdddWdXZ1dedZd[Z2dS)fLlamaCppallama.cpp model. To use, you should have the llama-cpp-python library installed, and provide the path to the Llama model as a named parameter to the constructor. Check out: https://github.com/abetlen/llama-cpp-python Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp(model_path="/path/to/llama/model") Nrclientstr model_pathz Optional[str] lora_base lora_pathin_ctx)aliasintn_partsseedTf16_kvboolF logits_all vocab_only use_mlock n_threadsz Optional[int]n_batch n_gpu_layerssuffix max_tokensg?zOptional[float] temperaturegffffff?top_plogprobszOptional[bool]echoOptional[List[str]]stopg?repeat_penalty(top_k@last_n_tokens_sizeuse_mmapg?floatrope_freq_scaleg@rope_freq_base)default_factoryDict[str, Any] model_kwargs streamingzOptional[Union[str, Path]] grammar_pathzOptional[Union[str, Any]]grammarverbosevaluesrreturnc s8z ddlm}m}Wn tytdwd}gd}fdd|D}dd ur3d|d<|d z ||fi|d <Wnty\}z td |d |d }~wwdrxdrxd}d} td|d| dtdtr| dd<Sdr| dd<S S)z4Validate that llama-cpp-python library is installed.r)Llama LlamaGrammarzCould not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-pythonr)r7r8rrrrrrr r!r"r#r%r5r4r?csi|]}||qSrD).0kr@rDX/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/llms/llamacpp.py sz1LlamaCpp.validate_environment..r&Nr;rz&Could not load Llama model from path: z. Received error r>r=zCCan only pass in one of grammar and grammar_path. Received grammar=z and grammar_path=.) Z llama_cpprBrC ImportErrorupdate Exception ValueError isinstancerZ from_string from_file) clsr@rBrCrZmodel_param_namesZ model_paramser>r=rDrGrHvalidate_environmentsR   zLlamaCpp.validate_environmentbefore)modecCst|}t||}|S)z>Build extra kwargs from additional params that were passed in.)r r)rQr@Zall_required_field_namesrDrDrHbuild_model_kwargss zLlamaCpp.build_model_kwargsc Cs>|j|j|j|j|j|j|j|j|jd }|j r|j |d<|S)z1Get the default parameters for calling llama_cpp.) r'r)r*r+r,r-stop_sequencesr0r2r>) r'r)r*r+r,r-r/r0r2r>)selfparamsrDrDrH_default_paramss zLlamaCpp._default_paramscCsid|ji|jS)zGet the identifying parameters.r)rrZrXrDrDrH_identifying_paramsszLlamaCpp._identifying_paramscCsdS)zReturn type of llm.ZllamacpprDr[rDrDrH _llm_typeszLlamaCpp._llm_typecCs<|jr |dur td|j}|d|jp|pg|d<|S)a Performs sanity check, preparing parameters in format needed by llama_cpp. Args: stop (Optional[List[str]]): List of stop sequences for llama_cpp. Returns: Dictionary containing the combined parameters. Nz2`stop` found in both the input and default params.rWr/)r/rNrZpop)rXr/rYrDrDrH_get_parameterss   zLlamaCpp._get_parametersprompt run_manager"Optional[CallbackManagerForLLMRun]kwargsc Ksp|jrd}|jd|||d|D]}||j7}q|S||}i||}|jdd|i|}|dddS) aCall the Llama model and return the output. Args: prompt: The prompt to use for generation. stop: A list of strings to stop generation when encountered. Returns: The generated text. Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp(model_path="/path/to/local/llama/model.bin") llm.invoke("This is a prompt.") )r`r/rar`choicesrtextNrD)r<_streamrfr_r) rXr`r/rarcZcombined_text_outputchunkrYresultrDrDrH_calls   zLlamaCpp._callIterator[GenerationChunk]c ksi|||}|jd |dd|}|D])}|dddd}t|dddd|id} |r=|j| j|j|d | VqdS) a\Yields results objects as they are generated in real time. It also calls the callback manager's on_llm_new_token event with similar parameters to the OpenAI LLM class method of the same name. Args: prompt: The prompts to pass into the model. stop: Optional list of stop words to use when generating. Returns: A generator representing the stream of tokens being generated. Yields: A dictionary like objects containing a string token and metadata. See llama-cpp-python docs and below for more. Example: .. code-block:: python from langchain_community.llms import LlamaCpp llm = LlamaCpp( model_path="/path/to/local/model.bin", temperature = 0.5 ) for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'", stop=["'"," "]): result = chunk["choices"][0] print(result["text"], end='', flush=True) # noqa: T201 T)r`streamrerr,Nrf)rfZgeneration_info)tokenr?Z log_probsrD)r_rgetr Zon_llm_new_tokenrfr?) rXr`r/rarcrYripartr,rhrDrDrHrg,s% zLlamaCpp._streamrfcCs|j|d}t|S)Nzutf-8)rtokenizeencodelen)rXrfZtokenized_textrDrDrHget_num_tokens_szLlamaCpp.get_num_tokens)r@rrAr)r@r:rAr)rAr:)rAr)N)r/r.rAr:)NN) r`rr/r.rarbrcrrAr) r`rr/r.rarbrcrrArk)rfrrAr)3__name__ __module__ __qualname____doc__r__annotations__rrrrrrrr r!r"r#r%r&r'r)r*r+r,r-r/r0r2r4r5r7r8dictr;r<r=r>r?rrSr classmethodrVpropertyrZr\r]r_rjrgrsrDrDrDrHrs                    <     - 3r) __future__rloggingpathlibrtypingrrrrrr Zlangchain_core.callbacksr Z#langchain_core.language_models.llmsr Zlangchain_core.outputsr Zlangchain_core.utilsr rZlangchain_core.utils.utilsrZpydanticrr getLoggerrtloggerrrDrDrDrHs