o Zh<'@sfddlZddlmZmZmZmZddlmZddlm Z ddl m Z dZ e eZGddde ZdS) N)AnyListMappingOptional)CallbackManagerForLLMRun)LLM) ConfigDictZgpt2c@seZdZUdZeZeed< dZe e ed< dZ e ed< dZ e ed< dZeed< ed d Ze d(dddd dede e d e ed ede ede defddZe d(dddede e d e ede def ddZe      d)ded e ed ede edede e de e de fddZede dedede de de f ddZedeee ffdd Zedefd!d"Z  d*d#ed$e eed%e ede def d&d'ZdS)+IpexLLMzIpexLLM model. Example: .. code-block:: python from langchain_community.llms import IpexLLM llm = IpexLLM.from_model_id(model_id="THUDM/chatglm-6b") model_idN model_kwargsmodel tokenizerT streamingZforbid)extra) tokenizer_id load_in_4bitload_in_low_bitrrrkwargsreturnc Ks|j||d||||dS)a Construct object from model_id Args: model_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder. tokenizer_id: Path for the huggingface repo id to be downloaded or the huggingface checkpoint folder which contains the tokenizer. load_in_4bit: "Whether to load model in 4bit. Unused if `load_in_low_bit` is not None. load_in_low_bit: Which low bit precisions to use when loading model. Example values: 'sym_int4', 'asym_int4', 'fp4', 'nf4', 'fp8', etc. Overrides `load_in_4bit` if specified. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. Fr r low_bit_modelrrr r _load_model)clsr r rrrrrX/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/llms/ipex_llm.py from_model_id's zIpexLLM.from_model_id)rc Ks|j||ddd||dS)a Construct low_bit object from model_id Args: model_id: Path for the ipex-llm transformers low-bit model folder. tokenizer_id: Path for the huggingface repo id or local model folder which contains the tokenizer. model_kwargs: Keyword arguments to pass to the model and tokenizer. kwargs: Extra arguments to pass to the model and tokenizer. Returns: An object of IpexLLM. TFNrr)rr r rrrrrfrom_model_id_low_bitQszIpexLLM.from_model_id_low_bitFrc Cs|zddlm}m} ddlm} m} Wn tytdw|p!i} |p%i}|p)|} d| vr2d| d<| ddvrBtd| dd | d}z | j | fi| }Wnt yd| j | fi| }Ynwd | vrrd d | D} d d d}|s|durd}||d<n d}||d<nd}z |j | |||| d}Wnt y|j ||||| d}Ynw| ||d|||| d|S)Nr) AutoModelAutoModelForCausalLM) AutoTokenizerLlamaTokenizerzCould not import ipex-llm. Please install `ipex-llm` properly following installation guides: https://github.com/intel-analytics/ipex-llm?tab=readme-ov-file#install-ipex-llm.devicecpu)r#ZxpuzXIpexLLMBgeEmbeddings currently only supports device to be 'cpu' or 'xpu', but you have: .trust_remote_codecSsi|] \}}|dkr||qS)r%r).0kvrrr sz'IpexLLM._load_model..T)Z use_cacher%from_pretrainedrrZ load_low_bit)load_function_namer load_kwargsr )r r r r r)Zipex_llm.transformersrr transformersr r! ImportError ValueErrorpopr* Exceptionitems_load_model_generalto)rr rrrrr rrrr r!Z _model_kwargsZ _tokenizer_idr"r r,r+r rrrrss          zIpexLLM._load_model model_classr+r,c Csjzt||}||fii||WSty4}ztd|jd|d|WYd}~dSd}~ww)z,General function to attempt to load a model.zFailed to load model using r$z: N)getattrr1loggererror__name__)r5r+r r,r Z load_functionerrrr3s zIpexLLM._load_model_generalcCs|j|jdS)zGet the identifying parameters.r r r;selfrrr_identifying_paramsszIpexLLM._identifying_paramscCsdS)Nzipex-llmrr<rrr _llm_typeszIpexLLM._llm_typepromptstop run_managerc Ks<|jrUddlm}|jj|dd}||jj}||jddd}|dur:ddlm }ddl m } || ||jg} nd} |jj |f|| d |} |jj | ddd } | S|jj|dd}||jj}|dur~ddlm }ddl m } || ||jg} nd} |jj |fd | i|} |jj | ddd t|d} | S) Nr) TextStreamerpt)Zreturn_tensorsT)Z skip_promptskip_special_tokens)StoppingCriteriaList)StopSequenceCriteria)streamerstopping_criteria)rErI)rr-rCr encoder4r r"Z)transformers.generation.stopping_criteriarFZtransformers.tools.agentsrGgeneratedecodelen) r=r@rArBrrCZ input_idsrHrFrGrIoutputtextrrr_callsX        z IpexLLM._call)N)NFNFNN)NN) r9 __module__ __qualname____doc__DEFAULT_MODEL_IDr str__annotations__r rdictr rr rboolrZ model_config classmethodrrrr staticmethodr3propertyrr>r?rrrPrrrrr s      ) ! _  r )loggingtypingrrrrZlangchain_core.callbacksrZ#langchain_core.language_models.llmsrZpydanticrrT getLoggerr9r7r rrrrs