o Zh%@sdZddlmZmZmZmZmZmZmZm Z m Z m Z ddl m Z mZddlmZddlmZddlmZmZmZmZmZddlmZmZmZmZddlmZdd l m!Z!dd l"m#Z#dd l$m%Z%d Z&Gd ddeZ'dS)zMLX Chat Wrapper.) AnyCallableDictIteratorListLiteralOptionalSequenceTypeUnion)AsyncCallbackManagerForLLMRunCallbackManagerForLLMRun)LanguageModelInput) BaseChatModel) AIMessageAIMessageChunk BaseMessage HumanMessage SystemMessage)ChatGenerationChatGenerationChunk ChatResult LLMResult)Runnable)BaseToolconvert_to_openai_tool) MLXPipelinez4You are a helpful, respectful, and honest assistant.c seZdZUdZeed<eedZeed<dZ e ed<de ffdd Z  d'd e e d ee ed eede d ef ddZ  d'd e e d ee ed eede d ef ddZ  d(d e e dedeed efddZde d efddZeded efddZed efddZ  d'd e e d ee ed eede d eef dd Z dd!d"e!e"e#ee fe$e%e&fd#ee"eee'd$efde d e(e)e fffd%d&Z*Z+S))ChatMLXaMLX chat models. Works with `MLXPipeline` LLM. To use, you should have the ``mlx-lm`` python package installed. Example: .. code-block:: python from langchain_community.chat_models import chatMLX from langchain_community.llms import MLXPipeline llm = MLXPipeline.from_model_id( model_id="mlx-community/quantized-gemma-2b-it", ) chat = chatMLX(llm=llm) llmcontentsystem_messageN tokenizerkwargsc s tjdi||jj|_dS)N)super__init__rr#)selfr$ __class__r%Z/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/chat_models/mlx.pyr'DszChatMLX.__init__messagesstop run_managerreturncKs0||}|jjd|g||d|}||SN)Zpromptsr-r.r%)_to_chat_promptr _generate_to_chat_resultr(r,r-r.r$ llm_input llm_resultr%r%r+r2Hs  zChatMLX._generatecs8||}|jjd|g||d|IdH}||Sr0)r1r _agenerater3r4r%r%r+r7Us  zChatMLX._agenerateFtokenizereturn_tensorscsH|stdt|dtstdfdd|D}jj||d|dS)zHConvert a list of messages into a prompt format expected by wrapped LLM.z+At least one HumanMessage must be provided!z$Last message must be a HumanMessage!csg|]}|qSr%)_to_chatml_format).0mr(r%r+ osz+ChatMLX._to_chat_prompt..T)r8Zadd_generation_promptr9) ValueError isinstancerr#Zapply_chat_template)r(r,r8r9Zmessages_dictsr%r>r+r1bszChatMLX._to_chat_promptmessagecCsNt|trd}nt|trd}nt|trd}n tdt|||jdS)z+Convert LangChain message to ChatML format.systemZ assistantuserzUnknown message type: )roler!)rArrrr@typer!)r(rBrEr%r%r+r;ws    zChatMLX._to_chatml_formatr6cCsBg}|jdD]}tt|jd|jd}||qt||jdS)Nrr )rBgeneration_info) generations llm_output)rHrrtextrGappendrrI)r6Zchat_generationsgZchat_generationr%r%r+r3s zChatMLX._to_chat_resultcCsdS)Nzmlx-chat-wrapperr%r>r%r%r+ _llm_typeszChatMLX._llm_typecksddlm}ddlm}zddlm}ddlm}m}ddlm}Wn ty.tdw|d|j j } | dd} | dd } | d d} | d d} | d d }| dd}| dd}|j |ddd}| |d}|j j}|| pxd|||}|d| | }t|||j j||dt| D]A\\}}}d}t|ts|j |}n|j |}|rtt|dd}|r|j||d|V||ks|dur||vrdSqdS)Nr) generate_step)make_logits_processors make_samplerzTCould not import mlx_lm python package. Please install it with `pip install mlx_lm`. model_kwargstempgZ max_tokensdrepetition_penaltyrepetition_context_sizetop_pg?min_pmin_tokens_to_keepTnp)r8r9)samplerlogits_processorsr )rB)chunk)Zmlx.corecoreZ mlx_lm.utilsrNZmlx_lm.sample_utilsrOrP ImportErrorgetrZpipeline_kwargsr1arrayr# eos_token_idzipmodelrangerAintdecodeitemrrZon_llm_new_token)r(r,r-r.r$ZmxrNrOrPrQrRZmax_new_tokensrTrUrVrWrXr5Z prompt_tokensrbr[r\tokenZprobnrJr]r%r%r+_streamsj           zChatMLX._stream) tool_choicetoolsrlautononec sdd|D}|durh|rht|dkrtdt|dt|tr.|dvr-dd |id }n6t|tr8|d }n,t|tr]|d dd |dd kr\td |d |d dd dntd|||d<tjdd|i|S)a*Bind tool-like objects to this chat model. Assumes model is compatible with OpenAI tool-calling API. Args: tools: A list of tool definitions to bind to this chat model. Supports any tool definition handled by :meth:`langchain_core.utils.function_calling.convert_to_openai_tool`. tool_choice: Which tool to require the model to call. Must be the name of the single provided function or "auto" to automatically determine which function to call (if any), or a dict of the form: {"type": "function", "function": {"name": <>}}. **kwargs: Any additional parameters to pass to the :class:`~langchain.runnable.Runnable` constructor. cSsg|]}t|qSr%r)r<Ztoolr%r%r+r?sz&ChatMLX.bind_tools..NrYzKWhen specifying `tool_choice`, you must provide exactly one tool. Received z tools.rnfunctionname)rFrqrz Tool choice z/ was specified, but the only provided tool was .zEUnrecognized tool_choice type. Expected str, bool or dict. Received: rlrmr%)lenr@rAstrbooldictr&bind)r(rmrlr$Zformatted_toolsr)r%r+ bind_toolssB       zChatMLX.bind_tools)NN)FN),__name__ __module__ __qualname____doc__r__annotations__rDEFAULT_SYSTEM_PROMPTr"r#rr'rrrrur rr2r r7rvr1rwr; staticmethodrr3propertyrMrrrkr r rr rrrrrry __classcell__r%r%r)r+r,s          L rN)(r}typingrrrrrrrr r r Z langchain_core.callbacks.managerr r Zlangchain_core.language_modelsrZ*langchain_core.language_models.chat_modelsrZlangchain_core.messagesrrrrrZlangchain_core.outputsrrrrZlangchain_core.runnablesrZlangchain_core.toolsrZ%langchain_core.utils.function_callingrZ%langchain_community.llms.mlx_pipelinerrrr%r%r%r+s0