o Zh@ @sjUdZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZmZmZmZmZmZmZmZmZmZmZddlmZddlmZm Z m!Z!m"Z"m#Z#m$Z$d d l%m&Z&d d l'm(Z(m)Z)m*Z*m+Z+m,Z,m-Z-d d l.m/Z/m0Z0m1Z1m2Z2e rddl3m4Z4m5Z5ddl6m7Z7e8Z9ee8e fZ:ee;efZe?e@ZAe GdddZBddZCddZDddZEdaFeee8ee8feGd<dee8ee8ffddZHdeedeefdd ZIed!e=dee<fd"d#ZJed!eddeedfd$d#ZJed!ee=deee<ddffd%d#ZJd!e=de8fd&d'ZKd(e8dd)fd*d+ZLd!e;defd,d-ZMd!e;defd.d/ZNd!e;dd)fd0d1ZOd2ee;d3ePdeee8ee2ffd4d5ZQd2ee;d3ePdeee8ee2ffd6d7ZRd8e;d3ePdeee8e2ffd9d:ZSd;ee2dee/fdd?ZUd@e2dAeVde/fdBdCZWdDee;dee/fdEdFZXdDee;dee/fdGdHZYd8e;dee/fdIdJZZdKdLdMdNdee;fdOdPZ[e\Z]eee8eGdQ<dRee8ddfdSdTZ^dRee8dePfdUdVZ_e\Z`ee8eGdW<dRe8ddfdXdYZadRe8dePfdZd[Zbd\edefd]d^Zcd_ee8d`ee8de"fdadbZddS)czEContains utilities used by both the sync and async inference clients.N)contextmanager) dataclass)Path) TYPE_CHECKINGAny AsyncIterableBinaryIOContextManagerDict GeneratorIterableListLiteralNoReturnOptionalSetUnionoverload) HTTPError)GenerationErrorIncompleteGenerationErrorOverloadedErrorTextGenerationError UnknownErrorValidationError)ENDPOINT)build_hf_headers get_sessionhf_raise_for_statusis_aiohttp_availableis_numpy_availableis_pillow_available)ChatCompletionStreamOutput ChatCompletionStreamOutputChoiceChatCompletionStreamOutputDeltaTextGenerationStreamOutput)ClientResponse ClientSessionImagez text-to-imagezimage-to-imagec@s2eZdZUdZeed<eed<eed<eed<dS) ModelStatusa This Dataclass represents the the model status in the Hugging Face Inference API. Args: loaded (`bool`): If the model is currently loaded into Hugging Face's InferenceAPI. Models are loaded on-demand, leading to the user's first request taking longer. If a model is loaded, you can be assured that it is in a healthy state. state (`str`): The current state of the model. This can be 'Loaded', 'Loadable', 'TooBig'. If a model's state is 'Loadable', it's not too big and has a supported backend. Loadable models are automatically loaded when the user first requests inference on the endpoint. This means it is transparent for the user to load a model, except that the first call takes longer to complete. compute_type (`Dict`): Information about the compute resource the model is using or will use, such as 'gpu' type and number of replicas. framework (`str`): The name of the framework that the model was built with, such as 'transformers' or 'text-generation-inference'. ZloadedstateZ compute_typeZ frameworkN)__name__ __module__ __qualname____doc__bool__annotations__strr r5r5X/var/www/html/lang_env/lib/python3.10/site-packages/huggingface_hub/inference/_common.pyr,Xs  r,cCststdddl}|S)NzMPlease install aiohttp to use `AsyncInferenceClient` (`pip install aiohttp`).r)r ImportErroraiohttp)r8r5r5r6_import_aiohttpyr9cCststdddl}|S)z.Make sure `numpy` is installed on the machine.zGPlease install numpy to use deal with embeddings (`pip install numpy`).rN)r!r7numpy)r;r5r5r6 _import_numpyr:r<cCststdddlm}|S)z,Make sure `PIL` is installed on the machine.zPlease install Pillow to use deal with images (`pip install Pillow`). If you don't want the image to be post-processed, use `client.post(...)` and get the raw response from the server.rr*)r"r7PILr+r*r5r5r6_import_pil_images  r>_RECOMMENDED_MODELSreturncCsBtdurtjtdtd}t|dd|DatS)Nz /api/tasks)headerscSsi|] \}}|t|dqS)Z widgetModels)_first_or_none).0taskdetailsr5r5r6 sz-_fetch_recommended_models..)r?rgetrrrjsonitems)responser5r5r6_fetch_recommended_modelss rKrIcCs$z|dpdWStyYdSw)Nr) IndexError)rIr5r5r6rBs  rBcontentcCdSNr5rMr5r5r6_open_as_binaryrQcCrNrOr5rPr5r5r6rQrRccst|tr2|ds|dr"td|t|jVdSt|}| s2t d|dt|trZtd|| d }|VWddS1sSwYdS|VdS) zOpen `content` as a binary file, either from a URL, a local path, or raw bytes. Do nothing if `content` is None, TODO: handle a PIL.Image as input TODO: handle base64 as input zhttps://zhttp://zDownloading content from NzFile not found at z. If `data` is a string, it must either be a URL or a path to a local file. To pass raw content, please encode it as bytes first.zOpening content from rb) isinstancer4 startswithloggerdebugrrGrMrexistsFileNotFoundErroropen)rMfr5r5r6rQs"    " cCsPt|}t|tr |n|}t|WdS1s!wYdS)z[Encode a raw file (image, audio) into base64. Can be byes, an opened file, a path or a URL.N)rQrTbytesreadbase64 b64encodedecode)rMdataZ data_as_bytesr5r5r6 _b64_encodes  $rb encoded_imager+cCst}|tt|S)z/Parse a base64-encoded string into a PIL Image.)r>rZioBytesIOr^ b64decode)rcr+r5r5r6 _b64_to_imagesrgcCt|S)acParse bytes from a Response object into a Python list. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_dict` and will not complain if the returned data is a dictionary. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rHloadsr`rPr5r5r6_bytes_to_listrkcCrh)acParse bytes from a Response object into a Python dictionary. Expects the response body to be JSON-encoded data. NOTE: This is exactly the same implementation as `_bytes_to_list` and will not complain if the returned data is a list. The only advantage of having both is to help the user (and mypy) understand what kind of data to expect. rirPr5r5r6_bytes_to_dictrlrmcCst}|t|S)zParse bytes from a Response object into a PIL Image. Expects the response body to be raw bytes. To deal with b64 encoded images, use `_b64_to_image` instead. )r>rZrdre)rMr+r5r5r6_bytes_to_imagesrnbytes_output_as_linesrEccs(|D]}t||}|dur|VqdS)z*Used in `InferenceClient.text_generation`.N%_format_text_generation_stream_outputrorE byte_payloadoutputr5r5r6 _stream_text_generation_response s rucCs2|2z3dHW}t||}|dur|Vq6dS)z/Used in `AsyncInferenceClient.text_generation`.Nrprrr5r5r6&_async_stream_text_generation_responses rvrscCsj|dsdS|d}t|dd}|ddur(t|d|dt |}|s3|j j S|S)Ndata:utf-8data:/nerror error_type) rUr`rHrjlstriprstriprG_parse_text_generation_errorr'parse_obj_as_instancetokentext)rsrEpayload json_payloadrtr5r5r6rq#s   rqtext_generation_outputccs(tt}|D]}t||Vq dS)z*Used in `InferenceClient.chat_completion`.Ninttime:_format_chat_completion_stream_output_from_text_generationrcreateditemr5r5r65_stream_chat_completion_response_from_text_generation6s  rcCs2tt}|2z 3dHW}t||Vq 6dSz/Used in `AsyncInferenceClient.chat_completion`.Nrrr5r5r6;_async_stream_chat_completion_response_from_text_generation?s  rrrcCsL|jdurtttd|jjddddg|dSttt|jjddg|dS)NZ assistant)ZrolerMr)delta finish_reasonindex)choicesr)rEr$r%r&rrr)rrr5r5r6rHs,  r bytes_linesccs&|D] }t|}|dur|VqdS)zFUsed in `InferenceClient.chat_completion` if model is served with TGI.NE_format_chat_completion_stream_output_from_text_generation_from_bytesrrrtr5r5r6+_stream_chat_completion_response_from_byteshsrcCs0|2z3dHW}t|}|dur|Vq6dSrrrr5r5r61_async_stream_chat_completion_response_from_bytesrsrcCs8|dsdS|d}t|dd}t|S)Nrwrxryrz)rUr`rHrjr}r~r$r)rsrrr5r5r6r|s   rclientr)rJr(cCs0|j2z 3dHW}|Vq6|IdHdSrO)rMclose)rrJrsr5r5r6_async_yield_froms r_NON_TGI_SERVERSmodelcCst|dSrO)raddrr5r5r6_set_as_non_tgisrcC|tvSrO)rrr5r5r6_is_tgi_serverr_NON_CHAT_COMPLETION_SERVERcCstd|t|dS)NzSet as non chat completion)printrrrr5r5r6"_set_as_non_chat_completion_servers rcCrrO)rrr5r5r6_is_chat_completion_serverrr http_errorcCs^zt|ddp |j}|d}|d}Wn ty |w|dur-t||}|||)z Try to parse text-generation-inference error message and raise HTTPError in any case. Args: error (`HTTPError`): The HTTPError that have been raised. Zresponse_error_payloadNr{r|)getattrrJrHrG Exceptionr)rrr{r| exceptionr5r5r6raise_text_generation_errors    rr{r|cCsH|dkrt|S|dkrt|S|dkrt|S|dkr t|St|S)NZ generationZincomplete_generationZ overloadedZ validation)rrrrr)r{r|r5r5r6rsr)er1r^rdrHloggingr contextlibr dataclassesrpathlibrtypingrrrrr r r r r rrrrrrrequestsrZhuggingface_hub.errorsrrrrrr constantsrutilsrrrr r!r"Z_generated.typesr$r%r&r'r8r(r)r=r+r4ZUrlTZPathTr\ZBinaryTZContentTZTASKS_EXPECTING_IMAGES getLoggerr.rVr,r9r<r>r?r3rKrBrQrbrgrkrmrnr2rurvrqrrrrrrrrsetrrrrrrrrr5r5r5r6s   D               &                     "