o ZhL"@sddlmZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z mZmZmZmZddlmZmZddlmZmZmZmZmZdd lmZdd lmZdd lm Z m!Z!m"Z"dd l#m$Z$e!%e&Z'dZ(erzddl)Z(Wne*yddl+Z,e,j(Z(Ynwd e$de$fddZ-d8ddZ.ddZ/ddZ0  d9de de1dee2fddZ3e-    d:dee4e fd eee4e fd!e1de1d"eee5e4ff d#d$Z6d;d&d'Z7e"e-dd(ddddddddddddd)d*e4d ee2d+e4d,e1d-ee4d.ee4d/ee4d0ee1d1eeee4e4fd2eeee4e4fd3eeee4e4fd4ee4d!e1d"eee5e4fde1fd5d6Z8Gd7d%d%eZ9dS)<Nwraps)Path)copytree)AnyDictListOptionalUnion) ModelHubMixinsnapshot_download)get_tf_versionis_graphviz_availableis_pydot_availableis_tf_available yaml_dump) CONFIG_NAME)HfApi)SoftTemporaryDirectoryloggingvalidate_hf_hub_args) CallableTfnreturncstfdd}|S)Ncs2t|dstdjd|g|Ri|S)Nhistoryz Cannot use 'z}': Keras 3.x is not supported. Please save models manually and upload them using `upload_folder` or `huggingface-cli upload`.)hasattrNotImplementedError__name__)modelargskwargsrR/var/www/html/lang_env/lib/python3.10/site-packages/huggingface_hub/keras_mixin.py_inner+s  z'_requires_keras_2_model.._innerr)rr%r#r"r$_requires_keras_2_model)sr&cCsdg}|D]'\}}|r|d|n|}t|tjr&|t||q|||fqt|S)aFlatten a nested dictionary. Reference: https://stackoverflow.com/a/6027615/10319735 Args: dictionary (`dict`): The nested dictionary to be flattened. parent_key (`str`): The parent key to be prefixed to the children keys. Necessary for recursing over the nested dictionary. Returns: The flattened dictionary. .)items isinstance collectionsMutableMappingextend _flatten_dictappenddict) dictionaryZ parent_keyr)keyvalueZnew_keyr#r#r$r.7s r.cCs`d}|jdur.|j}t|}tjj|d<d}|D]\}}|d|d|d7}q|S)z6Parse hyperparameter dictionary into a markdown table.NZtraining_precisionz*| Hyperparameters | Value | | :-- | :-- | z| z | z | )Z optimizerZ get_configr.kerasZmixed_precisionZ global_policynamer))rtableZoptimizer_paramsr2r3r#r#r$_create_hyperparameter_tableTs  r7c Cs(tjj||ddddddddd dS)N /model.pngFTTB`)Zto_fileZ show_shapesZ show_dtypeZshow_layer_namesZrankdirZ expand_nestedZdpiZ layer_range)r4utils plot_model)rsave_directoryr#r#r$ _plot_networkbs r>Trepo_dirr<metadatacCs|d}|r dSt|}|rtrtrt|||dur!i}d|d<d}|t|dd7}|d7}|d7}|d 7}|d 7}|durW|d 7}|d 7}|d 7}||7}|d7}|r|tj|dr||d7}|d7}|d7}d}|d|d7}|d7}||dS)zd Creates a model card for the repository. Do not overwrite an existing README.md file. z README.mdNr4 library_namez--- F)Zdefault_flow_stylez/ ## Model description More information needed z9 ## Intended uses & limitations More information needed z: ## Training and evaluation data More information needed z ## Training procedure z ### Training hyperparameters z; The following hyperparameters were used during training:  r8z ## Model Plot z
z$ View Model Plot z ./model.pngz ![Model Image](z) z
) existsr7rrr>rospath write_text)rr?r<r@Z readme_pathZhyperparametersZ model_cardZ path_to_plotr#r#r$_create_model_cardps:  rGFr=configinclude_optimizertagsc Kstdurtd|jstdt|}|jddd|rHt|ts+tdt |d|t  d}t ||Wdn1sCwYi}t|trT||d <n t|tr^|g|d <|d d} | durtd td |vrz|d | n| g|d <|jdur|jjikr|d } | rtd t| j ddd}t j |jj|dddWdn1swYt||||tjj||fd|i|dS)aL Saves a Keras model to save_directory in SavedModel format. Use this if you're using the Functional or Sequential APIs. Args: model (`Keras.Model`): The [Keras model](https://www.tensorflow.org/api_docs/python/tf/keras/Model) you'd like to save. The model must be compiled and built. save_directory (`str` or `Path`): Specify directory in which you want to save the Keras model. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. include_optimizer(`bool`, *optional*, defaults to `False`): Whether or not to include optimizer in serialization. plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. tags (Union[`str`,`list`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Nz>Called a Tensorflow-specific function but could not import it.z+Model should be built before trying to saveT)parentsexist_okzAProvided config to save_pretrained_keras should be a dict. Got: ''wrJ task_namez>`task_name` input argument is deprecated. Pass `tags` instead.z history.jsonzZ`history.json` file already exists, it will be overwritten by the history of this version.zutf-8)encoding)indent sort_keysrI)r4 ImportErrorZbuilt ValueErrorrmkdirr*r0 RuntimeErrortyperopenjsondumpliststrpopwarningswarn FutureWarningr/rrC UserWarningrGmodelsZ save_model) rr=rHrIr<rJmodel_save_kwargsfr@rOrEr#r#r$save_pretrained_kerassN#         rfKerasModelHubMixincOstj|i|S)a Instantiate a pretrained Keras model from a pre-trained model from the Hub. The model is expected to be in `SavedModel` format. Args: pretrained_model_name_or_path (`str` or `os.PathLike`): Can be either: - A string, the `model id` of a pretrained model hosted inside a model repo on huggingface.co. Valid model ids can be located at the root-level, like `bert-base-uncased`, or namespaced under a user or organization name, like `dbmdz/bert-base-german-cased`. - You can add `revision` by appending `@` at the end of model_id simply like this: `dbmdz/bert-base-german-cased@main` Revision is the specific model version to use. It can be a branch name, a tag name, or a commit id, since we use a git-based system for storing models and other artifacts on huggingface.co, so `revision` can be any identifier allowed by git. - A path to a `directory` containing model weights saved using [`~transformers.PreTrainedModel.save_pretrained`], e.g., `./my_model_directory/`. - `None` if you are both providing the configuration and state dictionary (resp. with keyword arguments `config` and `state_dict`). force_download (`bool`, *optional*, defaults to `False`): Whether to force the (re-)download of the model weights and configuration files, overriding the cached versions if they exist. resume_download (`bool`, *optional*, defaults to `False`): Whether to delete incompletely received files. Will attempt to resume the download if such a file exists. proxies (`Dict[str, str]`, *optional*): A dictionary of proxy servers to use by protocol or endpoint, e.g., `{'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}`. The proxies are used on each request. token (`str` or `bool`, *optional*): The token to use as HTTP bearer authorization for remote files. If `True`, will use the token generated when running `transformers-cli login` (stored in `~/.huggingface`). cache_dir (`Union[str, os.PathLike]`, *optional*): Path to a directory in which a downloaded pretrained model configuration should be cached if the standard cache should not be used. local_files_only(`bool`, *optional*, defaults to `False`): Whether to only look at local files (i.e., do not try to download the model). model_kwargs (`Dict`, *optional*): model_kwargs will be passed to the model during initialization Passing `token=True` is required when you want to use a private model. )rgZfrom_pretrained)r r!r#r#r$from_pretrained_kerass8rhz'Push Keras model using huggingface_hub.)rHcommit_messageprivate api_endpointtokenbranch create_prallow_patternsignore_patternsdelete_patternslog_dirrIrJr<repo_idrirjrkrlrmrnrorprqrrc Kst|d}|j|||ddj}tK}t||}t||f|| ||d|| durG| dur1gn t| tr9| gn| } | dt | |d|j d||||||| | | d WdS1s`wYdS) a Upload model checkpoint to the Hub. Use `allow_patterns` and `ignore_patterns` to precisely filter which files should be pushed to the hub. Use `delete_patterns` to delete existing remote files in the same commit. See [`upload_folder`] reference for more details. Args: model (`Keras.Model`): The [Keras model](`https://www.tensorflow.org/api_docs/python/tf/keras/Model`) you'd like to push to the Hub. The model must be compiled and built. repo_id (`str`): ID of the repository to push to (example: `"username/my-model"`). commit_message (`str`, *optional*, defaults to "Add Keras model"): Message to commit while pushing. private (`bool`, *optional*, defaults to `False`): Whether the repository created should be private. api_endpoint (`str`, *optional*): The API endpoint to use when pushing the model to the hub. token (`str`, *optional*): The token to use as HTTP bearer authorization for remote files. If not set, will use the token set when logging in with `huggingface-cli login` (stored in `~/.huggingface`). branch (`str`, *optional*): The git branch on which to push the model. This defaults to the default branch as specified in your repository, which defaults to `"main"`. create_pr (`boolean`, *optional*): Whether or not to create a Pull Request from `branch` with that commit. Defaults to `False`. config (`dict`, *optional*): Configuration object to be saved alongside the model weights. allow_patterns (`List[str]` or `str`, *optional*): If provided, only files matching at least one pattern are pushed. ignore_patterns (`List[str]` or `str`, *optional*): If provided, files matching any of the patterns are not pushed. delete_patterns (`List[str]` or `str`, *optional*): If provided, remote files matching any of the patterns will be deleted from the repo. log_dir (`str`, *optional*): TensorBoard logging directory to be pushed. The Hub automatically hosts and displays a TensorBoard instance if log files are included in the repository. include_optimizer (`bool`, *optional*, defaults to `False`): Whether or not to include optimizer during serialization. tags (Union[`list`, `str`], *optional*): List of tags that are related to model or string of a single tag. See example tags [here](https://github.com/huggingface/hub-docs/blob/main/modelcard.md?plain=1). plot_model (`bool`, *optional*, defaults to `True`): Setting this to `True` will plot the model and put it in the model card. Requires graphviz and pydot to be installed. model_save_kwargs(`dict`, *optional*): model_save_kwargs will be passed to [`tf.keras.models.save_model()`](https://www.tensorflow.org/api_docs/python/tf/keras/models/save_model). Returns: The url of the commit of your model in the given repository. )ZendpointT)rsrlrjrL)rHrIrJr<Nzlogs/*Zlogsr) Z repo_typersZ folder_pathrirlrevisionrnrorprq) rZ create_reporsrrrfr*r]r/rZ upload_folder)rrsrHrirjrkrlrmrnrorprqrrrIrJr<rdapitmpZ saved_pathr#r#r$push_to_hub_keras+sJ O   $rwc@s:eZdZdZddZe ddeeee ffddZ dS) rgaA Implementation of [`ModelHubMixin`] to provide model Hub upload/download capabilities to Keras models. ```python >>> import tensorflow as tf >>> from huggingface_hub import KerasModelHubMixin >>> class MyModel(tf.keras.Model, KerasModelHubMixin): ... def __init__(self, **kwargs): ... super().__init__() ... self.config = kwargs.pop("config", None) ... self.dummy_inputs = ... ... self.layer = ... ... def call(self, *args): ... return ... >>> # Initialize and compile the model as you normally would >>> model = MyModel() >>> model.compile(...) >>> # Build the graph by training it or passing dummy inputs >>> _ = model(model.dummy_inputs) >>> # Save model weights to local directory >>> model.save_pretrained("my-awesome-model") >>> # Push model weights to the Hub >>> model.push_to_hub("my-awesome-model") >>> # Download and initialize weights from the Hub >>> model = MyModel.from_pretrained("username/super-cool-model") ``` cCst||dSN)rf)selfr=r#r#r$_save_pretrainedsz#KerasModelHubMixin._save_pretrainedNrHc KsLtdurtdtj|st|||dtd} n|} tj| } | | _ | S)aHere we just call [`from_pretrained_keras`] function so both the mixin and functional APIs stay in sync. TODO - Some args above aren't used since we are calling snapshot_download instead of hf_hub_download. Nz>Called a TensorFlow-specific function but could not import it.r4)rsrt cache_dirrAZlibrary_version) r4rTrDrEisdirr r rcZ load_modelrH) clsZmodel_idrtr{Zforce_downloadproxiesZresume_downloadZlocal_files_onlyrlrHZ model_kwargsZstorage_folderrr#r#r$_from_pretraineds  z#KerasModelHubMixin._from_pretrainedrx) r __module__ __qualname____doc__rz classmethodr rr]rrr#r#r#r$rgs#  )r')TN)NFTN)rrg):collections.abcabcr+rZrDr_ functoolsrpathlibrshutilrtypingrrrr r Zhuggingface_hubr r Zhuggingface_hub.utilsr rrrr constantsrZhf_apirr;rrrZ utils._typingrZ get_loggerrloggerr4Ztf_kerasrTZ tensorflowtfr&r.r7r>boolr0rGr]r\rfrhrwrgr#r#r#r$s             ,  S;     y