o Zh @sdZddlmZmZddlmZmZmZmZm Z m Z ddl m Z e dZ ee gefZee ge efZdZdZe eZeGd d d Zd d eed deee fdededededef ddZdS)z.Contains helpers to split tensors into shards.) dataclassfield)AnyCallableDictListOptionalTypeVar)loggingTensorTlr Tzmodel{suffix}.safetensorsc@s\eZdZUeddZeed<eee fed<eee efed<eeefed<ddZ d S) StateDictSplitF)init is_shardedmetadatafilename_to_tensorstensor_to_filenamecCst|jdk|_dS)N)lenrr)selfrZ/var/www/html/lang_env/lib/python3.10/site-packages/huggingface_hub/serialization/_base.py __post_init__'szStateDictSplit.__post_init__N) __name__ __module__ __qualname__rrbool__annotations__rstrrrrrrrrr s  r cCsdS)Nr)tensorrrr/sr )get_storage_idfilename_patternmax_shard_size state_dictget_tensor_sizer!r"r#returncsi}g}i}d}d} |D]X\} } t| trtd| q|| } | dur7| |vr2|| | q| g|| <|| } | |krK| | 7} || | iq|| |krZ||i}d}| || <|| 7}| | 7} qt|dkrr||t|}|D]\} }|D]}|d|vr|D]} || || <qnqqz|dkr|jddtd| it | ifdd | Dd Si}i}t |D]%\}}|jd |dd d |d d|D]} || <qt | |<qtd| i||d S)a\ Split a model state dictionary in shards so that each shard is smaller than a given size. The shards are determined by iterating through the `state_dict` in the order of its keys. There is no optimization made to make each shard as close as possible to the maximum size passed. For example, if the limit is 10GB and we have tensors of sizes [6GB, 6GB, 2GB, 6GB, 2GB, 2GB] they will get sharded as [6GB], [6+2GB], [6+2+2GB] and not [6+2+2GB], [6+2GB], [6GB]. If one of the model's tensor is bigger than `max_shard_size`, it will end up in its own shard which will have a size greater than `max_shard_size`. Args: state_dict (`Dict[str, Tensor]`): The state dictionary to save. get_tensor_size (`Callable[[Tensor], int]`): A function that returns the size of a tensor in bytes. get_storage_id (`Callable[[Tensor], Optional[Any]]`, *optional*): A function that returns a unique identifier to a tensor storage. Multiple different tensors can share the same underlying storage. This identifier is guaranteed to be unique and constant for this tensor's storage during its lifetime. Two tensor storages with non-overlapping lifetimes may have the same id. filename_pattern (`str`, *optional*): The pattern to generate the files names in which the model will be saved. Pattern must be a string that can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix` Defaults to `"model{suffix}.safetensors"`. max_shard_size (`int` or `str`, *optional*): The maximum size of each shard, in bytes. Defaults to 5GB. Returns: [`StateDictSplit`]: A `StateDictSplit` object containing the shards and the index to retrieve them. rz8Skipping tensor %s as it is a string (bnb serialization)Nr)suffix total_sizecsi|]}|qSrr).0keyfilenamerr sz8split_state_dict_into_shards_factory..)rrr-Z05dz-of-) items isinstancerloggerinfoappendrformatr listkeys enumerate)r$r%r!r"r#Zstorage_id_to_tensorsZ shard_listZ current_shardZcurrent_shard_sizer)r+rZ storage_idZ tensor_sizeZ nb_shardsr7ZshardZtensor_name_to_filenameridxrr,r$split_state_dict_into_shards_factory+sr*            r:N)__doc__ dataclassesrrtypingrrrrrr r'r r intZTensorSizeFn_TZ StorageIDFn_TZMAX_SHARD_SIZEZFILENAME_PATTERNZ get_logger__file__r2r rr:rrrrs8