o
Zho
@ sx d Z ddlmZmZ ddlmZmZmZmZ er ddl Z
eeddee df d e d
e d efd d
Z
ddd e fddZdS )z Contains numpy-specific helpers. )
TYPE_CHECKINGDict )FILENAME_PATTERNMAX_SHARD_SIZEStateDictSplit$split_state_dict_into_shards_factoryN)filename_patternmax_shard_size
state_dictz
np.ndarrayr r
returnc C s t | ||tdS )aG
Split a model state dictionary in shards so that each shard is smaller than a given size.
The shards are determined by iterating through the `state_dict` in the order of its keys. There is no optimization
made to make each shard as close as possible to the maximum size passed. For example, if the limit is 10GB and we
have tensors of sizes [6GB, 6GB, 2GB, 6GB, 2GB, 2GB] they will get sharded as [6GB], [6+2GB], [6+2+2GB] and not
[6+2+2GB], [6+2GB], [6GB].
If one of the model's tensor is bigger than `max_shard_size`, it will end up in its own shard which will have a
size greater than `max_shard_size`.
Args:
state_dict (`Dict[str, np.ndarray]`):
The state dictionary to save.
filename_pattern (`str`, *optional*):
The pattern to generate the files names in which the model will be saved. Pattern must be a string that
can be formatted with `filename_pattern.format(suffix=...)` and must contain the keyword `suffix`
Defaults to `"model{suffix}.safetensors"`.
max_shard_size (`int` or `str`, *optional*):
The maximum size of each shard, in bytes. Defaults to 5GB.
Returns:
[`StateDictSplit`]: A `StateDictSplit` object containing the shards and the index to retrieve them.
)r
r get_tensor_size)r r
)r r r
r [/var/www/html/lang_env/lib/python3.10/site-packages/huggingface_hub/serialization/_numpy.py"split_numpy_state_dict_into_shards s "r tensorc C s | j S )N)nbytes)r r r r r
C s r
)__doc__typingr r _baser r r r numpynpstrintr r
r r r r s"
*