o Zh @sddlZddlZddlmZddlmZmZddlmZm Z m Z m Z m Z e Ze Ze dZGdddeZGdd d eZeGd d d ZeGd d d ZeGdddZdS)N) defaultdict) dataclassfield)DictListLiteralOptionalTuple) ZF64ZF32ZF16ZBF16ZI64ZI32ZI16ZI8ZU8ZBOOLc@eZdZdZdS)SafetensorsParsingErrorzRaised when failing to parse a safetensors file metadata. This can be the case if the file is not a safetensors file or does not respect the specification. N__name__ __module__ __qualname____doc__rrY/var/www/html/lang_env/lib/python3.10/site-packages/huggingface_hub/utils/_safetensors.pyr r c@r )NotASafetensorsRepoErrorzRaised when a repo is not a Safetensors repo i.e. doesn't have either a `model.safetensors` or a `model.safetensors.index.json` file. Nr rrrrrrrc@sReZdZUdZeed<eeed<eeefed<e ddZ eed<d d d Z d S) TensorInfoa<Information about a tensor. For more details regarding the safetensors format, check out https://huggingface.co/docs/safetensors/index#format. Attributes: dtype (`str`): The data type of the tensor ("F64", "F32", "F16", "BF16", "I64", "I32", "I16", "I8", "U8", "BOOL"). shape (`List[int]`): The shape of the tensor. data_offsets (`Tuple[int, int]`): The offsets of the data in the file as a tuple `[BEGIN, END]`. parameter_count (`int`): The number of parameters in the tensor. dtypeshapeZ data_offsetsFinitparameter_countreturnNcCs4z ttj|j|_WdStyd|_YdSw)N) functoolsreduceoperatormulrr TypeError)selfrrr __post_init__0s   zTensorInfo.__post_init__rN) r rrrDTYPE_T__annotations__rintr rrr#rrrrrs  rc@sVeZdZUdZeeefed<eeefed<e ddZ ee e fed<d d d Z dS) SafetensorsFileMetadataaMetadata for a Safetensors file hosted on the Hub. This class is returned by [`parse_safetensors_file_metadata`]. For more details regarding the safetensors format, check out https://huggingface.co/docs/safetensors/index#format. Attributes: metadata (`Dict`): The metadata contained in the file. tensors (`Dict[str, TensorInfo]`): A map of all tensors. Keys are tensor names and values are information about the corresponding tensor, as a [`TensorInfo`] object. parameter_count (`Dict[str, int]`): A map of the number of parameters per data type. Keys are data types and values are the number of parameters of that data type. metadatatensorsFrrrNcCs:tt}|jD] }||j|j7<q t||_dSN)rr'r*valuesrrdict)r"rZtensorrrrr#Osz%SafetensorsFileMetadata.__post_init__r$)r rrrrstrr& TENSOR_NAME_Trrrr%r'r#rrrrr(8s r(c@sjeZdZUdZeeed<eed<eee fed<ee e fed<e ddZ ee efed<d d d Zd S)SafetensorsRepoMetadataa Metadata for a Safetensors repo. A repo is considered to be a Safetensors repo if it contains either a 'model.safetensors' weight file (non-shared model) or a 'model.safetensors.index.json' index file (sharded model) at its root. This class is returned by [`get_safetensors_metadata`]. For more details regarding the safetensors format, check out https://huggingface.co/docs/safetensors/index#format. Attributes: metadata (`Dict`, *optional*): The metadata contained in the 'model.safetensors.index.json' file, if it exists. Only populated for sharded models. sharded (`bool`): Whether the repo contains a sharded model or not. weight_map (`Dict[str, str]`): A map of all weights. Keys are tensor names and values are filenames of the files containing the tensors. files_metadata (`Dict[str, SafetensorsFileMetadata]`): A map of all files metadata. Keys are filenames and values are the metadata of the corresponding file, as a [`SafetensorsFileMetadata`] object. parameter_count (`Dict[str, int]`): A map of the number of parameters per data type. Keys are data types and values are the number of parameters of that data type. r)ZshardedZ weight_mapfiles_metadataFrrrNcCsJtt}|jD]}|jD] \}}|||7<qq t||_dSr+)rr'r1r,ritemsr-)r"rZ file_metadatarZnb_parameters_rrrr#ws z%SafetensorsRepoMetadata.__post_init__r$)r rrrrrr&boolr/ FILENAME_Tr(rrr%r'r#rrrrr0Vs  r0)rr collectionsr dataclassesrrtypingrrrrr r.r4r/r% Exceptionr rrr(r0rrrrs