o
TZh @ sj d dl Z d dlZd dlmZmZmZ d dlmZ d dlm Z m
Z
m Z m Z ddl
mZ eG dd dZdS ) N)InitVar dataclassfield)Path)AnyDictOptionalUnion )configc sF e Zd ZU dZdZeeeef e d< dZ
e e d< dZ e e d< dZ
e e d< dZee e d< dZee e d < dZe e d
< dZe e d < dZe e d < d
Ze e d< dZee e d< dZee d< dZeeee f e d< dZeeeee f e d< dZe e d< e e dZ eee f e d< dZ ee e d< dd Z!d d d Z" fd d Z# Z$S )!DownloadConfiga
Configuration for our cached path manager.
Attributes:
cache_dir (`str` or `Path`, *optional*):
Specify a cache directory to save the file to (overwrite the
default cache dir).
force_download (`bool`, defaults to `False`):
If `True`, re-dowload the file even if it's already cached in
the cache dir.
resume_download (`bool`, defaults to `False`):
If `True`, resume the download if an incompletely received file is
found.
proxies (`dict`, *optional*):
user_agent (`str`, *optional*):
Optional string or dict that will be appended to the user-agent on remote
requests.
extract_compressed_file (`bool`, defaults to `False`):
If `True` and the path point to a zip or tar file,
extract the compressed file in a folder along the archive.
force_extract (`bool`, defaults to `False`):
If `True` when `extract_compressed_file` is `True` and the archive
was already extracted, re-extract the archive and override the folder where it was extracted.
delete_extracted (`bool`, defaults to `False`):
Whether to delete (or keep) the extracted files.
use_etag (`bool`, defaults to `True`):
Whether to use the ETag HTTP response header to validate the cached files.
num_proc (`int`, *optional*):
The number of processes to launch to download the files in parallel.
max_retries (`int`, default to `1`):
The number of times to retry an HTTP request if it fails.
token (`str` or `bool`, *optional*):
Optional string or boolean to use as Bearer token
for remote files on the Datasets Hub. If `True`, or not specified, will get token from `~/.huggingface`.
use_auth_token (`str` or `bool`, *optional*):
Optional string or boolean to use as Bearer token
for remote files on the Datasets Hub. If `True`, or not specified, will get token from `~/.huggingface`.
`use_auth_token` was deprecated in favor of `token` in version 2.14.0 and will be removed in 3.0.0.
ignore_url_params (`bool`, defaults to `False`):
Whether to strip all query parameters and fragments from
the download URL before using it for caching the file.
storage_options (`dict`, *optional*):
Key/value pairs to be passed on to the dataset file-system backend, if any.
download_desc (`str`, *optional*):
A description to be displayed alongside with the progress bar while downloading the files.
N cache_dirFforce_downloadresume_downloadlocal_files_onlyproxies
user_agentextract_compressed_file
force_extractdelete_extractedTuse_etagnum_proc max_retriestoken
deprecateduse_auth_tokenignore_url_params)default_factorystorage_options
download_descc C sH |dkrt d| dt || _d| jvr"| jtjd| jd< d S d S )Nr z'use_auth_token' was deprecated in favor of 'token' in version 2.14.0 and will be removed in 3.0.0.
You can remove this warning by passing 'token=z
' instead.hfr Zendpoint)warningswarn
FutureWarningr r r HF_ENDPOINT)selfr r( X/var/www/html/lang_env/lib/python3.10/site-packages/datasets/download/download_config.py
__post_init__R s
z DownloadConfig.__post_init__returnc C s | j di dd | j D S )Nc S s i | ]
\}}|t |qS r( )copydeepcopy).0kvr( r( r)
^ s z'DownloadConfig.copy..r( ) __class____dict__items)r' r( r( r) r, ] s zDownloadConfig.copyc sl |dkr-t | dd d ur-d| jvr|tjd| jd< nt | jd dd d u r-|| jd d< t || d S )Nr r r! r" )getattrr r r& super __setattr__)r' namevaluer2 r( r) r7 ` s
zDownloadConfig.__setattr__)r+ r )%__name__
__module__ __qualname____doc__r
r r strr __annotations__r boolr r r r r r r r r r intr r r r r r dictr r r r* r, r7
__classcell__r( r( r: r) r
s,
4
r )r, r# dataclassesr r r pathlibr typingr r r r r r r( r( r( r) s