o Zhh@sndZddlZddlmZddlmZmZmZm Z m Z GdddeZ d de e efd e d ee fd d ZdS)zDocument loader helpers.N)Path)List NamedTupleOptionalUnioncastc@s6eZdZUdZeeed< eed< eeed<dS) FileEncodingz File encoding as the NamedTuple.encoding confidencelanguageN)__name__ __module__ __qualname____doc__rstr__annotations__floatrrc/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/document_loaders/helpers.pyrs   r file_pathtimeoutreturnc sddlt|}dtdttffdd }tj'}|||}z|j|d}Wntjj y9t d|wWdn1sDwYt d d |DrYt d |d d |DS)aTry to detect the file encoding. Returns a list of `FileEncoding` tuples with the detected encodings ordered by confidence. Args: file_path: The path to the file to detect the encoding for. timeout: The timeout in seconds for the encoding detection. rNrrcsFt|d }|}Wdn1swYttt|S)Nrb)openreadrrdict detect_all)rfrawdatachardetrrread_and_detect#s  z.detect_file_encodings..read_and_detect)rz-Timeout reached while detecting encoding for css|] }|dduVqdS)r Nr).0r rrr 1sz(detect_file_encodings..zCould not detect encoding for cSs&g|]}|ddurtdi|qS)r Nr)r)r#encrrr 3s&z)detect_file_encodings..) r!rrr concurrentfuturesThreadPoolExecutorsubmitresult TimeoutErrorall RuntimeError)rrr"executorfuture encodingsrr rdetect_file_encodingss"    r2)r)rconcurrent.futuresr'pathlibrtypingrrrrrrrintr2rrrrs