o Zh @spddlmZmZmZmZddlmZddlmZddl m Z m Z ddl m Z edddd Gd d d e e Zd S) )AnyCallableDictList) deprecated)Document) BaseModelmodel_validator) BaseLoaderz0.3.18zThis class is deprecated and will be removed in a future version. You can swap to using the `ApifyDatasetLoader` implementation in `langchain_apify` package. See z"langchain_apify.ApifyDatasetLoader)ZsincemessageZalternative_importcseZdZUdZeed< eed< eege fed< dedeege fffdd Z e dde d ed efd d Z d ee fd dZZS)ApifyDatasetLoaderaLoad datasets from `Apify` web scraping, crawling, and data extraction platform. For details, see https://docs.apify.com/platform/integrations/langchain Example: .. code-block:: python from langchain_community.document_loaders import ApifyDatasetLoader from langchain_core.documents import Document loader = ApifyDatasetLoader( dataset_id="YOUR-DATASET-ID", dataset_mapping_function=lambda dataset_item: Document( page_content=dataset_item["text"], metadata={"source": dataset_item["url"]} ), ) documents = loader.load() apify_client dataset_iddataset_mapping_functioncstj||ddS)arInitialize the loader with an Apify dataset ID and a mapping function. Args: dataset_id (str): The ID of the dataset on the Apify platform. dataset_mapping_function (Callable): A function that takes a single dictionary (an Apify dataset item) and converts it to an instance of the Document class. )rrN)super__init__)selfrr __class__i/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/document_loaders/apify_dataset.pyr0s  zApifyDatasetLoader.__init__before)modevaluesreturncCsZz!ddlm}|}t|jd}r|jdd7<||d<W|Sty,tdw)zYValidate environment. Args: values: The values to validate. r) ApifyClient httpx_clientz user-agentz; Origin/langchainr z`Could not import apify-client Python package. Please install it with `pip install apify-client`.)r rgetattr http_clientheaders ImportError)clsrrclientrrrrvalidate_environment?s   z'ApifyDatasetLoader.validate_environmentcCs(|j|jjddj}tt|j|S)zLoad documents.T)clean)r ZdatasetrZ list_itemsitemslistmapr)rZ dataset_itemsrrrloadXszApifyDatasetLoader.load)__name__ __module__ __qualname____doc__r__annotations__strrrrrr classmethodr#rr( __classcell__rrrrr s"   r N)typingrrrrZlangchain_core._apirZlangchain_core.documentsrZpydanticrr Z)langchain_community.document_loaders.baser r rrrrs