o Zhk@szddlZddlZddlZddlmZmZddlZddlmZddl m Z e dZ e dZ e dZGdd d e ZdS) N)ListTuple)Document) BaseLoaderzBV\w+zav[0-9]+zp=(\d+)c @sbeZdZdZ   ddeedededefddZd eefd d Zd ed e ee ffd dZ dS)BiliBiliLoaderz9 Load fetching transcripts from BiliBili videos. video_urlssessdatabili_jctbuvid3cCsb||_d|_zddlm}Wn tytdw|r+|r-|r/|j|||d|_dSdSdSdS)a Initialize the loader with BiliBili video URLs and authentication cookies. if no authentication cookies are provided, the loader can't get transcripts and will only fetch videos info. Args: video_urls (List[str]): List of BiliBili video URLs. sessdata (str): SESSDATA cookie value for authentication. bili_jct (str): BILI_JCT cookie value for authentication. buvid3 (str): BUVI3 cookie value for authentication. Nr)videoTrequests package not found, please install it with `pip install bilibili-api-python`)r r r )r credential bilibili_apir ImportErrorZ Credential)selfrr r r r rd/var/www/html/lang_env/lib/python3.10/site-packages/langchain_community/document_loaders/bilibili.py__init__s   zBiliBiliLoader.__init__returncCs8g}|jD]}||\}}t||d}||q|S)z Load and return a list of documents containing video transcripts. Returns: List[Document]: List of Document objects transcripts and metadata. )Z page_contentmetadata)r_get_bilibili_subs_and_inforappend)rresultsurlZ transcript video_infodocrrrload6s    zBiliBiliLoader.loadrcCst|}z ddlm}m}Wn tytdw|r(|j||jd}nt |}|r@|jt |dd|jd}nt d||| }| d |i|js[d |fSd}t|} | rt|d t | d d d }n|d }|||} | dg} | r| ddd } | dsd| } t| } | jdkrt| jdg}ddd|D}d|dd|dd|}||fStd|d| jd |fStd|dd |fS) zU Retrieve video information and transcript for a given BiliBili URL. r)syncr r )bvidrN)aidrz(Unable to find a valid video ID in URL: rrZpagescidZ subtitlesZ subtitle_urlhttpzhttps:body cSsg|]}|dqS)contentr).0crrr usz>BiliBiliLoader._get_bilibili_subs_and_info..z Video Title: titlez, description: Zdescz Transcript: zFailed to fetch subtitles for z. HTTP Status Code: zNo subtitles found for video: z. Returning empty transcript.) BV_PATTERNsearchrrr rZVideogroupr AV_PATTERNint ValueErrorget_infoupdatePAGE_INDEX_PATTERNZ get_subtitleget startswithrequests status_codejsonloadsr(joinwarningswarn)rrrrr vr!rr#Z page_matchsubZsub_listZsub_urlresponseZraw_sub_titlesZraw_transcriptZraw_transcript_with_meta_inforrrrEsh   "         z*BiliBiliLoader._get_bilibili_subs_and_infoN)rrr) __name__ __module__ __qualname____doc__rstrrrrrdictrrrrrrs   r)r:rer=typingrrr8Zlangchain_core.documentsrZ)langchain_community.document_loaders.basercompiler-r0r5rrrrrs