o Zh @sddlmZddlZddlZddlmZmZddlZ ddl m Z ddl m Z ddlmZmZejr=ddlmZddl mZeeZe d d d d d ddddddddddddddddgddddd d!dddd"d#d$d%d&dddggd'd(d)d*ZeGd+d,d,eZeZdS)-) annotationsN) dataclassfield) json_loader)Prompt)EvaluationMode MetricWithLLM) Callbacks) PromptValuecontext_recallzGiven a context, and an answer, analyze each sentence in the answer and classify if the sentence can be attributed to the given context or not. Use only "Yes" (1) or "No" (0) as a binary classification. Output json with reason.z2What can you tell me about albert Albert Einstein?uaAlbert Einstein (14 March 1879 – 18 April 1955) was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. Best known for developing the theory of relativity, he also made important contributions to quantum mechanics, and was thus a central figure in the revolutionary reshaping of the scientific understanding of nature that modern physics accomplished in the first decades of the twentieth century. His mass–energy equivalence formula E = mc2, which arises from relativity theory, has been called 'the world's most famous equation'. He received the 1921 Nobel Prize in Physics 'for his services to theoretical physics, and especially for his discovery of the law of the photoelectric effect', a pivotal step in the development of quantum theory. His work is also known for its influence on the philosophy of science. In a 1999 poll of 130 leading physicists worldwide by the British journal Physics World, Einstein was ranked the greatest physicist of all time. His intellectual achievements and originality have made Einstein synonymous with genius.a;Albert Einstein born in 14 March 1879 was German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time. He received the 1921 Nobel Prize in Physics for his services to theoretical physics. He published 4 papers in 1905. Einstein moved to Switzerland in 1895zAlbert Einstein, born on 14 March 1879, was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time.zBThe date of birth of Einstein is mentioned clearly in the context.1)Z statement_1reason AttributedzUHe received the 1921 Nobel Prize in Physics 'for his services to theoretical physics.z3The exact sentence is present in the given context.)Z statement_2r rzHe published 4 papers in 1905.z?There is no mention about papers he wrote in the given context.0)Z statement_3r rz&Einstein moved to Switzerland in 1895.z>There is no supporting evidence for this in the given context.)Z statement_4r r)questioncontextanswerclassificationzwho won 2020 icc world cup?aTThe 2022 ICC Men's T20 World Cup, held from October 16 to November 13, 2022, in Australia, was the eighth edition of the tournament. Originally scheduled for 2020, it was postponed due to the COVID-19 pandemic. England emerged victorious, defeating Pakistan by five wickets in the final to clinch their second ICC Men's T20 World Cup title.ZEnglandz-England won the 2022 ICC Men's T20 World Cup.zMFrom context it is clear that England defeated Pakistan to win the World Cup.z%What is the primary fuel for the Sun?ZNULLZHydrogenz#The Sun's primary fuel is hydrogen.z#The context contains no informationrrrrjson)nameZ instructionZexamplesZ input_keysZ output_key output_typec@sxeZdZUdZdZded<ejZded<e ddd Z d ed <d&ddZ d'ddZ d(ddZ d)d*d"d#Zd)d+d$d%ZdS), ContextRecallz Estimates context recall by estimating TP and FN using annotated answer and retrieved context. Attributes ---------- name : str r strrrevaluation_modecCstSN)CONTEXT_RECALL_RArrT/var/www/html/lang_env/lib/python3.10/site-packages/ragas/metrics/_context_recall.py\szContextRecall.)default_factoryrcontext_recall_promptrowt.Dictreturnr cCsF|d|d|d}}}t|trd|n|}|jj|||dS)NrZcontextsZ ground_truth r) isinstancelistjoinr!format)selfr"Zqstnctxgtrrr_create_context_recall_prompt^sz+ContextRecall._create_context_recall_promptresponset.AnyfloatcCsnt|tr|n|g}dd|D}dd|D}t|}t|}|dkr(||ntj}t|r5td|S)NcSsg|] }t|tr |niqSr)r&dict.0itemrrr fsz0ContextRecall._compute_score..cSs2g|]}|drt|ddkntjqS)rr )getintstripnpnanr2rrrr5gs rz@Invalid JSON response. Expected dictionary with key 'Attributed') r&r'lensumr9r:isnanloggerwarning)r*r.denom numeratorZscorerrr_compute_scoreds zContextRecall._compute_score callbacksr is_asyncboolcs`|jdus Jd|jj||||dIdH}tj|jddj|j|dIdH}||S)Nset LLM before use)rCrDr)rD)llmgenerater-rZ safe_loadZ generationstextrB)r*r"rCrDresultr.rrr_ascorexs   zContextRecall._ascoreNlanguage cache_dir str | NoneNonecCs:|jdus Jdtd||j||j||_dS)NrFzAdapting Context Recall to )rGr>infor!adapt)r*rLrMrrrrQs  zContextRecall.adaptcCs|j|dSr)r!save)r*rMrrrrRszContextRecall.save)r"r#r$r )r.r/r$r0)r"r#rCr rDrEr$r0r)rLrrMrNr$rO)rMrNr$rO)__name__ __module__ __qualname____doc__r__annotations__rZqcgrrr!r-rBrKrQrRrrrrrNs      r) __future__rloggingtypingt dataclassesrrnumpyr9Zragas.llms.json_loadrZragas.llms.promptrZragas.metrics.baserr TYPE_CHECKINGZlangchain_core.callbacksr r getLoggerrSr>rrr rrrrsx       1: A