o Zh @sddlmZddlZddlZddlmZmZddlmZddl m Z ddl m Z ddl mZmZejr9ddlmZeeZe d d d gd d ddgdidddgdidddgdidddgdidgdZeGdddeZeddZdS)) annotationsN) dataclassfield)Dict) json_loader)Prompt)EvaluationMode MetricWithLLM) CallbacksZtext_entity_extractionzGiven a text, extract unique entities without repetition. Ensure you consider different forms or mentions of the same entity as a single entity.textoutputjsonaThe Eiffel Tower, located in Paris, France, is one of the most iconic landmarks globally. Millions of visitors are attracted to it each year for its breathtaking views of the city. Completed in 1889, it was constructed in time for the 1889 World's Fair.entities)z Eiffel TowerZParisZFranceZ1889z World's Fair)r r anThe Colosseum in Rome, also known as the Flavian Amphitheatre, stands as a monument to Roman architectural and engineering achievement. Construction began under Emperor Vespasian in AD 70 and was completed by his son Titus in AD 80. It could hold between 50,000 and 80,000 spectators who watched gladiatorial contests and public spectacles.)Z ColosseumZRomezFlavian AmphitheatreZ VespasianzAD 70ZTituszAD 80aLThe Great Wall of China, stretching over 21,196 kilometers from east to west, is a marvel of ancient defensive architecture. Built to protect against invasions from the north, its construction started as early as the 7th century BC. Today, it is a UNESCO World Heritage Site and a major tourist attraction.)zGreat Wall of Chinaz21,196 kilometersz7th century BCzUNESCO World Heritage SiteaJThe Apollo 11 mission, which launched on July 16, 1969, marked the first time humans landed on the Moon. Astronauts Neil Armstrong, Buzz Aldrin, and Michael Collins made history, with Armstrong being the first man to step on the lunar surface. This event was a significant milestone in space exploration.)zApollo 11 missionz July 16, 1969ZMoonzNeil Armstrongz Buzz AldrinzMichael Collins)nameZ instructionZ input_keysZ output_key output_typeZexamplesc@sxeZdZUdZdZded<ejZded<e ddd Z d ed <d Z d ed<d'ddZ d(ddZ d)dd Zd*d+d%d&Zd!S),ContextEntityRecallu Calculates recall based on entities present in ground truth and context. Let CN be the set of entities present in context, GN be the set of entities present in the ground truth. Then we define can the context entity recall as follows: Context Entity recall = | CN ∩ GN | / | GN | If this quantity is 1, we can say that the retrieval mechanism has retrieved context which covers all entities present in the ground truth, thus being a useful retrieval. Thus this can be used to evaluate retrieval mechanisms in specific use cases where entities matter, for example, a tourism help chatbot. Attributes ---------- name : str batch_size : int Batch size for openai completion. context_entity_recallstrrrevaluation_modecCstSN)TEXT_ENTITY_EXTRACTIONrr]/var/www/html/lang_env/lib/python3.10/site-packages/ragas/metrics/_context_entities_recall.pyjszContextEntityRecall.)default_factoryrcontext_entity_recall_promptint batch_sizeground_truth_entitiessetcontext_entitiesreturnfloatcCs&tt|t|}|t|dS)Ng:0yE>)lenr intersection)selfrr!Znum_entities_in_bothrrr_compute_scorensz"ContextEntityRecall._compute_scorer callbacksr is_asyncboolrcst|jdus Jd|jj|jj|d||dIdH}tj|jddj|j|dIdH}t|t r6|}|Si}|S)NzLLM is not initialized)r )promptr(r)r)r)) ZllmgeneraterformatrZ safe_loadZ generationsr isinstancedict)r&r r(r)resultZ response_dictrrr get_entitiesvs   z ContextEntityRecall.get_entitiesrowcsb|d|d}}|j|||dIdH}|jd|||dIdH}||dg|dgS)N ground_truthcontexts)r(r) r)r1joinr'get)r&r2r(r)r3r4rrr_ascores   zContextEntityRecall._ascoreN cache_dir str | NoneNonecCs |j|Sr)rsave)r&r9rrrr<s zContextEntityRecall.save)rr r!r r"r#)r rr(r r)r*r"r)r2rr(r r)r*r"r#r)r9r:r"r;)__name__ __module__ __qualname____doc__r__annotations__rgcrrrrr'r1r8r<rrrrrPs      rr)r) __future__rloggingtypingt dataclassesrrrZragas.llms.json_loadrZragas.llms.promptrZragas.metrics.baserr TYPE_CHECKINGZlangchain.callbacks.baser getLoggerr=loggerrrrrrrrsL       ?O