Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

Duration

Input Tokens

3292

Output Tokens

362

Cost

$0.00

Context

Input

List all female characters mentioned

Expected output

Camila, Lucia, Marta, Jaya, Mum

Model output

The female characters mentioned in the journal entries are:

1. **Camila** - Don Álvaro's daughter who runs a photography studio in Cartagena and showed the narrator around the city
2. **Lucia** - A friend of a friend in Madrid who showed the narrator the Reina Sofía museum and various bars
3. **Marta** - A retired photographer in San Sebastián who had a long conversation with the narrator
4. **Jaya** - One of the narrator's friends who met them for drinks at The Eagle pub in Manchester
5. **Mum** - The narrator's mother, mentioned briefly in reference to her attic in Manchester