Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

Duration

Input Tokens

3200

Output Tokens

211

Cost

$0.00

Context

Input

List all female characters mentioned

Expected output

Camila, Lucia, Marta, Jaya, Mum

Model output

Female characters mentioned in the text:

1. Camila - Don Álvaro's daughter who runs a photography studio in Cartagena and showed the narrator around the city
2. Lucia - A friend of a friend in Madrid who showed the narrator the Reina Sofía and various bars
3. Marta - A retired photographer the narrator had a long conversation with in San Sebastián
4. Jaya - Someone the narrator met for drinks at The Eagle in Manchester
5. The narrator's Mum (mother) - Mentioned when the narrator finds the journal in her attic