Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

Duration

Input Tokens

3201

Output Tokens

341

Cost

$0.00

Context

Input

List all female characters mentioned

Expected output

Camila, Lucia, Marta, Jaya, Mum

Model output

The female characters mentioned in the journal entries are:

1. **Camila** - Don Álvaro's daughter who runs a photography studio in Cartagena and shows the narrator around the city
2. **Lucia** - A friend of a friend who hosted the narrator in Madrid and showed them the Reina Sofía
3. **Marta** - A retired photographer the narrator had a conversation with in San Sebastián
4. **Mum** - The narrator's mother (mentioned when finding the journal in her attic)
5. **Jaya** - One of the narrator's friends who they meet for drinks at The Eagle pub in Manchester