Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

0
Duration
3s
Input Tokens
6706
Output Tokens
732
Cost
$0.00
Context
Input
Which patient talked on the phone with a doctor, and what were their symptoms?
Expected output
Melisa Farrow, who had 38.5°C fever, cough, her left side hurts, has pain when breathing and feels dizzy
Model output
Melisa Farrow talked on the phone with her nurse on 2025-05-20 due to worsening symptoms, including fever (38.5°C), chills, dizziness, and increased pleuritic chest pain. Oscar Jansson only sent an SMS to his doctor but did not explicitly discuss worsening symptoms during a phone call.