Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

Duration

15s

Input Tokens

7761

Output Tokens

458

Cost

$0.00

Context

Input

Which patient talked on the phone with a doctor, and what were their symptoms?

Expected output

Melisa Farrow, who had 38.5°C fever, cough, her left side hurts, has pain when breathing and feels dizzy

Model output

Based on the provided patient journals, Melisa Farrow spoke on the phone, but she talked with a nurse rather than a doctor. During this phone call on the morning of 2025-05-20, Melisa reported the following symptoms: chills, increased pain when breathing, aching on her left side when coughing, dizziness, and a fever of 38.5°C. This call resulted in her being told to come in immediately, and she was subsequently diagnosed with pneumonia and admitted to the hospital.