Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

Duration

Input Tokens

7761

Output Tokens

168

Cost

$0.00

Context

Input

Which patient talked on the phone with a doctor, and what were their symptoms?

Expected output

Melisa Farrow, who had 38.5°C fever, cough, her left side hurts, has pain when breathing and feels dizzy

Model output

For Oscar Jansson, the symptoms were breathing-related: mild breathing difficulty, extra effort to get a full breath, mild chest tightness, occasional dry cough, and mild pressure. He was ultimately diagnosed with mild persistent asthma.

For Melisa Farrow, the symptoms were a persistent cough, chest pain when lying down, slight shortness of breath, and eventually a fever. She was diagnosed with community-acquired pneumonia in her left lower lung lobe.