Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

Duration

Input Tokens

7462

Output Tokens

276

Cost

$0.00

Context

Input

Which patient talked on the phone with a doctor, and what were their symptoms?

Expected output

Melisa Farrow, who had 38.5°C fever, cough, her left side hurts, has pain when breathing and feels dizzy

Model output

Oscar Jansson talked to Dr. Helena Nyström. His initial symptoms included "weird breathing issues" described as taking "extra effort to get a full breath in," with tightness in his chest but not pain. He also reported occasional dry cough in the evenings. Later, he developed a mild fever (37.9°C at home), tiredness, body aches, occasional dry cough, persistent chest tightness, a short episode of wheezing during a run, and difficulty sleeping when lying flat.

Melisa Farrow talked to Dr. Arvid Lindholm. Her symptoms included an annoying cough for a few days, and a bit of chest pain when lying down, described as pressure that worsens with coughing. She also reported slight shortness of breath after walking fast up stairs. After an initial visit, her condition worsened, and she reported chills, increased pain when breathing, left-sided aches when coughing, dizziness, and a fever of 38.5°C.