Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

100
Duration
2s
Input Tokens
3184
Output Tokens
532
Cost
$0.00
Context
Input
What message did Oscar send to Dr. Nyström after recovering?
Expected output
He thanked her, said he was feeling much better, had resumed running, and would not need another visit for now
Model output
Oscar sent a message thanking Dr. Nyström, saying the medication had helped a lot, that he had been feeling much better, even ran without issues, will keep using the inhaler, does not need another visit for now, and will reach out if anything changes. He expressed his appreciation for the help.