Context Reasoning

Context understanding and reasoning tasks test accurate answers grounded in provided context. This capability is essential for knowledge-base support bots, policy lookup systems, and internal knowledge Q&A applications. Models are evaluated on their ability to provide accurate answers that are properly grounded in the given context rather than hallucinating information.

0
Duration
18s
Input Tokens
3088
Output Tokens
188
Cost
$0.00
Context
Input
How many cities does the author mention
Expected output
19
Model output
The author mentions the following cities:

From Colombia:
* Cartagena
* Leticia

From Spain:
* Bilbao
* Granada
* Córdoba
* Sevilla
* Cádiz
* Málaga
* Jaén
* Toledo
* Madrid
* Ávila
* Salamanca
* Zamora
* San Sebastián
* Valencia

From England:
* Manchester

From Morocco:
* Tangier
* Chefchaouen

In total, the author mentions 20 cities.