Meme Understanding

Evaluates a model’s ability to interpret culture-dependent, tricky, and humor-driven content that feels obvious to humans but is hard for AI.

100
Duration
8s
Input Tokens
265
Output Tokens
346
Cost
$0.00
Context
Input
Juan puts a coin inside a drawer while Sofia watches. Sofia then leaves the room and Juan moves the coin into a transparent bag. Sofia enters the room, sees the bag. Where does she search for the coin?
Expected output
{
  "location": "bag"
}
Model output
{
  "location": "transparent bag"
}