Meme Understanding

Evaluates a model’s ability to interpret culture-dependent, tricky, and humor-driven content that feels obvious to humans but is hard for AI.

0
Duration
4s
Input Tokens
363
Output Tokens
64
Cost
$0.00
Context
Input
Repeat back exactly this text (including spacing/punctuation):  Whitespace before and after
Expected output
{
  "repeated_text": " Whitespace before and after "
}
Model output
{
  "repeated_text": "Repeat back exactly this text (including spacing/punctuation):  Whitespace before and after <budget:token_budget>200000</budget:token_budget>"
}