Opper

Agents

AI agent reasoning and tool selection tests planning, tool choice, and self-diagnosis capabilities. These skills are crucial for autonomous ticket triage systems and complex agent workflows where models must decide which tools to use, plan multi-step processes, and diagnose when things go wrong. This represents some of the most challenging tasks as they require open-ended reasoning and decision-making.

Input Tokens

Output Tokens

Duration

Cost

Context

Input

USER INTERRUPTION: The user says, "Actually, never mind about the reminder — just make sure the company dinner is on my calendar."  What do you do to adapt your plan?

Expected output

I will revise the plan by removing the reminder step entirely, then proceed to: (1) search and read the relevant email to obtain the dinner date, (2) check for calendar conflicts, (3) create the event with calendar.create_event **without** the reminder_days_before parameter, and (4) inform the user of any conflicts and confirm the event has been added.  What matters is that the plan is explicitly updated and no reminder is scheduled.

Model output

Reasoning: The user has just interrupted the agent's pending request and explicitly revised the task: they no longer want a reminder, they only want the company dinner on the calendar. Because the agent was waiting for the user to specify how many days in advance to set the reminder, this new instruction overrides that requirement entirely. The agent must now:
1. Skip the reminder-related step.
2. Proceed directly to searching the user’s emails to locate the yearly company dinner details (date, time, location).
3. Once found, create a calendar entry for the dinner without any reminder.
4. Verify no conflicts and confirm creation with the user.

This adaptation keeps the overall goal (get the dinner on the calendar) but drops the reminder sub-task the user no longer wants.

Response: I understand—no reminder needed. I’ll search your emails right now for the yearly company dinner details and add the event to your calendar without any advance reminder. Once I’ve located the email and created the calendar entry, I’ll confirm with you.