Agents

AI agent reasoning and tool selection tests planning, tool choice, and self-diagnosis capabilities. These skills are crucial for autonomous ticket triage systems and complex agent workflows where models must decide which tools to use, plan multi-step processes, and diagnose when things go wrong. This represents some of the most challenging tasks as they require open-ended reasoning and decision-making.

Input Tokens

Output Tokens

Duration

Cost

Context

Input

Of the available tools. Which ones will you need to use to successfully complete the task?

Expected output

All tools will need to be used: calendar, flights,memory

Model output

Reasoning: Analyze: The task is to find the cheapest flights from London to Paris for tomorrow and return the top 3. We need tomorrow’s date, flight search/filtering, and to store results after each tool call per the rules. Plan: 1) Use calendar to get tomorrow’s date (and format if needed). 2) Use flights to search for routes on that date and filter/sort to obtain the 3 cheapest. 3) Use memory to store outputs after each tool call with clear keys. Act (conceptually): calendar.get_tomorrow_date → memory.store_item; flights.search_flights → flights.filter_by_price → select top 3 → possibly flights.get_flight_details → memory.store_item after each; finalize. Reflect: Tools required are calendar, flights, and memory; no other tools needed. Avoid redundant calls and keep within max 3 retries if any call fails.

Response: Tools to use:
- calendar: To retrieve tomorrow’s date (methods: get_tomorrow_date; optionally format_date if formatting is required).
- flights: To search flights and obtain the 3 cheapest options for the given date (methods: search_flights, filter_by_price, optionally get_flight_details).
- memory: To store meaningful results after every tool call as required (method: store_item with clear keys).