Agents

AI agent reasoning and tool selection tests planning, tool choice, and self-diagnosis capabilities. These skills are crucial for autonomous ticket triage systems and complex agent workflows where models must decide which tools to use, plan multi-step processes, and diagnose when things go wrong. This represents some of the most challenging tasks as they require open-ended reasoning and decision-making.

Duration

Input Tokens

3594

Output Tokens

311

Cost

$0.00

Context

Input

You need to list all open orders for a specific customer whose ID is `cust_42`. Which tool/method do you choose and what parameter object do you send?

Expected output

Use the **database** tool with parameters {"action": "list_orders", "customer_id": "cust_42"}.

Model output

Reasoning: The task asks to list all open orders for customer 'cust_42'. Among the available tools, the 'database' tool has the method 'list_orders' which specifically accepts a 'customer_id' parameter. The params_schema for 'list_orders' requires 'action' set to 'list_orders' and 'customer_id' as the customer ID string. No other tool provides order listing functionality. Therefore, the correct tool is 'database', method 'list_orders', and the parameter object is {'action': 'list_orders', 'customer_id': 'cust_42'}.

Response: Tool: database
Method: list_orders
Parameter object: {"action": "list_orders", "customer_id": "cust_42"}