Agents

AI agent reasoning and tool selection tests planning, tool choice, and self-diagnosis capabilities. These skills are crucial for autonomous ticket triage systems and complex agent workflows where models must decide which tools to use, plan multi-step processes, and diagnose when things go wrong. This represents some of the most challenging tasks as they require open-ended reasoning and decision-making.

Duration

Input Tokens

3783

Output Tokens

286

Cost

$0.00

Context

Input

You need to list all open orders for a specific customer whose ID is `cust_42`. Which tool/method do you choose and what parameter object do you send?

Expected output

Use the **database** tool with parameters {"action": "list_orders", "customer_id": "cust_42"}.

Model output

Reasoning: 1.  **Analyze the Request:** The user wants to list all open orders for a customer with ID `cust_42`.
2.  **Inspect Available Tools:** I need a tool that can list orders based on a customer ID. I review the provided tool definitions.
    *   The `database` tool has a method called `list_orders`.
3.  **Examine `list_orders` Schema:** According to the context, the `list_orders` method requires a parameter object containing `action: 'list_orders'` and `customer_id`.
4.  **Determine Parameters:** The required customer ID is explicitly given as `cust_42`.
    *   Tool to choose: `database`
    *   Method to use: `list_orders`
    *   Parameter object: `{"action": "list_orders", "customer_id": "cust_42"}`
5.  **Plan Next Action:** The next step is to invoke the `database.list_orders` tool with the derived parameters to fulfill the request.

Response: tool_code
database.list_orders(customer_id='cust_42')
end_code