Recap — Designing Tool-Using Agents

# Designing Tool-Using Agents LLMs become much more powerful when they can call functions. "What's the weather in Mumbai?" → LLM calls `get_weather("Mumbai")` → uses the result in its response. This pattern is called **tool use** or **function calling**. ## The basic flow ``` 1. You define tools (Python functions, REST endpoints, anything callable). 2. You describe the tools to the LLM in the prompt (schema + description). 3. User asks a question. 4. LLM decides: do I need a tool? Which one? With what arguments? 5. LLM responds with a tool-call request (structured output). 6. Your code intercepts, runs the tool, gets the result. 7. You feed the result back to the LLM. 8. LLM uses the result to compose the final answer. ``` This loop repeats. The LLM may call multiple tools in sequence. ## The tool definition In Anthropic / OpenAI / most APIs, tools are defined with a JSON schema: ```python tools = [ { "name": "get_weather", "description": "Get the current weather in a given city", "input_schema": { "type": "object", "properties": { "city": { "type": "string", "description": "City name, e.g., 'Mumbai' or 'New York'" }, "unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "default": "celsius" } }, "required": ["city"] } }, { "name": "search_products", "description": "Search the product catalog by keyword and optional category", "input_schema": { "type": "object", "properties": { "query": {"type": "string"}, "category": {"type": "string"}, "limit": {"type": "integer", "default": 10} }, "required": ["query"] } } ] response = client.messages.create( model="claude-sonnet-4-6", tools=tools, messages=[{"role": "user", "content": "What's the weather in Mumbai?"}], ) ``` The LLM examines the tools, decides to call `get_weather`, and returns: ```json { "tool_name": "get_weather", "tool_input": {"city": "Mumbai", "unit": "celsius"} } ``` Your code: 1. Detects the tool call. 2. Runs `get_weather("Mumbai", "celsius")`. 3. Gets result: `{"temp": 32, "condition": "humid"}`. 4. Sends back to LLM with `tool_result` block. LLM continues: "The weather in Mumbai is 32°C and humid." ## Designing good tool definitions The description IS the prompt. The model decides whether to call your tool based on the description, not the function name. Bad: ``` "name": "g", "description": "gets data" ``` Good: ``` "name": "get_customer_order_status", "description": "Look up the current status, tracking info, and estimated delivery date of a customer's order. Use this when the customer asks about their order, shipping, delivery, or tracking. Requires the order_id." ``` Verbose descriptions help the model decide WHEN to use the tool and WHAT arguments to fill in. ## Multi-step tool use Models can chain tools. Example: ``` User: "Is order #54321 going to arrive before my dinner reservation Friday?" LLM thinks: 1. I need the order status → call get_order_status(54321). 2. I need the user's calendar Friday → call get_calendar_events(date="Friday"). 3. Compare the delivery time vs the reservation time. 4. Respond. ``` The LLM emits tool calls one at a time, sees results, decides next steps. Modern frontier models handle 5-10 step tool chains gracefully. ## Common tool patterns in production ### Retrieval tool ``` "name": "search_knowledge_base", "description": "Search the company knowledge base for information about products, policies, FAQ." ``` This is basically RAG via tool use. The LLM decides when retrieval is needed. ### Database tool ``` "name": "query_orders_database", "description": "Run a SQL query against the orders database. Use for questions about specific order data." ``` Risky — the LLM is now writing SQL against your DB. In practice, use parametrized tools (`get_order_by_id`, `list_orders_for_user`) rather than raw SQL. ### Action tool ``` "name": "create_support_ticket", "description": "Create a customer support ticket. Provide customer ID, issue category, and description." ``` The LLM can take ACTIONS, not just retrieve. This is where things get exciting and dangerous. ## Safety considerations When LLMs call tools that take actions, you need: 1. **Confirmation for destructive operations.** Don't let the LLM delete data without explicit human approval. 2. **Scope limitations.** The `query_database` tool runs SELECT only, never DELETE. 3. **Audit logging.** Every tool call logged with input/output. 4. **Rate limits.** Prevent runaway loops. 5. **Sandboxing.** Code execution in containers, not your prod server. ## Agent loops and termination A simple agent: ```python def run_agent(user_m