# Chapter 5 — AI Agents & Multi-Agent Systems By the end of this chapter you will: - Implement function calling with OpenAI / Anthropic - Build an agent loop that doesn't infinite-loop - Compose multiple agents safely - Limit blast radius of tool calls --- ## The why > **An LLM that only chats is a librarian who only reads aloud. An LLM with agent capabilities is a librarian who walks to the shelves, pulls the book, cross-references three other volumes, updates the catalog, and emails you a summary — all without you giving a second instruction. Tool use, function calling, and planning are what transform a text generator into a system that actually does things.** A Michelin-starred kitchen runs on a brigade system. The head chef (the "agent") doesn't cook every dish. She plans the menu (planning), calls out orders to the sauce chef, grill chef, and pastry chef (tool calling), tastes every dish before it leaves the pass (evaluation), and adjusts the next round based on what's running low or what a VIP guest requested (memory + replanning). A single LLM chat completion is asking the head chef "what's a good sauce for salmon?" — you get an answer, then the conversation ends. An agent is the entire brigade: the LLM decides which station to call, calls the right chef with the right parameters, checks the result, decides whether to plate it or send it back, and remembers that table 7 is allergic to dairy for the rest of the evening. The difference between "ChatGPT told me how to query my database" and "an agent queried my database, analyzed the results, and posted a Slack summary at 9 AM every Monday" is the difference between a cookbook and a kitchen. --- ## Core concepts ### The agent loop: LLM + tools + memory + planning Every AI agent, from a 50-line script to Devin, follows the same fundamental loop. Understanding this loop is the difference between building something that works once and something that works at 3 AM without human intervention. ``` ┌─────────────────────────────────────────────────────┐ │ THE AGENT LOOP │ │ │ │ 1. USER INPUT → "What's the top-selling product │ │ in our European warehouses this quarter?" │ │ │ │ │ ▼ │ │ 2. LLM DECIDES → "I need to call get_sales_data() │ │ with region='EU' and period='Q2_2026'" │ │ │ │ │ ▼ │ │ 3. TOOL EXECUTION → Your function runs, returns │ │ [{product: "Widget X", revenue: 450000}, ...] │ │ │ │ │ ▼ │ │ 4. LLM EVALUATES → "Got 12 rows. Widget X is #1. │ │ Should I also check inventory levels?" │ │ │ │ │ ▼ │ │ 5. MORE TOOLS? → Yes → call check_inventory() │ │ → No → format final answer │ │ │ │ │ ▼ │ │ 6. FINAL RESPONSE → Natural language answer + │ │ optional structured data │ └─────────────────────────────────────────────────────┘ ``` Here's the minimal TypeScript agent loop — this is the engine that powers everything from customer support bots to autonomous coding agents: ```typescript // === agent-loop.ts === // The core loop: 60 lines that turn an LLM into an agent interface Tool { name: string; description: string; parameters: Record; execute: (args: Record) => Promise; } interface AgentConfig { model: string; systemPrompt: string; tools: Tool[]; maxIterations: number; // Safety valve — prevents infinite loops temperature: number; } async function agentLoop( userMessage: string, config: AgentConfig, conversationHistory: { role: string; content: string }[] = [] ): Promise { const messages = [ { role: "system", content: config.systemPrompt }, ...conversationHistory, { role: "user", content: userMessage }, ]; let iterations = 0; while (iterations < config.maxIterations) { iterations++; // Step 1: Ask the LLM what to do next const response = await fetch("https://api.openai.com/v1/chat/completions", { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, }, body: JSON.stringify({ model: config.model, messages, tools: config.tools.map((t) => ({ type: "function", function: { name: t.name, description: t.description, parameters: t.parameters, }, })), tool_choice: "auto", temperature: config.temperature, }), }); const completion = await response.json(); const assistantMessage = completion.choices[0].message; // Step 2: Add the assistant's response to conversation messages.push(assistantMessage); // Step 3: Check if the LLM wants to call a tool if (assistantMessage.tool_calls && assistantMessage.tool_calls.length > 0) { for (const toolCall of assistantMessage.tool_calls) { const tool = config.tools.find( (t) => t.name === toolCall.function.name ); if (!tool) { messages.push({ role: "tool", tool_call_id: toolCall.id, content: JSON.stringify({ error: `Tool "${toolCall.function.name}" not found`, }), }); continue; } // Step 4: Execute the tool with the arguments the LLM chose try { const args = JSON.parse(toolCall.function.arguments); const result = await tool.execute(args); messages.push({ role: "tool", tool_call_id: toolCall.id, content: result, }); } catch (error) { messages.push({ role: "tool", tool_call_id: toolCall.id, content: JSON.stringify({ error: `Tool execution failed: ${(error as Error).message}`, }), }); } } // Loop continues — LLM sees tool results and decides next step continue; } // Step 5: No tool calls — LLM is done, return final answer return assistantMessage.content || ""; } // Step 6: Exceeded max iterations — force a summary messages.push({ role: "user", content: "You've reached the maximum number of steps. Summarize what you've found so far in 2-3 sentences.", }); const finalResponse = await fetch( "https://api.openai.com/v1/chat/completions", { method: "POST", headers: { "Content-Type": "application/json", Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, }, body: JSON.stringify({ model: config.model, messages }), } ); const final = await finalResponse.json(); return final.choices[0].message.content; } ``` ### Tool/function calling: the bridge between language and action Function calling is the mechanism that lets an LLM say "I need to run this specific function with these specific arguments" instead of just "you should probably check the database." The LLM doesn't execute code — it outputs structured JSON that your application executes. ```typescript // === tool-definitions.ts === // Define tools with JSON Schema — the LLM uses these to decide what to call const tools: Tool[] = [ { name: "search_knowledge_base", description: "Search the internal knowledge base for articles matching a query. " + "Use this when users ask about company policies, product docs, or historical decisions.", parameters: { type: "object", properties: { query: { type: "string", description: "Natural language search query", }, max_results: { type: "number", description: "Maximum number of results to return (1-20)", default: 5, }, }, required: ["query"], }, execute: async (args) => { const results = await vectorSearch(args.query as string, { limit: (args.max_results as number) || 5, }); return JSON.stringify(results); }, }, { name: "send_slack_notification", description: "Send a message to a Slack channel. Use this when the user explicitly asks " + "to notify a team or post an update.", parameters: { type: "object", properties: { channel: { type: "string", description: "Slack channel name (e.g., #eng-alerts)", }, message: { type: "string", description: "The message text to send", }, }, required: ["channel", "message"], }, execute: async (args) => { await fetch("https://slack.com/api/chat.postMessage", { method: "POST", headers: { Authorization: `Bearer ${process.env.SLACK_BOT_TOKEN}`, "Content-Type": "application/json", }, body: JSON.stringify({ channel: args.channel, text: args.message, }), }); return JSON.stringify({ success: true }); }, }, ]; ``` The LLM decides which tool to call based entirely on the tool's `description` field. A vague description like "searches stuff" will cause the LLM to either never call the tool or call it for everything. Write descriptions as if you're explaining the tool to a colleague who's never seen your codebase: when to use it, what it returns, and what the parameters mean. Bad tool descriptions are the #1 cause of agent failures in production. ### ReAct pattern: Reason, then Act, then Observe The ReAct (Reasoning + Acting) pattern interleaves thought with action. Instead of the LLM planning everything upfront, it thinks about what it knows, decides on one action, observes the result, then thinks again. This is the pattern behind most production agents. ```typescript // === react-pattern.ts === // The ReAct system prompt — teaches the LLM to think-then-act const REACT_SYSTEM_PROMPT = `You are an AI assistant that solves problems step by step. For every user request, follow this pattern: 1. THOUGHT: Analyze what you know and what you need to find out. Write your reasoning as if thinking aloud. 2. ACTION: If you need information, call the appropriate tool. If you have enough information, provide the final answer. 3. OBSERVATION: After a tool returns results, evaluate them. Are they sufficient? Is something missing? Should you try a different approach? IMPORTANT RULES: - Never assume information you haven't retrieved via a tool. - If a tool returns an error, try an alternative approach before giving up. - If you've tried 3 different approaches and all failed, explain what you tried and ask the user for guidance. - Cite sources when using tool results.`; ``` ### Planning: CoT, ToT, and structured reasoning **Chain of Thought (CoT)** is the simplest form of planning — the LLM writes out its reasoning steps before answering. You enable it with a system prompt that says "think step by step" or by showing examples of step-by-step reasoning. For agents, CoT is essential: the LLM needs to explain WHY it's calling a tool, not just call it. **Tree of Thoughts (ToT)** extends CoT by exploring multiple reasoning paths simultaneously. The LLM generates several possible next steps, evaluates each, and pursues the most promising one. This is computationally expensive (you're making multiple LLM calls per step) but dramatically improves accuracy on tasks that require exploration — debugging complex code, solving math problems, or planning multi-step workflows. ```typescript // === tree-of-thoughts.ts === // Simplified ToT: generate N candidate next steps, score them, pick the best async function treeOfThoughtsStep( problem: string, candidates: number = 3 ): Promise { // Step 1: Generate candidates const brainstormPrompt = `Given this problem: ${problem} Generate ${candidates} different approaches to solve it. For each approach, write 1-2 sentences explaining the strategy and why it might work. Number them 1 through ${candidates}.`; const brainstormResponse = await callLLM(brainstormPrompt); // Step 2: Score each candidate const evaluationPrompt = `Here is a problem and several approaches: PROBLEM: ${problem} APPROACHES: ${brainstormResponse} Evaluate each approach on a scale of 1-10 based on: - Likelihood of success - Efficiency (fewest steps) - Risk of failure or incorrect result Return a JSON array: [{"approach": 1, "score": 8, "reasoning": "..."}, ...]`; const evaluation = JSON.parse(await callLLM(evaluationPrompt)); // Step 3: Pick the winner and execute const best = evaluation.sort((a: any, b: any) => b.score - a.score)[0]; return best; } ``` ### Multi-agent collaboration Single agents hit a ceiling. A coding agent that also manages your calendar and answers support tickets becomes confused — the system prompt grows to 5000 tokens, tool definitions balloon, and the agent starts mixing contexts. Multi-agent architectures solve this by giving each agent a focused role. ```typescript // === multi-agent-orchestrator.ts === // A coordinator agent delegates to specialist agents interface SpecialistAgent { name: string; description: string; capabilities: string[]; run: (input: string) => Promise; } const specialistAgents: SpecialistAgent[] = [ { name: "code-reviewer", description: "Reviews code for bugs, security issues, and style violations", capabilities: ["code_review", "security_audit", "style_check"], run: async (code) => { // Agent with tools: read_file, run_linter, check_dependencies return await codeReviewAgent(code); }, }, { name: "doc-writer", description: "Generates API documentation, READMEs, and changelogs", capabilities: ["generate_docs", "update_readme", "write_changelog"], run: async (context) => { return await docWriterAgent(context); }, }, { name: "test-generator", description: "Writes unit tests and integration tests for existing code", capabilities: ["write_tests", "expand_coverage", "mock_dependencies"], run: async (code) => { return await testGeneratorAgent(code); }, }, ]; async function orchestratorAgent(userRequest: string): Promise { // Step 1: The orchestrator decides which specialists to invoke const routingPrompt = `You are a coordinator. Given this request: "${userRequest}" Available specialists: ${specialistAgents.map((a) => `- ${a.name}: ${a.description}`).join("\n")} Return a JSON array of specialist names to invoke, in order. Example: ["code-reviewer", "test-generator"]`; const routingDecision = JSON.parse(await callLLM(routingPrompt)); // Step 2: Run each specialist, passing results forward let context = userRequest; const results: string[] = []; for (const agentName of routingDecision) { const agent = specialistAgents.find((a) => a.name === agentName); if (!agent) continue; const result = await agent.run(context); results.push(`[${agent.name}]: ${result}`); context = result; // Feed output of agent N as input to agent N+1 } // Step 3: Synthesize results const synthesisPrompt = `Synthesize these specialist reports into a cohesive response: ${results.join("\n\n")} Original request: ${userRequest}`; return await callLLM(synthesisPrompt); } ``` Multi-agent architectures shine when: (1) each role needs different tools — a code reviewer needs linter access, a doc writer needs wiki access; (2) prompts would conflict — "be thorough and critical" vs "be helpful and encouraging"; (3) you need parallel execution — code review AND test generation can run simultaneously. For simple workflows, a single agent with well-defined tools is faster and cheaper. --- ## Real-world example: How Cognition built Devin Cognition's Devin is the most prominent AI coding agent. It doesn't just suggest code — it plans features, writes code across multiple files, runs terminal commands, debugs test failures, and iterates until tests pass. The architecture reveals patterns every backend engineer should understand. **The execution environment is a sandbox.** Devin doesn't run commands on your machine. It spins up a container with a shell, code editor, and browser — every tool (write_file, run_test, search_web, git_commit) operates inside this sandbox. This is the only safe way to give an LLM shell access. The sandbox has network restrictions, filesystem quotas, and a time limit per session. If Devin runs `rm -rf /`, it nukes its own container — not your laptop. **Planning before coding.** Devin writes a plan before writing code. This isn't a gimmick — it's the ReAct pattern at scale. The planner agent breaks the feature into subtasks, the coder agent implements each subtask, and the reviewer agent checks the output. Failed tests loop back to the coder. This three-agent loop can run for 20-30 iterations on a complex feature. **The same pattern works at smaller scale.** CrewAI and AutoGen are open-source frameworks that implement multi-agent orchestration. CrewAI defines agents with roles, goals, and tools — then runs them sequentially or in parallel. AutoGen (from Microsoft) adds conversation patterns: two agents can debate a solution, or a manager agent can delegate to worker agents. These frameworks aren't magic — they're the agent loop with better abstractions, retry logic, and conversation management. --- ## Common mistakes | Mistake | Why it's wrong | What to do instead | |---------|---------------|-------------------| | No `maxIterations` limit on the agent loop | An LLM that keeps calling tools forever burns through your API budget in minutes. A single agent loop with a buggy tool can generate $50+ in API costs before anyone notices | Always set `maxIterations` (start with 10). Add a budget tracker that throws if total cost exceeds $X per request | | Giving the agent tools it doesn't need | The LLM may call the wrong tool, confuse parameters, or get distracted by irrelevant options. More tools = more decisions = more chances to be wrong | Give each agent exactly the tools it needs for its role. A code reviewer doesn't need Slack access. A notification agent doesn't need database write access | | Blindly executing tool results without validation | The LLM might hallucinate user IDs, SQL queries, or file paths. Executing `DELETE FROM users WHERE id=99999` because the LLM guessed an ID will corrupt your database | Validate all tool outputs before acting on them. Check that IDs exist, file paths are within allowed directories, and SQL queries pass a safety check (no DROP, no DELETE without WHERE) | | Using the same system prompt for agents and chatbots | "You are a helpful assistant" doesn't teach the LLM how to use tools, when to stop, or how to handle failures. The agent will either never call tools or call them endlessly | Write agent-specific system prompts that teach the ReAct pattern, define stopping criteria, and include error-handling instructions | | Not logging tool calls and their results | When an agent produces a wrong answer, you can't debug it without knowing which tools it called, with what arguments, and what they returned | Log every tool call with: timestamp, tool name, arguments, result (or error), and duration. Store in structured format (JSON) for analysis | | Running agents synchronously in request handlers | An agent loop with 5 tool calls can take 15-30 seconds. Your HTTP request will time out, and the user gets a 504 error | Use a queue (BullMQ, SQS) for long-running agent tasks. Return a task ID immediately, let the user poll or connect via WebSocket for results | | Skipping tool result truncation | A database query returning 50,000 rows will blow past the LLM's context window. The LLM sees truncated JSON, gets confused, and hallucinates the rest | Truncate tool results to a reasonable size (e.g., first 50 rows + count of remaining rows). Include metadata: "Returned 50 of 50,000 matching records" | --- ## Production notes - **Version your tool definitions.** When you change a tool's parameters (add a required field, rename an argument), old conversations in progress will break because the LLM generated tool calls with the old schema. Use a version prefix in tool names (`search_kb_v2`) and keep old versions alive for a deprecation window while in-flight conversations drain. - **Implement a tool timeout per invocation.** A single tool execution that hangs (database connection pool exhausted, external API down) will stall the entire agent loop. Wrap every `tool.execute()` call in a timeout promise. If the tool doesn't respond in N seconds, return an error to the LLM so it can try an alternative approach. - **Use a dedicated agent service, not your main API.** Agent workloads are bursty and CPU-intensive (multiple LLM calls + tool execution per request). Running them in your main API process will cause latency spikes for regular traffic. Deploy agents as a separate service with its own scaling rules. - **Add human-in-the-loop for destructive actions.** Any tool that modifies production data, sends customer-facing messages, or triggers a deployment should require human approval before execution. The agent proposes the action; a human clicks "approve" in a Slack message or dashboard. This adds friction intentionally — the cost of an incorrect automated action is always higher than the cost of a 30-second approval delay. - **Track agent-specific metrics.** Beyond standard API metrics, track: agent success rate (did it complete the task?), average iterations per task, tool call success rate (what percentage of tool calls returned useful results vs errors), and abandonment rate (tasks where maxIterations was hit). These metrics reveal whether your tool descriptions, system prompt, or tool implementations need improvement. --- ## Performance notes - **Parallelize independent tool calls.** If the LLM requests three tool calls in one response and those tools don't depend on each other (e.g., fetch user profile + fetch team members + fetch recent orders), execute them in parallel with `Promise.all()`. This cuts iteration time from 3× tool_latency to max(tool_latency). - **Cache tool results within a session.** If the agent calls `get_user(userId=42)` three times in one conversation, the second and third calls are wasted. Cache tool results in a Map keyed by `toolName + JSON.stringify(args)`. Invalidate the cache when a write operation changes the underlying data. - **Stream the agent's thinking to the user.** Nothing is worse than staring at a spinner for 30 seconds while the agent works. Use Server-Sent Events to stream each step: "🔍 Searching knowledge base...", "📊 Analyzing 12 results...", "✍️ Generating response...". This makes the latency feel like progress. - **Use cheaper models for planning, expensive models for final output.** The planning/decision step (which tool to call next?) can use a fast model like GPT-4o-mini or Claude Haiku. Save the expensive model (GPT-4o, Claude Opus) for the final synthesis step where quality matters most. This cuts per-request cost by 40-60% with minimal quality impact. - **Set a per-task token budget, not just per-request.** An agent that makes 10 LLM calls at 2000 tokens each costs more than a single 4000-token call. Track cumulative tokens across the entire agent session. Alert if a single user request consumes more than your threshold (e.g., 20K tokens). --- ## Security notes - **Never pass raw user input to shell or database tools.** Even if the LLM is supposed to generate safe commands, treat every tool argument as potentially malicious. Use parameterized queries for databases. Escape shell arguments. Validate file paths against an allowlist of directories. The LLM is not a security boundary — your tool implementations are. - **Restrict tool access by agent role.** A customer-facing support agent should never have access to internal admin tools (delete user, view billing details, modify system config). Implement this at the tool registry level: each agent gets a subset of all available tools. - **Sanitize tool results before they reach the LLM.** If a database query returns PII (email addresses, phone numbers, SSNs), that data is now in the LLM's context window and potentially in the provider's logs. Redact PII from tool results before passing them to the LLM, or use a self-hosted model for sensitive workloads. - **Rate-limit agent requests more aggressively than regular API requests.** An agent request can trigger 5-20 LLM calls + database queries + external API calls. One malicious user making 10 agent requests per second could cost you hundreds of dollars per hour. Use a token bucket or sliding window rate limiter with a low threshold for agent endpoints. - **Implement a kill switch per agent session.** If you detect anomalous behavior (the agent is calling the same tool in a loop, accessing unexpected resources, or generating suspicious output), you need to terminate that session immediately — not just stop processing but revoke any temporary credentials or tokens issued for that session. --- ## Exercises ### Beginner 1. **Build a single-tool agent:** Write an agent that has exactly one tool — `get_current_weather(city: string)`. The tool should return hardcoded weather data for 5 cities. Test that the agent: (a) calls the tool when asked about weather, (b) returns a natural language response incorporating the tool's data, and (c) doesn't call the tool when asked a non-weather question like "who won the world cup?" 2. **Add iteration limits:** Take the single-tool agent from exercise 1 and deliberately create a scenario where the tool returns confusing data (e.g., `{ error: "ambiguous city name" }`). Verify that your agent: (a) retries at most N times (where N is your `maxIterations`), (b) eventually returns a graceful failure message to the user, and (c) logs each failed attempt with the tool name and error. ### Intermediate 1. **Build a ReAct agent with 3 tools:** Create an agent with these tools: `search_docs(query)`, `list_available_apis()`, and `get_api_schema(apiName)`. The agent should help developers find the right API for their use case. Test with: "I need to send emails programmatically — what API should I use and what parameters does it need?" The agent should: search docs for "email", list available APIs, find the email API, retrieve its schema, and present the answer. 2. **Implement multi-agent delegation:** Build a coordinator agent that delegates to two specialist agents: a "translator" agent (detects language, translates to English) and a "summarizer" agent (takes English text and returns a 3-bullet summary). Test with a French paragraph. The coordinator should route French text → translator → summarizer → final output. Log the handoff between agents so you can trace the full pipeline. ### Advanced 1. **Build an agent with a budget tracker:** Extend any agent from the exercises above with a cost-tracking middleware. Track: tokens consumed per LLM call, total tokens per session, estimated cost (using current OpenAI/Anthropic pricing), and tool execution time. Add a configurable budget limit — when the session exceeds the limit, the agent should gracefully stop and return what it has so far. Test with a budget of $0.05 and verify the agent stops at the limit. 2. **Implement a human-in-the-loop approval flow:** Build an agent that can propose database schema changes. When the agent wants to execute `ALTER TABLE`, it should not run the command. Instead, it should: (a) output the proposed SQL, (b) explain what the change does and why it's needed, (c) wait for a "approved" or "rejected" response (simulated via a callback function or a mock HTTP endpoint), and (d) only execute on approval. The approval mechanism should have a 5-minute timeout — if no response, the agent should abandon the change and explain why. --- ## Interview questions ### Beginner 1. **Q: What's the difference between an LLM and an AI agent?** **A:** An LLM (Large Language Model) takes text input and produces text output — it's a completion engine. An AI agent wraps an LLM in a loop that can: call external tools (APIs, databases, file systems), maintain memory across multiple turns, plan multi-step workflows, and evaluate its own outputs. An LLM tells you HOW to query a database. An agent queries the database, analyzes the result, and tells you what it found. The agent loop (decide → act → observe → decide) is what separates them. 2. **Q: What is function calling in the context of LLMs?** **A:** Function calling (also called tool use) is a capability where an LLM outputs structured JSON specifying which function to call and with what arguments, instead of (or in addition to) natural language text. The LLM doesn't execute the function — your application reads the JSON, executes the function, and feeds the result back to the LLM. The LLM uses the function's name, description, and parameter schema (provided in the API request) to decide when and how to call it. It's the bridge between "the AI thinks something" and "the AI does something." 3. **Q: Why do you need a `maxIterations` limit in an agent loop?** **A:** Without a limit, an agent can enter an infinite loop if: a tool keeps returning unexpected results, the LLM gets confused and retries the same failing approach, or two tools create a dependency cycle (tool A's output makes the LLM call tool B, whose output makes it call tool A again). Each iteration costs money (LLM API call) and time. A `maxIterations` limit (typically 5-15) acts as a circuit breaker — when hit, the agent summarizes what it found and stops. In production, you should also track cost and abort if the session exceeds a dollar threshold. ### Senior 1. **Q: You're building a customer support agent that can refund orders, cancel subscriptions, and modify account details. How do you prevent the agent from taking incorrect destructive actions?** **A:** There are four layers of defense. First, **tool design**: destructive tools should require explicit confirmation parameters (e.g., `confirmed: boolean` that must be `true`) so the LLM can't accidentally trigger them. Second, **human-in-the-loop**: any tool that modifies production data, moves money, or sends customer-facing communications should queue the action for human approval rather than executing immediately. Third, **validation at the tool level**: before executing, validate that the order ID exists, the refund amount matches the original payment, and the user has permission to perform the action. Fourth, **audit logging**: every destructive action should log who requested it (end user + agent session ID), what was proposed, who approved it, and the final result. These logs are critical for both debugging and compliance. The principle: the LLM proposes, the application validates, the human approves. 2. **Q: How would you design a multi-agent system where agents need to share intermediate results without blowing up the context window?** **A:** Don't pass raw outputs between agents — pass summaries and structured data. Implement a shared "blackboard" (a key-value store like Redis) where agents write their findings in a structured format. Each agent reads only the keys relevant to its task. For example, the "code reviewer" agent writes `{ file: "auth.ts", issues: [...], score: 7 }` to the blackboard. The "test generator" agent reads only the `issues` key to generate targeted tests. The coordinator maintains a lightweight context — just the task description and references to blackboard keys. This keeps each agent's context window focused on its specific job. Additionally, implement context window monitoring: if an agent's accumulated messages approach the limit, summarize older messages before adding new ones. 3. **Q: An agent in production is behaving inconsistently — sometimes it's brilliant, sometimes it calls the wrong tools or produces garbage output. How do you systematically debug this?** **A:** Start with observability. You need structured logs for every agent session containing: the full message history (system prompt + user messages + assistant responses + tool calls + tool results), which model was used with what parameters, and the final output. Reproduce the failing session by replaying the exact message history — this tells you whether the issue is deterministic (same input → same bad output) or stochastic (temperature-related variance). If deterministic, inspect the tool descriptions and system prompt: is there ambiguity that could cause the LLM to pick the wrong tool? If stochastic, lower the temperature and add more explicit instructions in the tool descriptions. Next, implement evaluation: for each agent task type, maintain a set of 20-50 test cases with expected tool calls and outputs. Run these evals on every prompt or tool description change. Finally, consider a "grader" agent — a separate, cheaper LLM that reviews the main agent's tool calls and flags suspicious ones (e.g., "agent called delete_user for a request about changing email"). --- ## Summary AI agents transform LLMs from text generators into systems that act. The agent loop — decide, call tools, observe results, decide again — is the universal pattern. Function calling bridges language and action by letting the LLM output structured JSON that your application executes. The ReAct pattern interleaves reasoning with action, producing more reliable results than planning everything upfront. Multi-agent architectures split complex tasks across specialist agents with focused tools and prompts. The two hard problems in production agents are reliability (making the right tool calls consistently) and safety (preventing destructive actions). Solve reliability with great tool descriptions, structured logging, and eval suites. Solve safety with tool-level validation, human-in-the-loop for destructive actions, and per-agent tool restrictions. Agents are powerful but expensive — every iteration costs money, so budget tracking and iteration limits are not optional. --- ## Quick recall - **Agent = LLM + tools + memory + planning loop.** The LLM decides what to do, tools execute it, the loop continues until the task is done or the limit is hit. - **Function calling:** The LLM outputs structured JSON specifying which function to call with which arguments. Your code executes it, not the LLM. - **ReAct pattern:** Thought → Action → Observation → Thought. Interleave reasoning with tool calls. Better than upfront planning. - **Tool descriptions are prompt engineering.** The LLM decides which tool to call based on the description. Write them like you're explaining to a colleague. - **`maxIterations` is mandatory.** Without it, buggy tools or confusing results cause infinite loops that burn money. - **Multi-agent = divide and conquer.** Each specialist agent gets a focused role, minimal tools, and a role-specific system prompt. - **Human-in-the-loop for destructive actions.** The LLM proposes; a human approves. The cost of a 30-second delay is always less than the cost of a wrong automated action. - **Log everything.** Tool calls, arguments, results, errors, timing. You cannot debug an agent you cannot observe. - **Cheap models for planning, expensive models for final output.** GPT-4o-mini for "which tool next?" GPT-4o for "write the final answer." Cuts cost 40-60%. - **Tool results are untrusted input.** Validate, sanitize, and truncate everything before it reaches the LLM or your database. --- ## Quiz 1. **What is the core loop that every AI agent follows?** A) Ask user → respond → ask user → respond **B) Decide → call tools → observe results → decide again** C) Train → validate → test → deploy D) Receive request → query database → format response → send 2. **In function calling, who executes the function — the LLM or your application code?** A) The LLM executes it in a sandbox **B) Your application code executes it — the LLM only outputs which function to call and with what arguments** C) A cloud function provided by OpenAI D) The user's browser 3. **Why is `maxIterations` essential in an agent loop?** A) It's required by the OpenAI API **B) To prevent infinite loops that burn API budget — an agent with buggy tools can call them forever** C) To comply with GDPR D) To make the agent faster 4. **What does the ReAct pattern stand for?** A) React + Act — a UI framework for agents **B) Reasoning + Acting — interleave thought with tool calls** C) Reactive + Active — two agent modes D) Read + Act — ingest then execute 5. **When should you use multi-agent architecture instead of a single agent?** A) Always — more agents are always better B) Only when the task fits in one prompt **C) When different roles need different tools, prompts would conflict, or tasks can run in parallel** D) Only for frontend applications 6. **What's the most important defense for agents that can perform destructive actions?** A) A longer system prompt **B) Human-in-the-loop approval — the agent proposes, a human approves before execution** C) Faster API calls D) Using a larger model 7. **How do you reduce per-request cost in an agent system?** A) Use the most expensive model for everything for best quality **B) Use cheaper/faster models for planning/decision steps, expensive models only for final synthesis** C) Skip tool validation D) Remove all logging

Chapter 5 — AI Agents & Multi-Agent Systems

Part of L7