Back to Home
    Agentic AIAgentic AI 2025Advanced

    Agentic AI

    Complete guide to building AI agents — architecture, tool use, memory, planning patterns, multi-agent systems, frameworks, evaluation, and production best practices.

    20 min read
    agentsllmtool-useplanningmulti-agentmemorylanggraphopenai

    What Is an AI Agent?

    2 topics

    Agent vs. Chatbot vs. Pipeline

    Understanding what makes something an agent

    CHATBOT
      Input → LLM → Output
      Single call. No memory. No tools. No decisions.
    
    PIPELINE (Chain)
      Input → Step1 → Step2 → Step3 → Output
      Fixed sequence. LLM at each step. Deterministic path.
    
    AGENT
      Input → LLM decides → Tool call? → Observe result
                    ↑________________________|
      Dynamic. Self-directed. Loops until goal is met.
    
    AN AGENT HAS:
      ✅ A goal / task to complete
      ✅ Tools it can call (functions, APIs, code execution)
      ✅ Memory (context of what happened)
      ✅ A reasoning loop (decides what to do next)
      ✅ The ability to stop when done
    
    KEY DIFFERENCE:
      A pipeline follows instructions.
      An agent decides its own next steps.

    💡 Not everything needs to be an agent — pipelines are simpler and more predictable

    ⚡ Use agents when the number of steps isn't known upfront

    conceptsfundamentals

    The Agent Loop

    The core reasoning cycle every agent runs

    THE AGENT LOOP (Observe → Think → Act):
    
    ┌─────────────────────────────────────────┐
    │  1. PERCEIVE                            │
    │     Receive task + current context      │
    │     Load memory (past actions, results) │
    ├─────────────────────────────────────────┤
    │  2. THINK (LLM call)                    │
    │     Given goal + context + tools:       │
    │     - Should I call a tool?             │
    │     - Which tool? With what args?       │
    │     - Or is the goal complete?          │
    ├─────────────────────────────────────────┤
    │  3. ACT                                 │
    │     Execute tool call                   │
    │     (search, write code, call API…)     │
    ├─────────────────────────────────────────┤
    │  4. OBSERVE                             │
    │     Receive tool result                 │
    │     Add to context/memory               │
    ├─────────────────────────────────────────┤
    │  5. REPEAT or STOP                      │
    │     Loop back to step 2                 │
    │     Stop when: done / error / max steps │
    └─────────────────────────────────────────┘

    💡 Each loop iteration costs tokens — keep tool outputs concise

    ⚠️ Always define an exit condition — agents can loop forever without one

    architectureloop

    Tool Use & Function Calling

    3 topics

    Defining Tools (OpenAI)

    How to define and register tools for an agent

    python
    from openai import OpenAI
    import json
    
    client = OpenAI()
    
    # Define tools as JSON schema
    tools = [
        {
            "type": "function",
            "function": {
                "name": "search_web",
                "description": "Search the web for current information. Use when you need facts you don't know.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "query": {
                            "type": "string",
                            "description": "The search query"
                        },
                        "num_results": {
                            "type": "integer",
                            "description": "Number of results to return (1-10)",
                            "default": 5
                        }
                    },
                    "required": ["query"]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "run_python",
                "description": "Execute Python code and return the output. Use for calculations, data processing.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "code": {"type": "string", "description": "Python code to execute"}
                    },
                    "required": ["code"]
                }
            }
        }
    ]

    💡 Tool descriptions are part of the prompt — write them clearly and specifically

    ⚡ Include examples in descriptions for complex tools — it significantly improves accuracy

    toolsopenaifunction-calling

    The Agent Loop in Code

    Full implementation of the tool-use loop

    python
    import json
    from openai import OpenAI
    
    client = OpenAI()
    MAX_ITERATIONS = 10  # Safety limit
    
    def run_agent(task: str) -> str:
        messages = [
            {"role": "system", "content": "You are a helpful research assistant. Use tools to answer questions accurately."},
            {"role": "user", "content": task}
        ]
    
        for iteration in range(MAX_ITERATIONS):
            response = client.chat.completions.create(
                model="gpt-4o",
                messages=messages,
                tools=tools,
                tool_choice="auto",   # Let model decide
            )
    
            msg = response.choices[0].message
            messages.append(msg)  # Add assistant turn to history
    
            # No tool calls → agent is done
            if not msg.tool_calls:
                return msg.content
    
            # Execute each tool call
            for call in msg.tool_calls:
                fn_name = call.function.name
                fn_args = json.loads(call.function.arguments)
    
                print(f"[Agent] Calling {fn_name}({fn_args})")
                result = dispatch_tool(fn_name, fn_args)  # Your tool executor
    
                messages.append({
                    "role": "tool",
                    "content": json.dumps(result),
                    "tool_call_id": call.id
                })
    
        return "Max iterations reached without completing the task."
    
    # Run it
    answer = run_agent("What are the top 3 Python web frameworks in 2025?")
    print(answer)

    ⚠️ Always cap iterations — models can get stuck in loops

    💡 Log every tool call and result for debugging and auditing

    ⚡ tool_choice='required' forces a tool call; 'none' disables tools for that turn

    agent-loopimplementationpython

    Tool Dispatcher

    Safely routing tool calls to real implementations

    python
    import subprocess
    import requests
    
    def dispatch_tool(name: str, args: dict) -> dict:
        """Route tool calls to actual implementations with error handling."""
        try:
            if name == "search_web":
                return search_web(**args)
            elif name == "run_python":
                return run_python(**args)
            elif name == "read_file":
                return read_file(**args)
            else:
                return {"error": f"Unknown tool: {name}"}
        except Exception as e:
            return {"error": str(e), "tool": name}
    
    def run_python(code: str) -> dict:
        """Execute Python in a subprocess sandbox."""
        try:
            result = subprocess.run(
                ["python3", "-c", code],
                capture_output=True, text=True,
                timeout=10  # Never run untrusted code without a timeout!
            )
            return {
                "stdout": result.stdout,
                "stderr": result.stderr,
                "exit_code": result.returncode
            }
        except subprocess.TimeoutExpired:
            return {"error": "Code execution timed out (10s limit)"}
    
    def read_file(path: str) -> dict:
        """Read file with path validation."""
        from pathlib import Path
        safe_root = Path("/workspace")  # Restrict to safe directory
        target = (safe_root / path).resolve()
        if not str(target).startswith(str(safe_root)):
            return {"error": "Path traversal blocked"}
        return {"content": target.read_text()}

    ⚠️ NEVER execute code from agents without sandboxing — use Docker, E2B, or subprocess with limits

    ⚠️ Always validate file paths to prevent directory traversal attacks

    💡 Return structured errors so the LLM can self-correct

    toolssecurityimplementation

    Memory Systems

    2 topics

    Types of Agent Memory

    The four memory layers every production agent needs

    1. IN-CONTEXT MEMORY (Working Memory)
       What: The messages list passed to the LLM
       Size: Limited by context window (128k–1M tokens)
       Speed: Instant — no retrieval needed
       Use: Current conversation, recent tool results
    
    2. EXTERNAL MEMORY (Long-term / Episodic)
       What: Vector DB storing past interactions
       Size: Unlimited
       Speed: ~50-200ms retrieval
       Use: User preferences, past sessions, documents
       Tools: Pinecone, Weaviate, Chroma, pgvector
    
    3. SEMANTIC MEMORY (Knowledge Base)
       What: Indexed documents, FAQs, product info
       Size: Unlimited
       Speed: ~50-200ms retrieval
       Use: RAG over company knowledge, docs
       Tools: LlamaIndex, LangChain, Chroma
    
    4. PROCEDURAL MEMORY (Skills)
       What: Saved prompts, few-shot examples, tool definitions
       Size: Small — just the patterns
       Speed: Loaded at startup
       Use: Teaching agent HOW to do things consistently
    
    STRATEGY:
    - Short session? → in-context only
    - Multi-session user? → in-context + external
    - Large knowledge base? → all four types

    💡 Don't stuff everything into context — retrieve only what's relevant

    ⚡ Summarize old turns instead of truncating — preserves key facts

    memoryarchitecture

    Implementing Memory with pgvector

    Store and retrieve agent memory using PostgreSQL + pgvector

    python
    from openai import OpenAI
    import psycopg2
    import json
    
    client = OpenAI()
    
    def embed(text: str) -> list[float]:
        """Create embedding for text."""
        res = client.embeddings.create(model="text-embedding-3-small", input=text)
        return res.data[0].embedding
    
    def save_memory(conn, user_id: str, content: str, memory_type: str = "episodic"):
        """Save a memory to the vector store."""
        vector = embed(content)
        with conn.cursor() as cur:
            cur.execute(
                "INSERT INTO agent_memory (user_id, content, embedding, type) VALUES (%s, %s, %s, %s)",
                (user_id, content, json.dumps(vector), memory_type)
            )
        conn.commit()
    
    def recall_memory(conn, user_id: str, query: str, top_k: int = 5) -> list[str]:
        """Retrieve relevant memories using cosine similarity."""
        vector = embed(query)
        with conn.cursor() as cur:
            cur.execute(
                """
                SELECT content
                FROM agent_memory
                WHERE user_id = %s
                ORDER BY embedding <=> %s::vector
                LIMIT %s
                """,
                (user_id, json.dumps(vector), top_k)
            )
            return [row[0] for row in cur.fetchall()]
    
    # Use in agent
    def build_system_prompt(user_id: str, task: str, conn) -> str:
        memories = recall_memory(conn, user_id, task)
        memory_block = "\n".join(f"- {m}" for m in memories)
        return f"""You are a personal assistant.
    
    Relevant memories about this user:
    {memory_block}
    
    Use these to personalize your response."""

    💡 Use text-embedding-3-small — it's 5x cheaper than large with 90% of the quality

    ⚡ Add a metadata filter (user_id, type) to avoid retrieving irrelevant memories

    memorypgvectorembeddings

    Agent Patterns

    3 topics

    ReAct (Reason + Act)

    The most common agent pattern — think before acting

    python
    # ReAct forces the model to reason before each action
    # Paper: https://arxiv.org/abs/2210.03629
    
    REACT_SYSTEM_PROMPT = """
    You are a research agent. Solve tasks step by step.
    
    For each step, use this EXACT format:
    
    Thought: [reason about what to do next]
    Action: [tool_name]
    Action Input: {"param": "value"}
    Observation: [tool result will be inserted here]
    
    ... repeat Thought/Action/Observation as needed ...
    
    Final Answer: [your complete answer to the original question]
    
    IMPORTANT:
    - Only call ONE tool per step
    - Always check the Observation before the next Thought
    - Stop when you have enough information for a Final Answer
    """
    
    # Example trace:
    # Thought: I need to find current Python popularity stats
    # Action: search_web
    # Action Input: {"query": "Python most popular language 2025"}
    # Observation: {"results": [{"title": "Python #1 for 4th year...", ...}]}
    # Thought: I have the data I need
    # Final Answer: Python remains the #1 language in 2025...

    💡 ReAct traces are excellent for debugging — you see exactly what the model was thinking

    ⚡ Works best with GPT-4o, Claude 3.5+ — weaker models struggle to maintain the format

    reactpatternsprompting

    Plan-and-Execute

    Separate planning from execution for complex multi-step tasks

    python
    from openai import OpenAI
    import json
    
    client = OpenAI()
    
    def plan(task: str) -> list[str]:
        """Step 1: Generate a plan as a list of steps."""
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[
                {"role": "system", "content": "You are a planning assistant. Break complex tasks into ordered, concrete steps. Return JSON: {steps: [str]}."},
                {"role": "user", "content": f"Task: {task}"}
            ],
            response_format={"type": "json_object"}
        )
        return json.loads(response.choices[0].message.content)["steps"]
    
    def execute_step(step: str, context: str) -> str:
        """Step 2: Execute one step with tools."""
        # Run single-step agent with ReAct
        return run_agent(f"Context so far:\n{context}\n\nCurrent step: {step}")
    
    def replan(original_task: str, steps: list, completed: list, last_result: str) -> list[str]:
        """Step 3: Replan if execution reveals new information."""
        response = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": f"Task: {original_task}\nCompleted: {completed}\nRemaining: {steps}\nLast result: {last_result}\n\nDo remaining steps still make sense? Return updated plan as JSON: {{steps: [str]}}"
            }],
            response_format={"type": "json_object"}
        )
        return json.loads(response.choices[0].message.content)["steps"]
    
    # Run the full pattern
    task = "Research and write a comparison of the top 3 vector databases"
    steps = plan(task)
    results = []
    for step in steps:
        result = execute_step(step, "\n".join(results))
        results.append(result)
        steps = replan(task, steps[steps.index(step)+1:], steps[:steps.index(step)+1], result)

    💡 Replanning is the key differentiator — static plans fail when reality differs from expectations

    ⚡ Use a cheaper model for planning, a stronger model for execution

    plan-executepatterns

    Reflection & Self-Critique

    Agents that review and improve their own outputs

    python
    def reflect_and_improve(task: str, initial_output: str, criteria: list[str]) -> str:
        """
        Reflexion pattern: agent critiques its own output
        and produces an improved version.
        Paper: https://arxiv.org/abs/2303.11366
        """
        criteria_str = "\n".join(f"- {c}" for c in criteria)
    
        # Step 1: Self-critique
        critique = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": f"""Task: {task}
    
    Output to review:
    {initial_output}
    
    Evaluate against these criteria:
    {criteria_str}
    
    List specific issues and improvements needed. Be critical."""
            }]
        ).choices[0].message.content
    
        # Step 2: Revise based on critique
        improved = client.chat.completions.create(
            model="gpt-4o",
            messages=[{
                "role": "user",
                "content": f"""Original task: {task}
    First attempt: {initial_output}
    Critique: {critique}
    
    Now produce an improved version that addresses all issues."""
            }]
        ).choices[0].message.content
    
        return improved
    
    # Usage
    result = reflect_and_improve(
        task="Write a Python function to parse CSV files",
        initial_output=agent_output,
        criteria=["Handles edge cases", "Has error handling", "Includes type hints", "Has docstring"]
    )

    💡 Even one reflection pass significantly improves output quality

    ⚡ Great for code generation, writing, and reasoning tasks

    reflectionpatternsquality

    Multi-Agent Systems

    2 topics

    Multi-Agent Architectures

    Patterns for coordinating multiple specialized agents

    1. SUPERVISOR PATTERN
       ┌──────────────┐
       │  Supervisor  │ ← decides who does what
       └──────┬───────┘
         ┌────┴─────┬──────────┐
       Worker1  Worker2    Worker3
       (research) (write)   (review)
    
       Best for: Tasks that can be split by role/specialty
       Tools: LangGraph with supervisor node
    
    2. PIPELINE PATTERN (Sequential)
       Researcher → Analyst → Writer → Editor → Output
    
       Best for: Tasks with a clear handoff sequence
       Each agent adds/transforms the previous output
    
    3. PARALLEL PATTERN
       Input ──┬── Agent A (research topic 1) ──┐
               ├── Agent B (research topic 2) ──┤→ Aggregator → Output
               └── Agent C (research topic 3) ──┘
    
       Best for: Independent subtasks that can run concurrently
       Reduces total latency significantly
    
    4. DEBATE PATTERN
       Agent A: Proposes solution
       Agent B: Critiques it
       Agent A: Defends or revises
       Judge Agent: Selects best outcome
    
       Best for: High-stakes decisions, code review, fact-checking
    
    5. HIERARCHICAL
       Manager Agent
       ├── Sub-manager A → Workers
       └── Sub-manager B → Workers
    
       Best for: Very complex tasks needing decomposition at multiple levels

    💡 Start with the simplest architecture that solves the problem

    ⚡ Multi-agent adds latency and cost — only use when a single agent truly can't do the job

    multi-agentarchitecture

    Multi-Agent with LangGraph

    Building a supervisor + worker system with LangGraph

    python
    from langgraph.graph import StateGraph, END
    from langchain_openai import ChatOpenAI
    from typing import TypedDict, Literal
    import operator
    from typing import Annotated
    
    llm = ChatOpenAI(model="gpt-4o")
    
    class State(TypedDict):
        task: str
        plan: list[str]
        results: Annotated[list[str], operator.add]  # Accumulate results
        final_output: str
        next: str
    
    # --- Specialized agents ---
    def researcher(state: State) -> State:
        """Searches and collects information."""
        result = run_agent_with_tools(state["task"], tools=[search_web, scrape_url])
        return {"results": [f"Research: {result}"], "next": "writer"}
    
    def writer(state: State) -> State:
        """Synthesizes research into a draft."""
        context = "\n".join(state["results"])
        result = llm.invoke(f"Write a report based on:\n{context}\n\nTask: {state['task']}")
        return {"results": [f"Draft: {result.content}"], "next": "reviewer"}
    
    def reviewer(state: State) -> State:
        """Reviews and improves the draft."""
        draft = state["results"][-1]
        reviewed = llm.invoke(f"Review and improve this draft:\n{draft}")
        return {"final_output": reviewed.content, "next": END}
    
    def supervisor(state: State) -> Literal["researcher", "writer", "reviewer", "__end__"]:
        """Routes to next agent based on state."""
        return state.get("next", "researcher")
    
    # --- Build graph ---
    graph = StateGraph(State)
    graph.add_node("researcher", researcher)
    graph.add_node("writer", writer)
    graph.add_node("reviewer", reviewer)
    graph.set_entry_point("researcher")
    graph.add_conditional_edges("researcher", supervisor)
    graph.add_conditional_edges("writer", supervisor)
    graph.add_conditional_edges("reviewer", supervisor)
    
    app = graph.compile()
    result = app.invoke({"task": "Write a report on the state of AI agents in 2025"})
    print(result["final_output"])

    💡 Use Annotated[list, operator.add] to accumulate results across agents

    ⚡ Each node should do ONE thing — resist the urge to pack multiple responsibilities into one agent

    langgraphmulti-agentimplementation

    Frameworks & Tools

    2 topics

    Framework Comparison

    Choosing the right framework for your agent

    FRAMEWORK         BEST FOR                         COMPLEXITY
    ─────────────────────────────────────────────────────────────
    LangGraph         Stateful multi-agent workflows   High
    (github.com/langchain-ai/langgraph)
    - Explicit graph structure
    - Built-in checkpointing/memory
    - Human-in-the-loop support
    - Production-ready
    
    LangChain Agents  Quick single-agent prototypes     Medium
    (python.langchain.com/docs/modules/agents)
    - Large tool ecosystem
    - Many pre-built agent types
    - Good for RAG + agent combos
    
    OpenAI Assistants Hosted stateful agents            Low
    (platform.openai.com/docs/assistants)
    - Managed threads + memory
    - Built-in code interpreter, file search
    - No infra to manage
    - Vendor lock-in
    
    CrewAI            Role-based multi-agent teams      Medium
    (docs.crewai.com)
    - Intuitive role/goal/backstory model
    - Good for content pipelines
    - Less flexible than LangGraph
    
    Autogen           Conversational multi-agent        Medium
    (microsoft.github.io/autogen)
    - Agents that chat with each other
    - Great for debate/critique patterns
    - Microsoft / Azure native
    
    Smolagents        Lightweight, code-first           Low
    (huggingface.co/docs/smolagents)
    - Agents write Python to solve tasks
    - Minimal abstractions
    - Good for open-source models
    
    Pydantic AI       Type-safe, production-first       Medium
    (ai.pydantic.dev)
    - Strong typing throughout
    - Great DX for Python developers
    - Newer but growing fast

    💡 LangGraph for production, LangChain Agents for prototyping, OpenAI Assistants for speed

    ⚡ You can always start with a simple loop and migrate to a framework later

    frameworkscomparisontools

    Sandboxed Code Execution

    Safe environments for agents that run code

    python
    # Option 1: E2B — cloud sandboxes (recommended for production)
    # https://e2b.dev
    from e2b_code_interpreter import Sandbox
    
    with Sandbox() as sbx:
        execution = sbx.run_code("""
    import pandas as pd
    df = pd.DataFrame({"a": [1,2,3], "b": [4,5,6]})
    print(df.describe())
    """)
        print(execution.text)   # stdout
        print(execution.error)  # stderr
    
    # Option 2: Docker subprocess (self-hosted)
    import docker
    
    client = docker.from_env()
    
    def run_in_docker(code: str) -> str:
        result = client.containers.run(
            image="python:3.12-slim",
            command=["python", "-c", code],
            mem_limit="128m",
            cpu_period=100000,
            cpu_quota=50000,    # 50% CPU
            network_disabled=True,  # No network access!
            remove=True,
            timeout=10
        )
        return result.decode()
    
    # Option 3: Pyodide (browser/WASM — zero server risk)
    # https://pyodide.org
    # Run Python in browser — no server execution

    ⚠️ Never let agents execute code directly on your server without sandboxing

    💡 E2B is the easiest production option — managed, fast, supports many languages

    securitycode-executionsandbox

    Popular Agent Tools

    2 topics

    Web Search & Browsing

    Giving agents access to the web

    python
    # Tavily — best for agents (structured, fast, cheap)
    # https://tavily.com
    from tavily import TavilyClient
    client = TavilyClient(api_key="tvly-...")
    result = client.search("LangGraph tutorial 2025", max_results=5)
    # Returns: {results: [{title, url, content, score}]}
    
    # Brave Search API (privacy-focused alternative)
    # https://api.search.brave.com
    import requests
    res = requests.get(
        "https://api.search.brave.com/res/v1/web/search",
        headers={"X-Subscription-Token": "your-key"},
        params={"q": "your query", "count": 5}
    )
    
    # Serper (Google results via API)
    # https://serper.dev
    import requests
    res = requests.post(
        "https://google.serper.dev/search",
        headers={"X-API-KEY": "your-key"},
        json={"q": "your query"}
    )
    
    # Browser automation — Playwright (for JS-heavy sites)
    # https://playwright.dev
    from playwright.async_api import async_playwright
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        page = await browser.new_page()
        await page.goto("https://example.com")
        content = await page.content()

    💡 Tavily is purpose-built for agents — returns clean text, not raw HTML

    ⚡ Use Playwright only when you need to interact with the page (click, fill forms)

    toolssearchweb

    File, Database & API Tools

    Connecting agents to your data and systems

    python
    # File operations tool
    def read_file(path: str, start_line: int = 0, end_line: int = 100) -> dict:
        """Read file with line range to avoid dumping large files into context."""
        lines = open(path).readlines()
        return {
            "content": "".join(lines[start_line:end_line]),
            "total_lines": len(lines),
            "showing": f"{start_line}-{end_line}"
        }
    
    def write_file(path: str, content: str) -> dict:
        with open(path, "w") as f:
            f.write(content)
        return {"success": True, "path": path}
    
    # SQL database tool
    import psycopg2
    def query_db(sql: str, params: list = None) -> dict:
        """Run read-only SQL queries."""
        # IMPORTANT: Use read-only DB user in production!
        conn = psycopg2.connect(DATABASE_URL)
        with conn.cursor() as cur:
            cur.execute(sql, params or [])
            rows = cur.fetchall()
            cols = [d[0] for d in cur.description]
        return {"columns": cols, "rows": rows[:50]}  # Limit rows
    
    # HTTP / REST API tool
    import httpx
    async def call_api(method: str, url: str, body: dict = None, headers: dict = None) -> dict:
        """Make HTTP requests. Use for calling external APIs."""
        async with httpx.AsyncClient(timeout=30) as client:
            response = await client.request(method, url, json=body, headers=headers)
            return {"status": response.status_code, "body": response.json()}

    ⚠️ Give the DB tool a read-only database user — never write access unless explicitly needed

    💡 Limit rows/lines returned to avoid blowing up the context window

    toolsdatabasefilesapi

    Human-in-the-Loop

    2 topics

    When & How to Add Human Checkpoints

    Pausing agents for human approval on critical actions

    python
    # LangGraph interrupt — pause for human approval
    from langgraph.graph import StateGraph
    from langgraph.checkpoint.memory import MemorySaver
    
    def agent_node(state):
        # ... agent logic ...
        return state
    
    def action_node(state):
        """Pauses here until human approves."""
        # This node will be interrupted before execution
        action = state["planned_action"]
        execute_action(action)  # Only runs after approval
        return {**state, "action_done": True}
    
    builder = StateGraph(State)
    builder.add_node("agent", agent_node)
    builder.add_node("action", action_node)
    builder.set_entry_point("agent")
    builder.add_edge("agent", "action")
    
    checkpointer = MemorySaver()
    # interrupt_before pauses BEFORE executing the node
    app = builder.compile(
        checkpointer=checkpointer,
        interrupt_before=["action"]  # Pause before action_node
    )
    
    config = {"configurable": {"thread_id": "task-1"}}
    
    # Run until interrupt
    app.invoke({"task": "Delete all inactive users"}, config)
    
    # Inspect what the agent planned
    state = app.get_state(config)
    print("Agent plans to:", state.values["planned_action"])
    
    # Human approves — resume
    app.invoke(None, config)  # Resume from checkpoint

    💡 Use interrupt_before for destructive actions: delete, send email, deploy, charge card

    ⚡ interrupt_after lets the node run but pauses before the next node — good for review

    human-in-loopsafetylanggraph

    HITL Patterns for Production

    Practical human oversight strategies

    WHEN TO REQUIRE HUMAN APPROVAL:
      ✅ Sending emails / messages on behalf of user
      ✅ Making purchases or financial transactions
      ✅ Deleting data or records
      ✅ Deploying code to production
      ✅ Posting to social media or public channels
      ✅ Accessing sensitive PII data
      ✅ Actions with real-world side effects
    
    WHEN AGENTS CAN RUN AUTONOMOUSLY:
      ✅ Read-only research tasks
      ✅ Drafting content (with human review before send)
      ✅ Data analysis and summarization
      ✅ Internal calculations
      ✅ Test environments (not prod)
    
    IMPLEMENTATION OPTIONS:
    
    1. Synchronous (blocking)
       Agent stops → sends Slack/email to human
       Human responds → agent resumes
       Pro: Simple. Con: Slow.
    
    2. Asynchronous with timeout
       Agent sends request with deadline
       Human approves before timeout → executes
       Timeout expires → safe fallback (skip/escalate)
    
    3. Optimistic with rollback
       Agent executes but logs everything
       Human can reverse within N hours
       Pro: Fast. Con: Requires undo capability.
    
    4. Risk scoring
       Low risk (<30%) → auto-approve
       Medium risk (30-70%) → queue for review
       High risk (>70%) → require immediate approval

    💡 Ship with more HITL first, then gradually remove as you gain confidence in the agent

    ⚠️ 'The agent will ask if unsure' is not a safety strategy — define exact trigger conditions

    human-in-loopsafetypatterns

    Evaluation & Observability

    2 topics

    Evaluating Agent Performance

    Metrics and methods for measuring agent quality

    python
    # LangSmith — best-in-class for LangChain/LangGraph agents
    # https://smith.langchain.com
    
    import langsmith
    from langsmith import Client
    
    # Auto-tracing — wraps every LLM call, tool call, chain step
    os.environ["LANGCHAIN_TRACING_V2"] = "true"
    os.environ["LANGCHAIN_API_KEY"] = "ls__..."
    os.environ["LANGCHAIN_PROJECT"] = "my-agent"
    
    # Create an evaluation dataset
    client = Client()
    dataset = client.create_dataset("agent-eval-v1")
    client.create_examples(
        inputs=[{"task": "Summarize this article: ..."}],
        outputs=[{"answer": "expected summary..."}],
        dataset_id=dataset.id
    )
    
    # Define an evaluator
    from langchain.smith import RunEvalConfig
    eval_config = RunEvalConfig(
        evaluators=[
            "qa",              # LLM-graded correctness
            "criteria",        # Check specific criteria
            "embedding_distance",  # Semantic similarity
        ]
    )
    
    # Run evaluation
    results = langsmith.aevaluate(
        run_agent,          # Your agent function
        data=dataset,
        evaluators=[...],
        experiment_prefix="gpt4o-react-v2"
    )

    💡 LangSmith is free for up to 5k traces/month — start using it on day 1

    ⚡ Build an eval dataset BEFORE you start optimizing — you need a baseline

    evaluationlangsmithobservability

    Key Metrics to Track

    What to measure in production agent systems

    TASK-LEVEL METRICS:
      Task success rate     — Did the agent complete the goal?
      Steps to completion   — Fewer steps = more efficient
      Error rate            — How often does it fail?
      Retry rate            — How often does it self-correct?
    
    QUALITY METRICS:
      Factual accuracy      — Are claims verifiable? (LLM-graded)
      Instruction following — Did it do what was asked?
      Format compliance     — Did it use the right output format?
      Hallucination rate    — Is it making things up?
    
    COST & LATENCY:
      Tokens per task       — Input + output tokens
      Cost per task         — Tokens × price/token
      Latency p50/p95       — Median and tail latency
      Tool calls per task   — Proxy for complexity
    
    SAFETY METRICS:
      Refusal rate          — Declining appropriate requests?
      Harmful action rate   — Taking actions it shouldn't?
      HITL trigger rate     — How often humans get called?
    
    OBSERVABILITY TOOLS:
      LangSmith             — langsmith.com (best for LangChain)
      Langfuse              — langfuse.com (open-source, self-host)
      Arize Phoenix         — arize.com/phoenix
      Helicone              — helicone.ai (proxy-based, easy setup)

    💡 Track cost per successful task — not just cost per run

    ⚡ p95 latency matters more than average for user-facing agents

    metricsobservabilityproduction

    Production Best Practices

    2 topics

    Production Checklist

    Everything you need before shipping an agent to production

    PRE-LAUNCH CHECKLIST:
    
    SAFETY
      ☐ Max iterations capped (never infinite loop)
      ☐ Tool inputs validated (no injection attacks)
      ☐ Code execution sandboxed (E2B, Docker)
      ☐ File paths restricted (no directory traversal)
      ☐ DB access is read-only (unless write is required)
      ☐ HITL gates on destructive actions
      ☐ Rate limiting on API tools
    
    RELIABILITY
      ☐ Retry logic with exponential backoff on tool failures
      ☐ Fallback behavior when tools are unavailable
      ☐ Timeout on every external call
      ☐ Graceful degradation (fail safe, not fail loud)
      ☐ State checkpointed (resume after crash)
    
    OBSERVABILITY
      ☐ Every LLM call logged (prompt + response)
      ☐ Every tool call logged (name + args + result)
      ☐ Latency and cost tracked per task
      ☐ Errors sent to alerting (PagerDuty, Slack)
      ☐ Eval dataset created and baseline measured
    
    COST CONTROL
      ☐ Per-task token budget defined
      ☐ Alerts on unexpected cost spikes
      ☐ Cheap model for planning, strong model for execution
      ☐ Prompt caching enabled (Anthropic/OpenAI)
    
    USER EXPERIENCE
      ☐ Streaming output so user sees progress
      ☐ Clear indication agent is working (spinner/status)
      ☐ User can cancel a running task
      ☐ Helpful error messages (not raw stack traces)

    💡 Ship with all safety checks on, then remove them one by one as trust is earned

    ⚠️ 'It works in testing' does not mean it's safe in production

    productionchecklistsafety

    Error Handling & Retry Strategy

    Making agents resilient to failures

    python
    import time
    import random
    from functools import wraps
    
    def retry_with_backoff(max_retries=3, base_delay=1.0, exceptions=(Exception,)):
        """Decorator for exponential backoff retry."""
        def decorator(fn):
            @wraps(fn)
            def wrapper(*args, **kwargs):
                for attempt in range(max_retries):
                    try:
                        return fn(*args, **kwargs)
                    except exceptions as e:
                        if attempt == max_retries - 1:
                            raise
                        delay = base_delay * (2 ** attempt) + random.uniform(0, 0.5)
                        print(f"[Retry] Attempt {attempt+1} failed: {e}. Retrying in {delay:.1f}s...")
                        time.sleep(delay)
            return wrapper
        return decorator
    
    @retry_with_backoff(max_retries=3, base_delay=1.0)
    def call_llm(messages):
        return client.chat.completions.create(model="gpt-4o", messages=messages)
    
    # Tool-level error handling — return errors for LLM to self-correct
    def safe_tool_call(fn, *args, **kwargs) -> dict:
        try:
            result = fn(*args, **kwargs)
            return {"success": True, "result": result}
        except ValueError as e:
            return {"success": False, "error": f"Invalid input: {e}", "hint": "Check the parameter format"}
        except TimeoutError:
            return {"success": False, "error": "Tool timed out", "hint": "Try a simpler query"}
        except Exception as e:
            return {"success": False, "error": str(e)}
        # The LLM can read the error and self-correct on the next iteration

    💡 Return structured errors to the LLM — it will often self-correct without human intervention

    ⚡ Add hints in error messages to guide the model toward a fix

    error-handlingreliabilitypatterns

    Related Articles

    Background reading and deeper explanations for this sheet.

    Agent Routing vs Model Routing: Real Production Patterns, Tradeoffs, and Implementation Guide

    Choosing between agent routing and model routing can make or break your AI product’s cost, speed, and reliability. In this practical guide, you’ll learn what each pattern is, when to use it, and how production teams combine both to ship robust systems. We’ll walk through real examples, architecture patterns, and code you can adapt immediately.

    keyword overlap

    AWS AI & App Modernization Guide: SageMaker, Bedrock, ECS/EKS, and Lambda in Practice

    AWS offers multiple paths to build intelligent, scalable applications—but choosing the right service can feel overwhelming. In this practical guide, you’ll learn what SageMaker, Bedrock, ECS/EKS, and Lambda are best at, how they fit together, and how to apply them in real-world architectures with beginner-friendly clarity and technical depth.

    keyword overlap

    Build an Agentic App with FastAPI and Azure AI Foundry: A Practical Beginner-to-Pro Guide

    Learn how to design, build, and deploy an agentic application using FastAPI and Azure AI Foundry with practical, real-world examples. This guide walks you from architecture and setup to tool-calling, memory, observability, and production deployment patterns.

    keyword overlap

    Building Autonomous Agents Without Overengineering: A Practical Guide for Real-World Teams

    Autonomous agents can automate meaningful work, but many projects fail because teams overbuild too early. This practical guide shows you how to design, ship, and improve useful agents with simple patterns, clear guardrails, and real-world examples—without turning your stack into a science project.

    keyword overlap