Create a Generative Email Triage App with FastAPI, Postgres, LangGraph, and Azure Foundry

Email overload is a real operational problem for support, sales, and operations teams. A generative email triage app can automatically classify incoming messages, extract intent, suggest responses, and route tasks to the right queue. In this guide, you’ll build one using FastAPI for APIs, Postgres for durable storage, LangGraph for robust LLM workflow orchestration, and Azure Foundry for enterprise-grade model access and governance. The goal is practical: you’ll learn not just what to build, but why each design choice matters in production.

# 1) What You’re Building and Why This Stack Works

The target system processes inbound emails and outputs structured triage decisions, such as:

• Category: billing, outage, feature request, legal, spam
• Priority: low, medium, high, urgent
• Sentiment: frustrated, neutral, positive
• Action: auto-reply draft, assign human, escalate
• Confidence: numeric score for quality control

This stack is effective because each component has a clear responsibility:

• FastAPI: async APIs, validation, and clean service boundaries
• Postgres: reliable transactional storage for emails, outputs, and audit trails
• LangGraph: stateful multi-step reasoning with retries, branching, and tool calls
• Azure Foundry: managed model access, prompt/version governance, and enterprise controls

Key architectural principle: keep LLM calls stateless and deterministic where possible, but wrap them in stateful orchestration (LangGraph) and persistent logging (Postgres) for recoverability and compliance.

# 2) System Architecture and Data Flow (Production View)

# Core flow

1. Email enters system through webhook (e.g., Microsoft Graph, Gmail API, or SMTP ingestion).
2. FastAPI receives payload, normalizes fields, stores raw event in Postgres.
3. FastAPI triggers LangGraph workflow with email ID and content.
4. LangGraph runs steps: classify, extract entities, risk-check, draft response, route.
5. Results are saved in Postgres and optionally pushed to ticketing tools (Zendesk, Jira, ServiceNow).

# Recommended Postgres tables

• emails: id, source, sender, subject, body, received_at, thread_id
• triage_results: email_id, category, priority, sentiment, confidence, recommended_action
• draft_replies: email_id, model, prompt_version, draft_text, safety_flags
• workflow_runs: run_id, graph_version, step, status, error, latency_ms
• audit_logs: actor, action, before_json, after_json, timestamp

# Why this matters in real teams

Imagine a fintech support team receives 8,000 emails/day. If your LLM output is not persisted with prompt version and model version, you cannot reliably explain why a message was escalated or auto-responded. This is a compliance risk. The schema above enables auditability and measurable quality iteration.

# 3) Build the FastAPI Service Layer

# Project structure

app/
  main.py
  api/
    ingest.py
    triage.py
  db/
    models.py
    session.py
  services/
    langgraph_runner.py
    azure_foundry_client.py
  schemas/
    email.py
    triage.py

# FastAPI endpoint example

from fastapi import FastAPI, BackgroundTasks
from pydantic import BaseModel
from app.services.langgraph_runner import run_triage_graph

app = FastAPI(title="Generative Email Triage API")

class InboundEmail(BaseModel):
    message_id: str
    sender: str
    subject: str
    body: str
    received_at: str

@app.post("/ingest/email")
async def ingest_email(payload: InboundEmail, bg: BackgroundTasks):
    # 1) Persist raw email (pseudo)
    email_id = save_email_to_db(payload.dict())

    # 2) Run graph async so webhook responds quickly
    bg.add_task(run_triage_graph, email_id)

    return {"status": "accepted", "email_id": email_id}

# Beginner-friendly production tips

• Return quickly from webhook endpoints to avoid source retries.
• Use idempotency key = message_id to prevent duplicates.
• Log correlation IDs for each workflow run.
• Add request/response schema validation with Pydantic.

Tip: Don’t let your API wait for full LLM triage. Queue or background execution improves reliability and user experience.

# 4) Orchestrate Triage Logic with LangGraph

# Why LangGraph over a single prompt

A single prompt can classify emails, but real triage requires control flow:

• If confidence is low, route to human review.
• If legal/compliance signals appear, trigger specialized checks.
• If customer is VIP and urgent, escalate immediately.

LangGraph enables explicit nodes and conditional edges, making the process testable and safer.

# Example graph nodes

• Normalize: clean signatures, remove quoted history
• Classify: category + priority + sentiment
• Policy Check: PII, regulated content, legal hold
• Draft: suggested response with style constraints
• Route: queue assignment and SLA mapping

def classify_node(state):
    prompt = f"""
    You are an email triage assistant.
    Return JSON with keys: category, priority, sentiment, confidence.
    Email:
    Subject: {state['subject']}
    Body: {state['body']}
    """
    result = azure_foundry_llm(prompt)
    return {**state, **result}


def route_node(state):
    if state["confidence"] < 0.70:
        state["route"] = "human_review"
    elif state["priority"] == "urgent":
        state["route"] = "incident_queue"
    else:
        state["route"] = "standard_support"
    return state

In real operations, this branching prevents over-automation and protects quality.

# 5) Connect to Azure Foundry for Model Inference and Governance

# Practical setup considerations

• Store API credentials in Key Vault or environment secrets, not source code.
• Version prompts as artifacts (prompt_v1, prompt_v2) and track output quality.
• Use content filters and safety policies before auto-sending drafts.
• Capture model name, deployment, and temperature per run for reproducibility.

# Simple client wrapper pattern

import os
from openai import AzureOpenAI

client = AzureOpenAI(
    api_key=os.environ["AZURE_OPENAI_API_KEY"],
    api_version="2024-10-21",
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
)


def azure_foundry_llm(prompt: str) -> dict:
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": "Return strictly valid JSON."},
            {"role": "user", "content": prompt},
        ],
        temperature=0.1,
    )
    text = response.choices[0].message.content
    return parse_json_safely(text)

# Real-world example: support escalation

Input email: “We were charged twice and cannot process payroll today. Please help ASAP.”

• Category: billing_issue
• Priority: urgent
• Sentiment: frustrated
• Route: incident_queue
• Draft: empathetic + asks for invoice IDs + promises 1-hour response SLA

That draft should be flagged human-approve required for urgent financial impact cases.

# 6) Testing, Evaluation, and Deployment Checklist

# Evaluation metrics that matter

• Classification accuracy by category and priority
• False urgent rate (too many unnecessary escalations)
• Missed urgent rate (critical incidents not escalated)
• Draft acceptance rate by human agents
• Latency p95 end-to-end triage time

# Testing strategy

1. Build a labeled dataset of historical emails (start with 500–2,000 samples).
2. Run offline batch evaluation before production rollout.
3. Deploy with shadow mode: generate outputs but do not auto-act.
4. Enable partial automation only for high-confidence low-risk classes.

# Deployment essentials

• Containerize FastAPI with health checks.
• Use managed Postgres with backups and read replicas where needed.
• Add retries with exponential backoff around model calls.
• Implement dead-letter queue for failed workflow runs.
• Set alerts for error spikes, latency regressions, and low confidence drift.

Warning: Never auto-send generated replies in regulated workflows (finance, legal, healthcare) without policy checks and approval gates.

# Conclusion

Building a generative email triage app using FastAPI, Postgres, LangGraph, and Azure Foundry gives you a practical and scalable foundation for handling high-volume inboxes. FastAPI keeps ingestion and APIs clean, Postgres ensures traceability, LangGraph adds dependable decision flows, and Azure Foundry brings enterprise model operations. Start with classification and routing, then layer in drafting and selective automation as your metrics improve. If you focus on observability, evaluation, and safe rollout, you can reduce response times dramatically while keeping humans in control where it matters most.