GenAI Course · Module 8 · Session 28

Structured Output & Reliability

⏱ 13 min read🎯 Intermediate🖼 2 diagrams🧪 2 live demos🐍 Python code

A chatbot can reply in free text, but software needs structured data: an amount as a number, a date in a fixed format, a label from an approved list. In this session, let us learn how to make LLM output reliable enough to feed directly into your code.

In this session

Why free text breaks software
JSON mode, response schemas and function/tool schemas
Pydantic models and output parsers
Guardrails, validation and retry strategies, with code that we actually ran

01The problem: free text breaks code

Say you want to read bank SMS messages and store each transaction in a database:

SMS: Rs.2,499.00 debited from A/c XX1234 on 25-10-26 to AMAZON. UPI Ref 6312.

02Three ways to get structured output

From weakest to strongest guarantee: JSON mode → schema → schema + validation in your code.

json_mode.py

from openai import OpenAI
client = OpenAI()

resp = client.chat.completions.create(
    model="gpt-4o-mini",
    response_format={"type": "json_object"},      # JSON mode
    messages=[
        {"role": "system", "content": "Extract the transaction. Reply in JSON with keys: amount, type, account_last4, merchant, date (YYYY-MM-DD)."},
        {"role": "user", "content": "Rs.2,499.00 debited from A/c XX1234 on 25-10-26 to AMAZON."},
    ],
)
print(resp.choices[0].message.content)

Function / tool schemas

A tool schema describes a function the model may “call”. The model never runs it; it returns a name and arguments, and your code runs it (see the Tool Calling lesson in Agentic AI).

tool_schema.json

{
  "type": "function",
  "function": {
    "name": "block_card",
    "description": "Block the customer's card after they report it lost or stolen",
    "parameters": {
      "type": "object",
      "properties": {
        "card_last4": {"type": "string", "pattern": "^[0-9]{4}$"},
        "reason": {"type": "string", "enum": ["lost", "stolen", "fraud"]}
      },
      "required": ["card_last4", "reason"]
    }
  }
}

03Pydantic models and output parsers

Pydantic lets you define the shape of your data as a Python class. It can produce a JSON Schema for the LLM, and it validates what comes back.

schema.py

from typing import Literal
from pydantic import BaseModel, Field

class Transaction(BaseModel):
    amount: float = Field(gt=0, description="Amount in rupees")
    type: Literal["debit", "credit"]
    account_last4: str = Field(pattern=r"^\d{4}$")
    merchant: str
    date: str = Field(description="YYYY-MM-DD")

Now feed it a typical messy reply and see exactly what is wrong:

validate.py

from pydantic import ValidationError

bad = '{"amount": "2,499", "type": "Debit", "account_last4": "XX1234", "merchant": "AMAZON", "date": "2026-10-25"}'
try:
    Transaction.model_validate_json(bad)
except ValidationError as e:
    for err in e.errors():
        print(err["loc"], err["msg"])

('amount',) Input should be a valid number, unable to parse string as a number ('type',) Input should be 'debit' or 'credit' ('account_last4',) String should match pattern '^\d{4}$'

In LangChain, the same Pydantic class can be passed straight to a chat model. It builds the schema, calls the model and returns a validated object:

langchain_structured.py

# pip install langchain-openai
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
extractor = llm.with_structured_output(Transaction)

txn = extractor.invoke("Rs.2,499.00 debited from A/c XX1234 on 25-10-26 to AMAZON.")
print(txn.amount, txn.type)      # a real Transaction object, not a string

04Guardrails, validation and retries

Even with schemas, treat every LLM output as untrusted input. A reliable pipeline has these layers:

Never let an unvalidated LLM reply touch your database, payments or customers.

extract_with_retry.py

from pydantic import ValidationError

def extract(sms, call_llm, max_tries=3):
    prompt = f"Extract the transaction as JSON matching this schema:\n{Transaction.model_json_schema()}\nSMS: {sms}"
    for attempt in range(1, max_tries + 1):
        raw = call_llm(prompt)
        try:
            return Transaction.model_validate_json(raw)
        except ValidationError as e:
            print(f"Attempt {attempt} failed: {e.error_count()} errors")
            prompt += f"\nYour last answer was invalid: {e.errors(include_url=False)}. Fix it and reply with JSON only."
    raise RuntimeError("Could not get valid JSON")   # send to human review

Attempt 1 failed: 3 errors amount=2499.0 type='debit' account_last4='1234' merchant='AMAZON' date='2026-10-25'

Output from running this code with a stand-in LLM whose first reply was the messy JSON above.

Good habits

Use temperature 0 (or close) for extraction.
Use enums (Literal[...]) for labels so the model can’t invent new ones.
Add business-rule checks after schema checks (amount limits, valid dates, known merchants).
Log every failure: they show you how to improve the prompt or schema.
Libraries such as Guardrails AI, NeMo Guardrails and Instructor can help, but the idea is always the same: validate, then retry or fall back.

05Summary

Software needs structured output, not free text.
JSON mode guarantees valid JSON; schemas and tool definitions guarantee the right fields.
Pydantic defines the schema and validates replies; LangChain’s with_structured_output does both for you.
Add guardrails, validation, retries and a fallback before anything reaches production systems.

06Quick quiz

1. What does JSON mode guarantee?

2. How do you stop the model inventing new labels?

3. Validation failed. Best next step?

4. With a tool schema, who actually runs the function?

5. After 3 failed retries, you should…

07References

Created by Ajeetkumar