Mastering Structured Outputs: JSON Schema with OpenAI and Anthropic

Parsing information out of unstructured model responses is one of the most frustrating parts of building AI-driven software. Prompting the model to "Return only valid JSON" frequently results in markdown wrappers (```json), missing keys, or trailing commas that crash your parser. Today, **OpenAI** and **Anthropic** provide native ways to guarantee that the model outputs structured JSON matching a strict schema.

The Paradigm Shift: Schema Constraints

Native structured output systems do not simply format the text post-generation. Instead, during token generation, the API constraints the model’s vocabulary logits. If the next character must be a quotation mark or a bracket to satisfy the schema, the model is physically prevented from selecting any other token. This results in **100% valid JSON** matching your definition.

1. OpenAI Structured Outputs (JSON Schema)

OpenAI supports native schema enforcement. Using Pydantic in Python, you can define your schema as a standard class, and the SDK automatically serializes it to JSON Schema.

Python Example with OpenAI and Pydantic

from openai import OpenAI
from pydantic import BaseModel, Field

client = OpenAI()

# Define the target structure
class UserProfile(BaseModel):
    name: str
    age: int
    programming_languages: list[str] = Field(description="Languages they write fluently")
    is_active: bool

# Make the API call
completion = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "user", "content": "Extract: John is 28, writes Python and Go."}
    ],
    response_format=UserProfile,
)

user = completion.choices[0].message.parsed
print(user.name)  # Output: John
print(user.programming_languages)  # Output: ['Python', 'Go']

2. Anthropic Structured Outputs (Tool Choice)

Anthropic does not have a dedicated response_format parameter for JSON schemas. Instead, Anthropic leverages its **Tool Use** API. By defining a tool that accepts your schema as its input parameters and forcing the model to call that tool, you achieve identical schema-compliant structured outputs.

Python Example with Anthropic Tool Choice

import anthropic

client = anthropic.Anthropic()

# Define schema parameters as a JSON schema
user_schema = {
    "name": "save_user_profile",
    "description": "Save extracted user profile information.",
    "input_schema": {
        "type": "object",
        "properties": {
            "name": {"type": "string"},
            "age": {"type": "integer"},
            "programming_languages": {
                "type": "array", 
                "items": {"type": "string"}
            },
            "is_active": {"type": "boolean"}
        },
        "required": ["name", "age", "programming_languages", "is_active"]
    }
}

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    tools=[user_schema],
    # Force the model to call this tool
    tool_choice={"type": "tool", "name": "save_user_profile"},
    messages=[
        {"role": "user", "content": "Extract: John is 28, writes Python and Go."}
    ]
)

# Extract tool input
tool_use = [block for block in response.content if block.type == "tool_use"][0]
print(tool_use.input) # Output: {'name': 'John', 'age': 28, ...}

Trade-offs and Performance