Structured Output With Tool Use

Summary: Asking an LLM to "return JSON" in the prompt is just a formatting hint the model can ignore; tool use forces the model through a real schema, and a validation library still checks the values afterward.

At a glance

Question	Answer
What is it?	A way to make an LLM call return data in a guaranteed shape
Why does it matter?	Plain prompt instructions for JSON don't enforce structure — the model can still drift, omit fields, or wrap it in prose
When do I use it?	Any time downstream code parses the model's output programmatically (routing decisions, extracted fields, classifications)

The problem

Prompting "respond only in JSON with fields X and Y" is text in the prompt, not a constraint on the model. It can still:

• add commentary before/after the JSON;
• omit a required field;
• use the wrong type (a string where a number was expected);
• invent a field name that's close but not exact.

Step 1: define the schema as a real API parameter

Instead of describing the shape in words, you declare it as a tool — a JSON Schema object passed to the API, not the prompt:

tools = [
    {
        "name": "record_decision",
        "description": "Record whether the story has enough information to proceed.",
        "input_schema": {
            "type": "object",
            "properties": {
                "decision": {"type": "string", "enum": ["sufficient", "insufficient"]},
                "reason": {"type": "string", "description": "Short explanation for the decision"}
            },
            "required": ["decision", "reason"]
        }
    }
]

• name / description: tell the model what this tool is for — the description is part of how the model decides what to fill in.
• input_schema: a JSON Schema object. properties lists each field with its type; enum restricts a string field to an exact set of allowed values (here, only "sufficient" or "insufficient" — not free text).
• required: fields the model cannot omit.

This is the actual contract. The model isn't guessing the shape from prose — it's filling in a defined object.

Step 2: force the model to use it

Passing tools alone only makes the tool available — tool_choice controls whether and how it's actually used:

tool_choice	Behavior
{"type": "auto"}	Default when tools is provided. Claude decides whether to call any tool at all.
{"type": "any"}	Claude must use one of the provided tools, but not a specific one.
{"type": "tool", "name": "..."}	Forces Claude to always use one named tool — what this note's example uses.
{"type": "none"}	Prevents tool use entirely. Default when tools isn't provided.

response = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "tool", "name": "record_decision"},
    messages=[{"role": "user", "content": f"Story: {story_text}\n\nDoes this have enough information to proceed?"}]
)

With tool_choice forced to any or tool, the response contains a tool_use content block whose input matches the schema, instead of freeform prose to parse.

Worth knowing: forced tool use (any or tool) isn't compatible with extended thinking on every model — check current model support before combining the two. There's also a strict: true option on a tool definition that guarantees the returned input matches the schema exactly, which can reduce (though not eliminate) the need for a separate validation pass.

Step 3: validate the values, not just the shape

A schema-conformant response guarantees shape (right fields, right types) — it doesn't guarantee the values make sense together, or catch a bug in the schema itself. Layer a validation library on top regardless of whether strict is set:

from pydantic import BaseModel, ValidationError
from typing import Literal

class Decision(BaseModel):
    decision: Literal["sufficient", "insufficient"]
    reason: str

tool_block = next(block for block in response.content if block.type == "tool_use")

try:
    result = Decision(**tool_block.input)
except ValidationError as e:
    raise

On a ValidationError, log it and decide whether to retry the call or flag the record for manual review — don't fail silently.

Worth knowing: libraries like Instructor package this entire loop — schema definition, the forced call, and an automatic retry when validation fails — on top of Pydantic (or Zod in TypeScript). Worth reaching for once you've written the same try/validate/retry block for the third time.

Remember

• Prompt-only JSON instructions are a hint, not a guarantee — the model can still drift.
• tools makes a schema available; tool_choice decides whether and how it's actually used — any/tool are what force it.
• strict: true tightens the guarantee on the API side, but validating with something like Pydantic is still worth doing.

Related notes

• LLM Workflow Patterns - routing and evaluator-optimizer both rely on this technique for reliable output