Structured Outputs: How to Force AI to Return Reliable JSON

Three ways to get JSON out of a model

• JSON mode — the model commits to valid JSON, but not to your schema. Cheap, fast, weak.
• Response schemas / structured outputs — the model commits to a JSON Schema you supply. The right default in 2026.
• Tool calls / function calling — the model selects a 'function' and returns arguments matching its schema. Best for branching workflows.

Designing a schema that survives production

• Keep fields shallow — nested objects 2 levels deep max.
• Make every field required and use enums wherever possible.
• Add a 'confidence' float and a 'reasoning' string — they make debugging trivial.
• Version the schema in the field name (v2_category) so you can roll forward without breaking history.

Failure modes you'll hit

• Provider-side schema cache delays — first call after a deploy takes 2-5 seconds.
• Long-tail tokens overflow — set a max_tokens ceiling above your worst-case output.
• Schema-valid but semantically wrong output (right shape, wrong content) — only catchable with eval suites.
• Silent provider downgrade to non-structured mode under load — log raw responses.

Retry strategy

On parse failure, retry once with the same prompt at temperature 0. If it fails again, retry with the parser error appended ('Your last response was invalid because…'). After three failures, route to human review — don't loop forever.

Next step

The Opsacea Toolkit ships reference schemas for the most common workflows — triage, extraction, classification and drafting — plus the n8n nodes wired with proper retry logic.