What are structured outputs?

You send a schema with the request: text.format with type: "json_schema" and strict: true in OpenAI’s Responses API, output_config.format in Anthropic’s Messages API, and response_format with a schema in Gemini. Generation is then restricted to output that fits. Anthropic notes that the first request with a new schema is slower while its grammar compiles, and that compiled grammars are cached for 24 hours.

The guarantee has documented gaps. If the model refuses, or the reply hits the max output tokens limit, the output may not match the schema, so check for a refusal or an incomplete status (OpenAI) or the stop_reason (Anthropic) before parsing. Anthropic also notes that the capitalisation of enum values isn’t guaranteed. And every provider supports only part of JSON Schema.

A schema that works on one provider can fail on another. OpenAI requires every field to be listed in required (make one optional with a union such as ["string", "null"]) and supports minimum and maximum. Anthropic returns a 400 error for numeric limits such as minimum. Gemini accepts them but warns that very large or deeply nested schemas may be rejected.

Structured outputs also don’t check that values are true. A schema guarantees an email field holding a string, not the right email, so validate the content, and watch for hallucinated values in fields the source never mentioned.

Without a schema, replies can come back malformed or missing fields, and each one costs a retry or a parser crash. With structured outputs on, you handle the documented edge cases (refusal, truncation) instead of guessing at broken JSON.

Structured outputs vs JSON mode

OpenAI’s JSON mode (type: "json_object") guarantees only that the output parses, not that it matches any schema, and the API returns an error unless the word “JSON” appears somewhere in the context. OpenAI calls Structured Outputs the evolution of JSON mode.

Structured outputs vs function calling

Both use JSON Schema. Function calling shapes the arguments of a tool call your code will run; structured outputs shape the reply itself. Anthropic’s strict tool use (strict: true) is the structured outputs guarantee applied to tool arguments.

• Free toolJSON Repair→Fix broken JSON from LLM output.
• Guide · 10 min readHow to fix invalid JSON from LLMs (and stop getting it)Why ChatGPT, Claude and Gemini return broken JSON, and how to fix it: structured outputs, schema validation, retries, safe repair and streaming.

• Function callingFunction calling, also called tool use, is an LLM API feature in which the model replies with a structured request to run a function you described, with JSON arguments, which your code executes before sending the result back.
• Max output tokensMax output tokens is the cap on how many tokens a model may generate in one response, set per request up to the model’s own limit, with any reasoning tokens counted inside it.
• HallucinationA hallucination is a confident, plausible-sounding statement from a language model that is false or unsupported by its input, such as an invented citation, date, function or quote.
• OpenAI-compatible APIAn OpenAI-compatible API is a model API that accepts OpenAI’s Chat Completions request format, so you can call it with the official OpenAI SDK by changing only the base URL, the API key and the model name.

All 40 terms in the AI glossary