How to fix invalid JSON from LLMs (and stop getting it)

Why do LLMs return invalid JSON?#

Because, unless you constrain it, a model writes JSON the way it writes prose: one token at a time, with nothing checking the syntax. It learned from JavaScript, Python and chat transcripts, where single quotes, trailing commas, comments and True are normal, and chat models like to wrap code in Markdown and add a friendly sentence. The common failures:

• Wrapping: a sentence before the JSON, or a Markdown code fence around it.
• Loose syntax: single quotes, unquoted keys, trailing commas, comments, Python’s True, False and None.
• Bad strings: unescaped double quotes or raw line breaks inside a value.
• Truncation: the reply hits max_tokens and stops mid-value, leaving brackets open.
• Placeholders: ... in an array where the model shortened a list instead of writing every item.

A single one of these makes JSON.parse or json.loads reject the whole reply. The fix is mostly upstream.

How do you make an LLM return valid JSON?#

Turn on structured outputs. All three major APIs can constrain the model to a JSON Schema you supply, which is far more reliable than asking for JSON in the prompt. The OpenAI and Anthropic SDKs also accept a Pydantic model (Python) or a Zod schema (TypeScript) and convert it for you; for Gemini you pass the JSON Schema that Pydantic or Zod generates, as in the Gemini example below:

from openai import OpenAI
from pydantic import BaseModel

client = OpenAI()


class ContactInfo(BaseModel):
    name: str
    email: str
    plan_interest: str
    demo_requested: bool


response = client.responses.parse(
    model="gpt-6-astra",
    input=[
        {"role": "system", "content": "Extract the contact information."},
        {"role": "user", "content": "John Smith (john@example.com) is interested in our Enterprise plan and wants to schedule a demo for next Tuesday at 2pm."},
    ],
    text_format=ContactInfo,
)

contact = response.output_parsed

from anthropic import Anthropic
from pydantic import BaseModel


class ContactInfo(BaseModel):
    name: str
    email: str
    plan_interest: str
    demo_requested: bool


client = Anthropic()

response = client.messages.parse(
    model="claude-opus-5-5",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Extract the key information from this email: John Smith (john@example.com) is interested in our Enterprise plan and wants to schedule a demo for next Tuesday at 2pm.",
        }
    ],
    output_format=ContactInfo,
)
print(response.parsed_output)

from google import genai
from pydantic import BaseModel


class ContactInfo(BaseModel):
    name: str
    email: str
    plan_interest: str
    demo_requested: bool


client = genai.Client()

response = client.models.generate_content(
    model="gemini-3.8-flash",
    contents="Extract the contact information: John Smith (john@example.com) is interested in our Enterprise plan and wants to schedule a demo for next Tuesday at 2pm.",
    config={
        "response_format": {"text": {"mime_type": "application/json", "schema": ContactInfo.model_json_schema()}},
    },
)

contact = ContactInfo.model_validate_json(response.text)

Is JSON mode enough?

No. OpenAI’s older JSON mode (type: "json_object") makes the output parse but doesn’t enforce any schema, and you must still say “JSON” in a message or the API returns an error. OpenAI recommends Structured Outputs instead wherever the model supports it. The main reason to stay on JSON mode is an older model without schema support, and then you have to validate everything yourself.

How to validate LLM JSON with a schema#

Parse, then validate every reply against a schema, even with structured outputs on. Google’s docs say it plainly: structured output guarantees syntactically correct JSON, not correct values. Anthropic adds that string enum values may come back with different capitalisation. A schema catches both:

import { z } from "zod";

const Contact = z.object({
  name: z.string(),
  plan: z.enum(["free", "pro", "enterprise"]),
  seats: z.number().int().positive(),
});

const raw = '{"name": "Ada", "plan": "Enterprise", "seats": "5"}';
const result = Contact.safeParse(JSON.parse(raw));
if (!result.success) console.log(z.prettifyError(result.error));

// ✖ Invalid option: expected one of "free"|"pro"|"enterprise"
//   → at plan
// ✖ Invalid input: expected number, received string
//   → at seats

from typing import Literal
from pydantic import BaseModel, Field, ValidationError

class Contact(BaseModel):
    name: str
    plan: Literal["free", "pro", "enterprise"]
    seats: int = Field(gt=0)

raw = '{"name": "Ada", "plan": "Enterprise", "seats": "5"}'
try:
    Contact.model_validate_json(raw)
except ValidationError as e:
    print(e)

# 1 validation error for Contact
# plan
#   Input should be 'free', 'pro' or 'enterprise' [type=literal_error, input_value='Enterprise', input_type=str]
#     For further information visit https://errors.pydantic.dev/2.14/v/literal_error

Notice the difference. Zod rejected the string "5"; Pydantic quietly converted it to the number 5, because its default “lax” mode coerces numeric strings. That’s often what you want from a model, but if it isn’t, set model_config = ConfigDict(strict=True): the same input then fails with “Input should be a valid integer”. model_validate_json also reports syntax errors with a position, for example “Invalid JSON: trailing comma at line 1 column 13”, so one call covers both checks.

What to do when the JSON is cut off#

Check why generation stopped before you parse anything. A reply that ran out of tokens is incomplete, and no repair can bring back what was never written. Structured outputs don’t protect you here: OpenAI and Anthropic both document that a reply cut off by the token limit may not match your schema.

When it happens, retry with a higher output limit, ask for less (fewer items, shorter strings), or split the job into smaller requests. To size the limit, count a typical good reply with the token counter and leave generous headroom: reasoning models also spend output tokens before they write the JSON. Our guide to counting tokens in Python and JavaScript shows how to do it in code.

How to retry when validation fails#

Send the error back. A model that sees “Input should be ‘free’, ‘pro’ or ‘enterprise’” usually fixes its next reply, which is cheaper and safer than guessing what it meant. Cap the attempts so a stubborn failure doesn’t loop forever:

from typing import Literal
from pydantic import BaseModel, Field, ValidationError

class Contact(BaseModel):
    name: str
    plan: Literal["free", "pro", "enterprise"]
    seats: int = Field(gt=0)

def get_contact(prompt, call_model, max_attempts=3):
    """call_model(messages) -> str is your API call. Retries with the error."""
    messages = [{"role": "user", "content": prompt}]
    for _ in range(max_attempts):
        text = call_model(messages)
        try:
            return Contact.model_validate_json(text)
        except ValidationError as e:
            messages += [
                {"role": "assistant", "content": text},
                {"role": "user", "content": f"That JSON failed validation:\n{e}\nReply with the corrected JSON only."},
            ]
    raise RuntimeError(f"No valid JSON after {max_attempts} attempts")

# Stand-in for a real model: wrong the first time, right the second.
replies = iter(['{"name": "Ada", "plan": "Enterprise", "seats": 5}',
                '{"name": "Ada", "plan": "enterprise", "seats": 5}'])
print(get_contact("Extract the contact.", lambda messages: next(replies)))
# name='Ada' plan='enterprise' seats=5

• Don’t retry blindly. A refusal or a token-limit stop needs a different fix (a higher limit, a smaller task or a changed prompt), not the same request again.
• Keep two or three attempts. Each retry resends the conversation, so it costs input tokens again; the LLM cost calculator shows what a retry rate does to a monthly bill, and estimating LLM API costs explains the sums.
• Log failures. Repeated errors on the same field usually mean the schema or prompt is unclear. Describe the field better instead of retrying harder.
• Fall back deliberately. After the last attempt, fail loudly or queue the item for a person. Don’t silently store half-valid data.

When should you repair broken JSON?#

Repair when you can’t change the request: old models without structured outputs, logs you’re cleaning up, or replies pasted by a person. A repair library fixes syntax well, but it has to guess when the text is ambiguous, and some guesses change your data without an error:

So treat repair as a way to see the data, not a way to trust it. Validate the repaired value with your schema, record that it was repaired, and if the reply was truncated, retry instead of keeping the patched version. In Python, the json_repair package does the same job as jsonrepair; for a one-off, our JSON repair tool shows every change it makes.

How to parse streaming partial JSON#

Accumulate the chunks and parse the full text once the stream ends; parse partial text only to preview it. Gemini’s docs say streamed structured output arrives as partial JSON strings that concatenate into the final object, and Anthropic streams tool inputs as input_json_delta events whose partial_json fragments you join and parse when the block stops. OpenAI recommends using its SDK helpers to stream structured outputs.

To show progress before the end, use a parser that accepts incomplete input. Pydantic’s from_json has an allow_partial option, which Anthropic’s streaming docs point to:

from pydantic_core import from_json

chunk = '{"name": "Ada Lovelace", "skills": ["maths", "poe'
print(from_json(chunk, allow_partial=True))
print(from_json(chunk, allow_partial="trailing-strings"))

# {'name': 'Ada Lovelace', 'skills': ['maths']}
# {'name': 'Ada Lovelace', 'skills': ['maths', 'poe']}

By default it drops the unfinished string; "trailing-strings" keeps it. In JavaScript, running jsonrepair on the text so far gives a similar preview. Either way, render partial values but don’t act on them: run your schema check and the stop-reason check on the final text before saving anything or calling a tool.