How to choose a model for coding agents: the criteria that matter

What matters when choosing a model for a coding agent?#

A coding agent is a loop: the model reads files, calls tools, edits code and runs tests until the task is done. The model you want is the cheapest one that finishes your tasks reliably inside that loop. Seven criteria decide it:

1. Tool-use reliability: well-formed calls, the right tool, no loops.
2. Context window: room for the repository slices, tool output and history a task accumulates.
3. Output limit: room to write a large file or a long patch in one reply.
4. Speed: how long a task takes end to end, which shapes how you work with the agent.
5. Price per task: what a finished task costs, including caching, not the price per token.
6. Reasoning controls: whether you can turn thinking up for hard steps and down for easy ones.
7. Where it runs: the provider’s API, a cloud you already use, or open weights you host yourself.

Rankings change often, so this guide doesn’t name a winner. It shows how to judge each criterion and gives live prices for the current tool-capable models.

Why agents cost more than chat, and how to price a task#

Every step of the loop is a new API request that resends everything so far: the system prompt, tool definitions, the task, every file read and every test log. Here is one small example task, a bug fix in eight steps:

The task bills 237,300 input tokens against 5,700 output tokens. Without caching, input is 89% of the cost on Claude Sonnet 5.5, even though its output price is 5× its input price. With caching, each step pays the cheap cached rate for everything it re-sent, and the bill for this task falls by 23% to 73% across the models below that list a cache price. A longer task has more steps, each resending more, so the gap grows.

In this example, with caching applied wherever a model lists a cache price, the cheapest model costs $0.00975 per task (gpt-oss-120b) and the most expensive $1.01 (GPT-6 Astra). A cheaper model only saves money if it finishes the task in a similar number of steps: a model that needs twice as many steps, or two attempts, can cost more per finished task.

Tool-use reliability#

An agent is only as good as its worst tool call. A malformed argument, the wrong tool, or a call that repeats forever wastes steps and money. Look for these, and test them on your own tools:

• Schema guarantees. Anthropic’s strict: true makes Claude’s tool calls always match your schema, and OpenAI’s strict mode makes function calls “reliably adhere to the function schema, instead of being best effort”. Turn them on where the provider supports them.
• Tool count. Every definition is billed as input on every step. OpenAI suggests aiming for fewer than 20 functions at the start of a turn, and both providers offer tool search to load definitions on demand.
• Parallel calls. Reading three files at once saves steps. Both APIs support parallel tool calls and let you switch them off when order matters.
• Asking instead of guessing. Anthropic notes that Claude Opus is much more likely than Sonnet to ask for a missing required parameter rather than infer one, and that this is less dependable on less capable models.

The practical test: run 10 to 20 real tasks from your backlog and count malformed calls, wrong-tool calls and steps per task, not just whether the task passed.

Context window and output limit#

Agents fill context quickly with file contents and tool output, so the window sets how long a task can run before the agent has to drop or summarise history. Of the 291 tool-capable text models in our data, 98 accept at least 1,000,000 tokens. Bigger isn’t free, though: each step pays for the whole context, and recall gets worse as context grows (see context windows explained).

The output limit matters when the agent writes whole files or long patches in one reply, and it includes any thinking. In our data, the limit per reply is 128,000 tokens on Claude Sonnet 5.5, 128,000 tokens on GPT-6.1 Sol and 65,536 tokens on Gemini 3.8 Flash. The context window checker shows whether a given file set fits with room for the reply.

For long tasks, compaction (summarising older steps on the server) and clearing old tool results keep the context small. Anthropic offers both for Claude, and OpenAI offers compaction in its Responses API.

Speed and reasoning modes#

Speed decides how you use an agent: a fast model suits interactive pairing, a slow, thorough one suits tasks you hand off and review later. Anthropic labels its current lineup from “slower” (Fable) to “fastest” (Haiku), and both Anthropic and OpenAI sell faster processing for some models at a higher price.

218 of the 291 tool-capable models in our data can reason (think) before they act, and that thinking is billed as output. All three major providers let you set how much: effort on Claude, reasoning.effort on OpenAI and thinking_level on Gemini 3. A sensible pattern is a high setting for planning and debugging, and a low one, or a smaller model, for mechanical steps like renaming or formatting.

Where you can run it: APIs, clouds and open weights#

If your company already buys cloud from one vendor, availability can decide the shortlist before quality does.

• Claude is sold through Anthropic’s API, Amazon Bedrock, Google Cloud, Microsoft Foundry and Claude Platform on AWS. Bedrock and Google Cloud set their own prices.
• OpenAI’s models are sold through OpenAI’s API, Microsoft Foundry on Azure and Amazon Bedrock.
• Gemini is sold through the Gemini API and Google Cloud’s Gemini Enterprise Agent Platform (formerly Vertex AI), which also hosts Claude and open models.
• Open-weight models can be downloaded and run on your own hardware or rented from many hosts. Our data lists 124 open-weight models with tool calling. Running a large one yourself needs a lot of GPU memory; the VRAM calculator estimates how much, and our guide to how much VRAM you need explains the maths.

What benchmarks like SWE-bench tell you#

SWE-bench is a widely used coding-agent benchmark. Each task is a real GitHub issue: the system gets the issue text and a copy of the repository, must change the code to fix it, and passes only if tests that failed before the original fix now pass. The original set has 2,294 tasks from 12 popular Python repositories.

• SWE-bench Verified is a 500-task subset whose problem statements, tests and solvability were checked by human annotators, created with OpenAI.
• SWE-bench Multilingual has 300 tasks from 42 repositories in 9 programming languages, useful if you don’t write Python.
• Bash Only runs every model in the same minimal agent (mini-SWE-agent), so the scores compare models rather than agent harnesses. The site also charts results against cost and step count.

Use benchmarks to build a shortlist, not to make the final call. A leaderboard entry is a whole system (model plus agent), most tasks are Python, and your repository, tools and conventions are different. Scores also change as new models arrive, which is why we don’t quote them here.

A simple way to choose#

1. Shortlist three models in the model comparison: filter for tool calling and the context you need, then add one cheaper model and one stronger one.
2. Run your own tasks. Take 10 to 20 closed issues from your backlog and run each through your agent on each model, with caching on.
3. Measure per task: pass rate, cost (from the API’s usage figures), wall-clock time and failed tool calls.
4. Pick the cheapest model that passes, and consider routing easy tasks to a smaller model.
5. Recheck every few months. New models and price cuts arrive often; the prices on this page update daily.