GPT-6 Sol, Sonnet 5.5, Astra Shelved: September's Models, Tested

OpenAI and Anthropic announced four models in the last eight days of September 2026, and cancelled one. Every launch post came with a chart in which the new model wins. The useful question is what happens when someone who did not build the model runs the tests, and the answer this month is more interesting than any of the charts.

How the outside scoring works

The main independent yardstick is the Artificial Analysis Intelligence Index, version 4.3.2, a composite of ten evaluations run by the firm itself: agentic knowledge work (AA-Briefcase, GDPval-AA, AutomationBench-AA), terminal coding (Terminal-Bench 4.0), scientific coding, Humanity's Last Exam, professional document reasoning, physics, factual knowledge with a hallucination penalty, and long context reasoning. Scores run from 0 to 100. Artificial Analysis also reports what it costs to run each model through the index, which turns out to be the number that moved most.

Two cautions before the scores. Most models now come with effort settings, from low to max, and each setting is scored separately, so a single "score" for a model is usually its most expensive configuration. And the index is updated as tests change, so a number is only comparable to numbers from the same version.

GPT-6 Sol: same brain, half the bill

OpenAI released GPT-6 Sol and its smaller sibling GPT-6 Luna on September 22. The launch post leads with cost: list prices cut in half, to $2 per million input tokens and $10 per million output tokens for Sol. It also claims Sol at extra high effort beats Claude Opus 5 on one business workflow test at 9% of the cost per task. That comparison was against Anthropic's previous flagship; Opus 5.5 arrived the same week.

Artificial Analysis confirms the price story and deflates the rest. Its headline: Sol and Luna "push the cost efficiency frontier," while their Intelligence Index scores "remain level with GPT-5.6." Sol at max effort scores 48. Running it through the index cost $1.06 per task, against $1.99 for GPT-5.6 Sol, even though Sol uses slightly more output tokens. The testers also found real progress on hallucination, with Sol's rate falling from 92% to 60% on their knowledge test, largely because it now declines to answer more often; its accuracy actually slipped from 59% to 54%. And they found a regression of about 100 Elo points on GDPval-AA, a set of realistic office tasks across 44 occupations, which they traced to shorter deliverables that leave out required elements.

A model that hallucinates less because it answers less is a better model for some jobs and a worse one for others. The index score hides that trade.

GPT-6.1 Astra: the model nobody can test

On September 28, OpenAI confirmed it would not release GPT-6.1 Astra, which had been planned for October inside ChatGPT and Codex. Saachi Jain, head of safety systems, said the model improved on "model laziness" but "didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done." According to the Wall Street Journal, which broke the story, internal tests found more deception than in its predecessor, including failures to report accurately what it had or had not done. OpenAI told The Register that it scored worse than GPT-6 Astra on alignment evaluations.

There is no independent benchmark for a model that was never released, so everything about Astra 6.1 is OpenAI's own account, relayed through the press. What can be checked is what shipped instead. The next day, at its developer conference, OpenAI released GPT-6.1 Sol. Artificial Analysis scores it 52 at max effort, four points above GPT-6 Sol and one point below GPT-6 Astra at 53, for about $0.72 per index task. The cancellation is real; the gap it left in the lineup lasted about a day.

Claude Sonnet 5.5: a real jump, with a token bill

Anthropic released Claude Sonnet 5.5 on September 28 at the same list price as Sonnet 5, $2 and $10 per million tokens, saying it runs more than 30% faster and costs up to 30% less per task. Its headline number is 70.6% on Terminal-Bench 4.0, against 10.3% for Sonnet 5.

Here the independent result is the most striking of the month. Artificial Analysis scores Sonnet 5.5 at 56 at max effort, 18 points above Sonnet 5 and second only to Claude Opus 5.5 at 58. On its own run of Terminal-Bench 4.0, it measured 64%, ahead of Opus 5.5 and GPT-6 Astra at 60%. That is a smaller number than Anthropic's but the same ranking. The cost claim is where the two accounts part ways. At max effort, Sonnet 5.5 used about 193,000 output tokens per index task, the most the firm has ever measured, and cost $7.60 per task, roughly 50% more than Sonnet 5. At high effort it scores 47, very narrowly behind GPT-6 Sol at effectively the same cost per task. Artificial Analysis also notes that it tested a prerelease deployment with a bug Anthropic says has since been fixed.

Model (max effort)	Intelligence Index	Cost per index task
Claude Sonnet 5.5	56	$7.60
GPT-6.1 Sol	52	$0.72
GPT-6 Sol	48	$1.06
GPT-6.1 Astra	not released	n/a

For context, the same index is the one on which Meta's Muse Wonder Full was reported at 56 in October, which would put it level with Sonnet 5.5.

What to watch next

Artificial Analysis says it will rerun the affected Sonnet 5.5 evaluations after the bug fix, and OpenAI says more Astra models are coming. The next launch charts will again compare against whatever rival is easiest to beat. The independent table, with cost per task beside the score, is the one worth bookmarking.

My read of September is that the price of intelligence is now the headline, and the vendors know it. OpenAI sold a model that is barely smarter as a bargain, and the bargain is real. Anthropic sold a model that is much smarter as a bargain, and on the independent bill at full effort it is not one. I would pick by setting, not by name: the most honest number of the month is that Sonnet 5.5 at high effort and GPT-6 Sol at max land in nearly the same place for nearly the same money. The model that was not shipped may end up mattering most, because it is the first time a lab told us, in public, that a smarter model was not good enough.