🔍 Read the full analysis: The AI Model I’d Pick For My News Pipeline Isn’t The One In Use on ThorstenMeyerAI.com
Get the little things that make your day delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thorsten Meyer says a test of 71 model configurations for his news pipeline found GPT-6 Astra produced the highest-scoring articles, while GPT-6 Luna was selected for production at about one-hundredth of Astra’s cost. The test also found that higher reasoning settings did not reliably improve Luna’s scores and that two backup models produced fabricated quotes or unusable output. The results come from one publisher’s test and do not establish how the models perform across other newsrooms.
Thorsten Meyer says he selected GPT-6 Luna to write articles in his news pipeline after testing 71 model configurations, even though GPT-6 Astra received the highest writing scores. In results published October 13, Meyer reported that Astra scored 4.00 out of 5 in both English and German, while Luna scored lower but cost about $0.0012 per article, compared with roughly $0.12 for Astra.
The test ran from September 8 to October 13, 2026, using Meyer’s production prompt and real news sources. Models included paid API and subscription services, free-tier options and open-weight models run on his hardware. Each source was turned into an English and a German article, then reviewed by two AI judges: GPT-6.1 Sol and Claude Opus 5.5. The judges assessed source fidelity, fabrication, clarity about uncertainty, news value, structure and polish, with German language quality also scored.
Meyer used two initial sources: a 6,340-character product filing and a 619-character political news item. He said the shorter source better reflected much of his production work. The strongest configurations also completed a robustness run of 10 additional articles, mostly based on short sources. Meyer reports that the robustness scores included 95% bootstrap intervals; he treated small differences in the two-article initial test cautiously and considered the larger run more reliable.
In the robustness run, Astra scored 3.93 across 12 articles, while Luna scored 3.67. Astra’s reported interval was 3.83–4.02, compared with 3.59–3.76 for Luna. Luna’s source-fidelity score was 4.71 out of 5, and Meyer said all of its articles produced valid output after the repair step used in production. He reported a median production time of 17 seconds per Luna article, at about $0.0013 each; Astra took around 50 seconds in his test.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Cost Shapes the Production Choice
The results illustrate a practical trade-off for publishers using AI to draft large volumes of articles: a top score does not necessarily make a model the workable production choice. Meyer says Astra’s stronger writing performance did not justify a price about 100 times Luna’s for his operation. That is his cost decision, not evidence that Luna is the best choice for every publisher or that its quality meets every newsroom’s standards.
The test also points to a safety risk beyond the main model. Meyer reported that two low-cost backup models in his previous failover chain produced invented quotations or output that could not be used. In a news workflow, fabricated quotes can mislead readers and create serious editorial and reputational risks. A pipeline’s fallback behavior, validation and human review can matter as much as the model selected for routine use.
Meyer also found no dependable quality gain from spending more on Luna’s reasoning settings. If that result holds in his production workload, higher settings would add time and cost without a reliable score improvement. The finding is specific to his task, prompt, sources and evaluation method; it should not be read as a general conclusion about reasoning settings for other work.
As an affiliate, we earn on qualifying purchases.
A Test Built Around One Publisher
Meyer’s central question was not which model ranks highest on a general benchmark, but which one could produce usable articles from the sources and prompt his own sites use. The comparison therefore focused on a particular workflow: converting supplied material into English and German news copy, then checking whether the output remained valid after the pipeline’s repair step.
He reports that GPT-6 Luna was run at six reasoning-effort levels. Scores varied across the initial English and German tests, but he judged the differences to be within the noise. The highest setting, called max, cost about 1.43 cents per article and took 211 seconds, compared with 0.12 cents and 17 seconds at none. Meyer said the max setting was about 12 times as costly and slow, without a reliable quality gain.
The comparison also cautions against carrying a model’s reputation in one area into another. Meyer cited Claude Opus 5.5’s standing on a coding benchmark, but said it scored 3.52 in his news-writing robustness run, below Luna. That is a result from his article test, not a judgment on the model’s coding performance.
“For a high-volume task, the right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”
— Thorsten Meyer, describing his model-selection rule
Limits of the Reported Results
The results come from one publisher’s workflow, one production prompt and a limited collection of real sources. The initial comparison used two sources, with 10 more articles in the robustness run for the strongest configurations. The source material does not provide the full article set, judge-level scoring details or independent replication, so readers cannot assess every aspect of the evaluation from the published summary alone.
The scores were assigned by AI judges rather than a reported panel of human editors. Although both judges saw the original source, the available account does not establish how their assessments compare with newsroom judgments or how often they would agree with one another on the same article. Meyer also says the test used a repair step to determine whether an article was valid, but does not describe its full operation in the supplied material.
It is also unclear whether the cost and timing figures will remain stable as model prices, usage patterns and source lengths change. Meyer’s findings do not show whether Luna’s output is suitable for publication without human editing, nor do they establish how either model would perform on different beats, languages or editorial standards.
Luna Remains in the News Route
Meyer says GPT-6 Luna is now the production model in his news route, following his October 7 decision that Astra was too expensive for the task. He also reports removing GPT-4.1 nano and gpt-oss-120b from the fallback chain after their test failures. The supplied account does not say whether he plans to add replacement backups or change the pipeline’s human review process.
For now, the reported next step is continued use of Luna at the none reasoning setting, with its output subject to the pipeline’s existing validity repair step. Meyer has not provided a schedule for another benchmark or a broader evaluation. Any future comparison would need to test additional sources and monitor whether article fidelity, usable-output rates, costs and latency remain consistent in production.
Key Questions
Why did Meyer choose GPT-6 Luna instead of GPT-6 Astra?
Meyer said Astra scored higher, including 3.93 across 12 articles in the robustness run, but cost about $0.12 per article. Luna scored 3.67 in that run and cost roughly $0.0012 to $0.0013 per article in the reported tests and production use. He chose Luna as the lower-cost option for his high-volume workflow.
Did higher reasoning effort improve Luna’s articles?
Meyer reported that score differences across Luna’s six reasoning settings were within the test’s noise. The max setting took 211 seconds and cost about 1.43 cents per article, versus 17 seconds and 0.12 cents at none. He said the more expensive setting did not deliver a reliable quality gain in this test.
What happened with the former backup models?
Meyer said GPT-4.1 nano scored 1.62 in English and 1.68 in German, with source fidelity of 1.00, and that judges flagged invented quotes. He said gpt-oss-120b produced zero valid articles out of four in its test and generated named quotes that were not in the source. Both were removed from the route on October 7, according to his account.
Do these results prove Luna is the best model for news?
No. They describe Meyer’s test of a specific prompt, workflow, language pair and set of sources. The results do not establish how Luna performs for other publishers, topics, languages or editorial standards, and the reported scoring relied on two AI judges.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
