First, abstracting from all the wordsin my long analysis on OpenAI Astra yesterday, the overall point was that Astra may not be the breakthrough that OpenAI wants you to think it was. I expressed concerns about how little they revealed about what they did. Moreover, I thought I didn’t stress it there, we all know they are facing huge economic and competitive pressures; they need a win. I am often left with the feeling that their public facing communications are better marketing than they are science.

My jaw dropped a couple hours after my own post, reading this fromLevent Alpöge, an accomplished mathematician who works at Anthropic:

One(obviously smart) guy at Anthropic was able to “replicate” (as best he could given OpenAI’s very sparse report of what they actually did) half of OpenAI’s results within 24 hours. And not with Astra or some other new model, but with the already publicly-released Fable.

I had a couple reactions to this.

a. Alpöge’s fast partial replication made me wonder whether OpenAI’s real advance here was finding (through AI)which open problems were amenable to a certain kind of search and verify technique. OpenAI’s Noam Brown acknowledged, without much detail, that there were failures on some (and maybe many) others:

Unfortunately, beyond that OpenAI does not report which problems they tried and failed at. They have given us a numerator without a denominator, always a worrisome sign.

This doesn’t take away from OpenAI’s accomplishment - if the system solved a bunch of open problems that’s very impressive – but it again raises questions about what the scope is: all math? A certain subset of problems that can be searched in a certain way?

Maybe once that subset has been flagged, maybe many systems can do them. That’s kind of what Levent Alpöge‘s partial replication shows.

And many what was most critical was not some innovation in Astra per se but the idea (probably from a human) to do this particular search.

b. Alpöge‘s fast partial replication undermines the widespread presumption that Astra is a major breakthrough. Astra is an obviously an advance to some degree, but that degree may well turn out to be merely incremental relative to other recent models.

Related to this, I was interested to read inThe Informationthis morning that OpenAI hasn’t decided whether to call Astra (which was an internal code name) GPT 6 or GPT 5.7. If OpenAI truly thought Astra was a quantum leap,, I doubt that would be such a hard choice.

How good Astra is in open-ended real-world problems very much remains to be seen.

Second, a longtime reader of this newletter, Ihor Gowda, pointed me toa fantastic lecture given by Terence Tao, by any measure one of the world’s leading mathematicians; many see Tao astheleading mathematician. His lecture is lucid, accessible, and genuinely important.

What’s more you don’t need to be a professional mathematician to read it. It should be mandatory reading for anyone thinking about how AI will affect mathematics, and strongly recommended for anyone who cares about AI’s impact on the world.

One of the issues that he raises he calls “proof indigestion”; what happens if AI creates a lot of stuff, much of it true, but not necessarilyuseful:

Tao’s July 26th lecture, written before the Astra announcement, also prefigures another key point that I made yesterday:being good at creating certain sorts of proofs doesn’t mean that an AI system can contribute to all (or even most) of the goals of mathematics. He too distinguishes between solving open problems (which Astra seems strong at, perhaps with some important limits) and building theory (no evidence thus far that Astra can do that):

Tao is no Luddite, or even anti-AI. He is very open to AI, and eager to find a place for it in mathematics.His lectureis a beautiful read on how to do just that — to bring AI into mathematics, as thoughtfully as possible. Highly recommended.

Am working hard to keep you up to date in the most honest way possible and hope you will consider supporting this newsletter.

P.S. Cartoon of the day, illustratingBrandolini’s law:

Noawadays, as the engineer Wouter Vreugdenhil, who brought the cartoon to my attention, points out, “that’s the game.”

@GaryMarcusOpenAI sustain investor faith with minimal effort. Proving them wrong takes years and expertise most people lack. That's the game.路♂️ \",\"username\":\"WouterW_Vreug\",\"name\":\"Wouter Vreugdenhil\",\"profile_image_url\":\"https://pbs.substack.com/profile_images/1696517475741880320/-mw0nSoT_normal.jpg\",\"date\":\"2026-08-03T06:46:22.000Z\",\"photos\":[{\"img_url\":\"https://pbs.substack.com/media/HOx0pY5WQAA4uii.jpg\",\"link_url\":\"https://t.co/Jv7olgUFSo\"}],\"quoted_tweet\":{},\"reply_count\":0,\"retweet_count\":1,\"like_count\":8,\"impression_count\":3548,\"expanded_url\":null,\"video_url\":null,\"video_preview_media_key\":null,\"belowTheFold\":true}\" data-component-name=\"Twitter2ToDOM\">",{"_244":190,"_245":273,"_246":274},"581d1360-b9fc-4a8b-bf02-28936dd366d0",{"_19":273,"_31":275,"_192":276,"_33":277,"_215":278,"_251":278,"_21":222,"_252":279,"_254":280,"_208":40,"_68":-5,"_70":40,"_170":281,"_244":190,"_270":293},"OpenAI’s amazing — but vastly oversold — new model Astra","OpenAI's new model, Astra, has demonstrated impressive capabilities by solving 10 major open problems in mathematics and theoretical computer science. However, experts caution against the \"fallacy of composition,\" where success in a specific domain like math is extrapolated to imply general intelligence or universal problem-solving ability. The article emphasizes that math's suitability for verification and synthetic data generation makes it an easier domain for AI, unlike more complex, less formalizable real-world problems, drawing parallels to past AI successes that did not generalize broadly.","Eight or nine misconceptions about Astra. See if you can spot the biggest fallacy.","https://substackcdn.com/image/fetch/$s_!FupX!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fpbs.substack.com%2Fmedia%2FHOn2XyYasAA_rAp.jpg","2026-08-02T21:25:27","https://garymarcus.substack.com/p/openais-amazing-but-vastly-oversold",{"_19":282,"_173":283,"_175":115,"_68":284,"_201":26,"_202":291,"_204":292},"dcd197967ff13eb1a170790f19a82f6793f925533910029d76cd854ae2e3db0d","d2d376b2c07be87e857663f4d021a2bd74652d7a57a5d75680bb33b367597b4d5ece718a315a6f0453c6dbc1eb4e3165900722c844da8cf74989b2d6c2d6efef",[285,286,287,289,290],[208,275],[35,278],[263,288,40,265],"30023:983dd797fe72ad62fc8e3ece7eed1f126550972885c8f835d9af5528dad655a3:581d1360-b9fc-4a8b-bf02-28936dd366d0",[198,40],[200,26,40],"OpenAI’s amazing — but vastly oversold — new model Astra\nEight or nine misconceptions about Astra. See if you can spot the biggest fallacy.\n\n\nnostr:naddr1qvzqqqr4gupzpxpa67tluu4dvt7gu0kw0mk37yn92ztj3pwglq6ant649rddv4drqqjr2wp3vscnxd3s943rjenr956xzwrz943xvvpj95erswfnxejxgvekxejrqqdhfvx ",1785705927,"

Astra, a new model that OpenAI is testing internally, is amazing. No denying that:

@OpenAI’s next major model family, solved 10 major open problems in mathematics, quantum complexity, and theoretical computer science.\\n\\nWe believe it will be a major step for scientific reasoning.openai.com/index/ten-adva…\",\"username\":\"polynoamial\",\"name\":\"Noam Brown\",\"profile_image_url\":\"https://pbs.substack.com/profile_images/1872883170292457472/8ywVGO5M_normal.jpg\",\"date\":\"2026-08-01T08:17:42.000Z\",\"photos\":[{\"img_url\":\"https://pbs.substack.com/media/HOn2XyYasAA_rAp.jpg\",\"link_url\":\"https://t.co/jHuulDwV46\"}],\"quoted_tweet\":{\"full_text\":\"10 proofs from our next major model Astra on long-standing open problems in mathematics and theoretical computer science (also including new circuit lower bounds for computing the permanent!)\\n\\nGPT-5.6 has already enabled so much exciting work in math and science. Can’t wait to\",\"username\":\"wjmzbmr1\",\"name\":\"Lijie Chen\",\"profile_image_url\":\"https://pbs.substack.com/profile_images/1810196750009094144/nq1dEwmM_normal.jpg\"},\"reply_count\":664,\"retweet_count\":2107,\"like_count\":14941,\"impression_count\":8395185,\"expanded_url\":null,\"video_url\":null,\"video_preview_media_key\":null,\"belowTheFold\":false}\" data-component-name=\"Twitter2ToDOM\">

Part I: The Biggest Fallacy

But at the same time, a whole raft of people, some fairly prominent, are running around making a deeply flawed argument about the implications.

See if you can spot the fallacy. Here are three examples among many.

Another tweet went so far as to claim that “the species just crossed a one-way threshold”; Elon Musk took itas evidence that we had reached The Singularity.

Each is making essentially the same error.

The fallacy has a name; it’s called thefallacy of composition.

Thewiki on itis filled with great examples; here are just the first few.

What’s the manifestation of the fallacy in the current case? Thinking that a system that is great at a certain kind of math problem is great at all math, great at science or even quite possibly great at everything. The “AGI-is-near” community keeps committing the same logical fallacy over and over. Every time there’s an advance, I see the same error.

Here’s how the fallacy works.

1. Someone pretends that all cognition is created equally. (Totally untrue.)

2. Whenever AI achieves success on some form of (fancy) cognition, they want you to believe that success on all forms of AI is imminent.

You don’t have to be a cognitive psychologist to realize that this inference just doesn’t follow. We all know, for example, that expertise in math doesn’t guarantee genius in all domains. Someone who is great at math or physics or programming may struggle with writing or understanding human relationships (and conversely a great writer may be weak at math, etc).Expertise in one domain does not at all guarantee expertise in all or even most domains.

That’s precisely *why* people like Howard Gardner and Robert Sternberg developed multidimensional theories of intelligence, why the SAT tests math separately from verbal, etc.

Astra appears to be —we still haven’t seen the methodology—great at math, or at least some forms of math, but that does not mean that it will avoid hallucinations or solve the reliability problems other GenAI systems have. It doesn’t even mean it will be able to read PDFs reliably. And it doesn’t mean it will be the first generative AI to be able to obey hard rules, either. (Which should terrify you.)

In particular, Astra is obviously excellent at some problems; but that doesn’t mean it will be excellent or even competent at problems thatare hard to formalize. It doesn’t mean it will be magic. It doesn’t mean it’s AGI or ASI or any of that.

It’s *very* impressive. But I see no reason whatsoever to think Astra is AGI let alone ASI. If it can score even a 5/10 onmy 2024 bet with Miles BrundageI will be surprised.

Part II. There is an important, principled reason to think that success on math is a special case which will not generalize as much people might hope.

I wouldn’t go on about this fallacy at such length if (a) it wasn’t wildly common and (b) I didn’t think that there was very good reason to think that the fallacy appliedhere.

It’s not an accident that math is where these models are shining. Math lends itself to two things: verification (using symbolic tools), and massive amounts of cheaply produced synthetic data where you can guarantee that the answers are correct. The same applies to coding — but it isnottrue in general. You can generate as many math facts as you want; you can’t simulate the open-ended world. You can verify math; you can’t verify a military strategy in the same way.

This is not news. I have been pointing this out at least since January 2025 [first line should have said codingandmath], echoing something Ernie Davis and I said about Go (and why AlphaGo would not be a panacea) in 2019:

OpenAI can do what Astra does in math becausemath allows for external tools to do verification and to create synthetic data.

That doesn’t make Astra less impressive, but as we learned a decade ago from IBM’s shambolic and ultimately failed attempt to turn Jeopardy-winning Watson into a cancer-fighting machine,success in one domain does not guarantee success in all.

The kind of math OpenAI is dealing with here is radically different from many (perhaps most) real-world problems in which verification neither guarantees solutions nor allows one to produce infinite training data effectively for free.

To expect it to be a universal solvent is to show you don’t understand that basic fact.

Part III: Seven more things to know about Astra

Yesterday's tweet and blog were marketing, not science. Neither of those nor the 249-page math article that went with them give any information about how this was accomplished:
In an email commenting on the first draft of this essay Ernie Davis added, quite rightly:
To properly evaluate the significance of these discoveries we would need some
critical pieces of information.

First: How many conjectures were attempted? If the OpenAI team picked these 10 conjectures at random from the space of all outstanding significant mathematical conjectures and Astra solved all 10, that would be amazing; but that seems altogether unlikely. If they cherry-picked 50 conjectures that they thought Astra would succeed on and it succeeded on 10, that’s still amazing, but significantly less so. If they ran Astra on all 1000 or so open Erdos conjectures and on 10,000 other open conjectures, then its failure to solve those is significant information on its limits as a mathematical reasoner.

Second: OpenAI brags loudly that this was done for a total computation cost of $2000. It seems a safe bet that this includes only the conjectures where Astra succeeded, not the ones where it failed. But what was the cost in terms of the salaries of the highly-paid mathematicians and computer scientists who worked on this project? I’d be astonished if it was less than $20,000 and would not be surprised if it was upward of $200,000.
We don’t know this, and the way things are going, we may never know this. We still have zero information about the OpenAI system that achieved gold-medal performance at the 2025 International Mathematical Olympiad, and very little information about the Google system.
Astra is good at some math but it isvery doubtful that math is “solved”(as a lot of people on X seemed to believe). It seems to be good atcertain kindsof math
$\"\"$
But quite possibly not all. Here are some examples fromEric Weinstein on Xabout some kinds of math it might be less good at
$\"\"$
Ernie Davis also adds “The problem of autoformalization --- turning mathematics written by a human mathematician into a strictly logical form -- does not seem close to being solved, though certainly progress has been made. For example: For the last two years, mathematician Kevin Buzzard has been leading aprojectto formalize Andrew Wiles' proof of Fermat's Last Theorem in Lean. It is quite safe to say that no AI system is capable of carrying out that monumental task. Presumably if they could do it unassisted, they would scoop him, and if they could cut down his work from years to weeks, he would be using them.”
Consistent with my general prediction that facility in math doesn’t guarantee universality,it appears thatAstra’s proofwriting is not on par with the proofs themselves(echoing something Ernie Davis and I observed here a year ago)” [click through for full tweet]
I seriously doubt that Astra will magically solve all the things that don’t work reliably in current models.Will it be able to reliably extract numbers from arbitrary PDFs? Doubt it. Will it solve Sabine Hossenfelder’s desire to have AI write scripts [click the tweet for more details] for her YouTube series? Doubt it.
Astra is not (as Musk suggested) the Singularity. Enough said. (But see my recent essay on the topic; nothing has really changed).
It is absurd to claim that the Astra debut is “plausibly the most significant day in the history of mathematics”as a bunch of people went around saying, based on a quote from one run of Fable (which might give different answers if asked twice). There was no new theory, no new techniques, and as impressive as it is, it is not on remotely on par with the development of calculus, algebra, logarithms, probability, the decimal system, information theory, the concept of zero. Says Davis “The claim quoted from Fable that \"several of those would individually have been the result of the decade\" is absurd … As a reality check, 14 of David Hilbert's 23 problems have been solved since 1900 --- that's one every nine years --- and these results are nowhere near that league. You could easily compile a list of 100 much more important results that have been proved since 1926.”
I doubt it will lead directly toa cure for cancer, or enormous advances in“materials research, energy production, drug discovery, Everything”, as some excitable AI “influencers” envision.Nor does this mean we just entered “the era of automated scientific discovery”. Against overwrought claims like these…

@OpenAI's next model family, Astra! We are now entering the era of automated scientific discovery!\\n\\nEssentially, mathematics and theoretical computer science have entered the post-human-bottleneck era of\",\"username\":\"DeryaTR_\",\"name\":\"Derya Unutmaz, MD\",\"profile_image_url\":\"https://pbs.substack.com/profile_images/2040869160432107520/t6sD3Nbl_normal.jpg\",\"date\":\"2026-08-01T12:47:27.000Z\",\"photos\":[],\"quoted_tweet\":{\"full_text\":\"An internal version of Astra, @OpenAI’s next major model family, solved 10 major open problems in mathematics, quantum complexity, and theoretical computer science.\\n\\nWe believe it will be a major step for scientific reasoning. https://t.co/iP6cyheZ7i https://t.co/jHuulDwV46\",\"username\":\"polynoamial\",\"name\":\"Noam Brown\",\"profile_image_url\":\"https://pbs.substack.com/profile_images/1872883170292457472/8ywVGO5M_normal.jpg\"},\"reply_count\":29,\"retweet_count\":45,\"like_count\":463,\"impression_count\":25678,\"expanded_url\":null,\"video_url\":null,\"video_preview_media_key\":null,\"belowTheFold\":true}\" data-component-name=\"Twitter2ToDOM\">

.. this just came out, echoing the new article byand others that I linked to a few days ago.

Part IV: Summary

Despite immense pushback on X yesterday (most of it ad hominem, some literally involving fabrication, a lot of it involving outright lies), my overall hot take on Astra remains what I first posted:

What’s sad about this is that Ernie Davis and I raised a lot of the same points about how onecouldproperly examine new systems a year ago:

I get that people are excited about Astra, but anyone hoping for magic is likely to be disappointed.

Subscribe now

",[],"selectedArticleImage","rootArticleImageUrl","consensusHtml","tweets",[],"userSchema","relatedArticles",[247,274],"actionData","errors"]

OpenAI's Astra Model's Mathematics Breakthrough Faces Scrutiny

Story coverage