BREAKING: Secret US AI evaluation framework has been partly revealed
Very partly
TL;DR
- The U.S. has a secret AI security framework for evaluating which frontier models can be released.
- Protect Democracy filed a FOIA request and lawsuit to uncover this policy.
- The government released 132 pages of records, but most of the content is redacted.
- Key figures involved include Michael Kratsios and Ethan Klein.
- Protect Democracy intends to continue legal action to push for transparency in the AI review framework.
Remember, even as Trump shouts from the rooftops of Truth Social that he is violently opposed to regulating AI …
… he is in fact regulating AI, witha secret AI security frameworkfor evaluating which “frontier” models can be released. Almost nobody knows what’s actually in it.
§
A lot of people are rightly upset about it.
In that spirit, I told you last week aboutProtect Democracy’s FOIA requestand their lawsuit to find out what that policy is.
To my astonishment, the government responded promptly! With132 pages of records! What’s moreProtect Democracykindly shared them with me.
For your convenience and edification, here is the first page, which very much sets the tone for the rest:
Aside from revealing, unsurprisingly, that Michael Kratsios (Director of OSTP) and Ethan Klein (U.S. Chief Technology Officer) are in on the conversation, almost nothing is revealed.
A screenshot of the first 10 pages:
and the next 10:
If you enjoy grayed-out redactions, you can read the whole thinghere. But since nearly all of the good stuff is redacted, you won’t find out much.
Or, wait… maybe mostly blank pagesarethe policy?
Either way, U.S. policy on screening AI remains opaque for another day.
Conspiracy theorists might start to wonder what the government is hiding.
P.S. The good news is thatProtect Democracyplans to keep fighting for transparency. In an email I just received from the lead counsel on the case, Deana El-Mallawany told me “Across the spectrum of views on AI risk and how to mitigate it, there's one clearly wrong way: in secrecy, with the terms, criteria, and justifications for regulatory action known only to a few of the most powerful political actors and economic actors in the world. That scenario is fertile ground for corruption and autocratic abuse -- as well as mistakes that could be avoided with greater transparency. That's why we will continue to push the administration to produce its secret review framework via this litigation and expect to reach agreement on a production schedule very soon.”