I was asked to give my opinion on a new tech policy letter published by twenty-five organizations, titled Open Weights and American AI Leadership.
Nvidia published the PDF, and Microsoft mirrored it to its corporate responsibility site. Jensen Huang very strangely opened a social media account based on a Nazi Swastika to promote it.
The letter asks Washington not to restrict downloadable AI models. It declares openness a foundation of AI safety:
[Open models] allow a broad community of researchers and developers to examine their behavior, identify vulnerabilities, develop safeguards, and improve them over time.
Eight days earlier, the prominent open sourcing announcement by the infamously Hitler-saluting Elon Musk demonstrated that he’s playing games with language as usual.
Like a (supervised) full self driverless car that isn’t driverless, guess whose “open source” propaganda is the opposite act? Do you ever get the impression that he just lies and doesn’t care that millions of people will die because of him (14 million projected by 2030)?
Herr Elon, supervised open source isn’t open source
On July 12, a researcher publishing as cereblab released a wire-level analysis of Grok Build, the terminal coding agent from xAI. The method used was ordinary interception: version 0.2.93 of the client routed through mitmproxy, with the full captures published.
The findings were like seeing a Tesla on fire.
Grok Build was secretly packaging users’ entire tracked Git repositories, full commit history included, and uploading them as git bundles to a Google Cloud Storage bucket named grok-code-session-traces, where it was controlled by Musk’s xAI.
The numbers revealed that it was a codebase issue. On a 12 GB test repository of files the model never read, the model channel moved about 192 KB of task-relevant traffic while the storage channel moved 5.10 GiB, a ratio of roughly 27,800 to one. All 82 storage calls returned HTTP 200.
A canary credential planted in a .env file appeared verbatim and unredacted in the captured traffic. Let me say that again. The developer secrets were leaked by design, cleartext.
The researcher then cloned the captured bundle and recovered a file the agent had been explicitly instructed never to open. Users reported SSH keys, password databases, documents, and photographs leaving their machines.
The tool’s “Improve the model” toggle, the one control a developer would read as consent, had no effect on the upload. It was a dummy dashboard, just like the lies of a Tesla dashboard claiming 300 miles while the car tops out at 150 miles. The switch turned out to govern only training permission. The code was taken regardless of the setting. Tesla’s range deception ended the same way, with an August 2023 class action:
Tesla is facing a class-action lawsuit filed by customers who say they were misled by the company’s exaggerated range claims. The lawsuit was filed yesterday, days after a report revealed that Tesla exaggerated its electric vehicles’ range so much that many drivers thought their cars were broken.
The response to the Grok design failures was five steps. Each step is usually a simple and standard practice, which has verification. Instead, each was made incorrectly by xAI and to prevent verification.
The first fix was silent. A day after publication, the researcher retested the identical client and found a new server-side flag, disable_codebase_upload: true, now arriving with each session. The upload stopped. The flag was flipped remotely, announced nowhere, and verified on exactly one machine and one account. Whether it is global, staged, or permanent is unknown, because the mechanism that ended the collection is invisible by design. That’s the definition of closed.
Elon Musk then promised deletion of all data Grok Build had ever stored. Lol. This guy. He will say anything that he thinks people want to hear. Remember October 2016 when he said driverless would be completed for cross-country trips without touching the steering wheel by the end of 2017! Sure, sure Elon.
Deletion claims in cloud infrastructure are attestable. A named forensic firm, a published scope, deletion certificates covering replicas, backups, access logs, and derived artifacts including training data. Companies produce these documents routinely, because attestation is what anchors words to reality.
xAI produced… a post on X. In other words, nothing. A massive breach of confidentiality. The kind of design failure that should bring massive fines for negligence, ended with what?
The company disclosed no scope. The number of affected users, the duration of the collection, the volume received, whether the bundles were accessed or processed after arrival: every one of these figures sits in xAI’s logs. All were withheld. The wire captures establish what left users’ machines. And xAI has never admitted what that included.
On July 16, xAI published the Grok Build source code on GitHub under Apache 2.0, in Musk’s words to build trust in the product. With all the money in the world, the huge engineering teams and the American government eating from his hands, he produced … a repository with a single commit.
Zero pull requests. Zero history.
The one forensic question that matters is what the shipped binary did between launch and disclosure. He provided a repository without history. That’s basically a closed repository. It cannot answer any questions. The code as published shows what xAI wishes to be seen after the incident.
Whether the upload path was removed, renamed, or relocated to the server side is unknowable from the “proof” that appears as clean laundry. Git history is a chain of custody. This release destroyed the chain and presented the result as transparency. There is a bitter symmetry here. The company that harvested its users’ full git histories shipped its own repository without any history.
Get it?
Users’ private repositories, including code they never published anywhere, were pulled into xAI’s closed infrastructure without consent. Meanwhile xAI’s disclosure covered only the client. The server side that received the bundles, the storage bucket, the retention configuration, the access controls, the processing pipeline: all of it stays hidden, holding other people’s intellectual property, let alone all their secrets.
A failed apology isn’t the right word. This was a strategic play, a game to undermine trust while claiming to be providing it. A silent flag, deletion without attestation, disclosure without scope, code without history, openness without the server. There were five independent design decisions all delivered without anything that could be verified or held accountable. That’s closed behavior, unaccountability defined. At that point the failures stop being some trait or feature, and become the entire product itself.
Ok, ok, let’s talk about The Letter
The coalition letter treats “open” as a binary state. Flip the open switch, confer audit value automatically? That argument for safety depends on a single thread: openness enables verification, transparency anchors claims being made. The letter tells us:
Just as open-source software demonstrated that transparency can be more secure than obscurity, AI safety may depend on giving more people the ability to test and strengthen the models on which society relies.
It then goes on to name the outputs we should expect from openness: benchmarking, red teaming, vulnerability discovery. However, the letter forgets to mention that all of it depends on history, provenance, training data, reproducible evaluation, server-side scope. Those are the inputs, which Elon Musk proved he could completely destroy while claiming to be the “open” guy.
The reason I explained the Grok Build breach is because it proves the letter doesn’t work in the bed that Elon Musk is making for America. The release is open by every criterion the letter uses. Apache 2.0. Downloadable. Inspectable. Modifiable. The broad community of researchers can examine it. What the community cannot do is answer any question that matters, because every input to verification was stripped before publication.
If “open” is so easily gamed by people who want to humiliate the security property, then Grok Build is that proof. Grok Build demonstrably lacks critical security and can’t be trusted, because it can’t be verified. It’s the Tesla of coding. Therefore “open,” as the letter uses the word, is a label rather than a property, just like “driverless” isn’t. The letter’s central safety claim fails on a live case that predates its own publication by twelve days.
You can not tell me the signatories don’t know the difference, or don’t know exactly what Musk is doing by destroying the meaning of words. The rather uncomfortable timing is Musk suddenly claimed he was going to be “open” and then NVidia jumped on his Nazi Swastika platform to promote it with their letter promoting being “open”.
More importantly, Nvidia’s Nemotron 3 Ultra shipped in June with weights, post-trained checkpoints, training datasets including 173 billion tokens of refreshed code, and full training recipes under the Linux Foundation’s OpenMDW-1.1 license. That is disclosure of a normal company. Independent evaluators could score it, and did. Nvidia’s release practice draws exactly the line between openness as an auditable property and openness as a clown costume to undermine the meaning of openness.
The letter Nvidia hosts is not making the distinction that needs to be made. One sentence would have done it. One paragraph could have defined the qualification, published history, disclosed data, reproducible evaluation, and therefore expelled counterfeits like xAI.
The silence is damning. Twenty-five organizations declared openness a security foundation while xAI was loudly counterfeiting it in public, with users’ credentials sitting in a cloud bucket and the repository “open” without any history. Nobody said a word about the breach of “open”, undermining trust in it. Hugging Face, a signatory now embroiled in a huge trust breach of its own, hosts the Grok weights Musk releases on his depreciation schedule, each model opened only after its successor ships. The coalition’s standard is that anything, anything counts toward the “open” cause, including a release engineered to contradict the cause.
The theft
The letter contains one paragraph that performs concrete legal work, and it concerns something other than open weights. On July 21, Treasury Secretary Scott Bessent announced on Fox Business that the government would examine Chinese open models for intellectual property theft and could sanction the companies behind them, citing watermarks from American models found in Chinese systems.
There’s a very technical AI word for it called distillation, but you and I would call it theft
Three days later, twenty-five organizations signed a document declaring distillation “a widely used technique for model improvement, evaluation, and validation” and asking that unlawful extraction be handled through “targeted legal and commercial frameworks rather than sweeping restrictions.”
Targeted legal and commercial frameworks means contract disputes and civil damages. The clause converts an announced national security enforcement theory into private litigation between labs. Every open-model developer on the signature list trains on frontier outputs.
The letter therefore has a paragraph deployed as a retroactive legal defense drafted as policy principle. The timing needs to be called out. Axios reported on July 20 that the administration was reviving efforts to restrict Chinese models, that Commerce had previously weighed Entity List designations for Chinese labs, and that closed-lab allies had approached the White House periodically with ban proposals. Then a three-page letter that went through the legal department grinder of twenty-five organizations, with coordinated hosting and a choreographed launch, was in preparation well before the story ran. The leak and the letter reveal a political (profit) battle raging inside the administration, between pro-restriction and pro-access.
The letter’s risk handling then admits that released weights are “beyond the original developer’s control, and modified versions are difficult to trace or reverse.” And it goes on to demand protections “tied to real and demonstrated harms rather than assuming that closed systems are safer by default.” A demonstrated-harm standard applied to a release that can’t be reversed is a lie. It’s like saying don’t let the horses out of the barn after they are gone.
I have no real idea why both of these paragraphs, in full contradiction, survived twenty-five legal department reviews. My guess is because they used AI instead of humans. Put simply, the demonstrated harm is what would kill pre-release vetting, yet the administration is considering pre-release vetting.
The history lacks historians
The letter boils down to a request for the usual stuff. More compute access for startups and researchers. More subsidy, more, moar! Taxpayer money is supposed to go towards shared datasets and evaluation frameworks. That’s another subsidy. And then a request to delay restrictions: basic deregulation. And finally, legal immunity.
Three transfers and an immunity, wrapped in a safety argument. And the elephant in the room is that Musk’s bad-actor behavior refutes all of it.
The letter opens by claiming the lineage of “1980s open-source software pioneers.” The gall. The term open source dates to 1998. The 1980s movement was something different, called free software, launched by Richard Stallman’s GNU announcement of 1983, an ideological project the signatories’ predecessors fought for two decades.
I mean WTAF. IBM? Palantir? Microsoft? You’re telling me these are claiming to hold the open source baton? Microsoft’s own January 2025 manifesto under Brad Smith argued the reverse position, American leadership framed against Chinese competition with export controls attached, and carried the identical asks: public money, public protection, private control.
Their position reversed within eighteen months, while their extraction outcome sounds the same. When the argument flips completely while the bill stays the same, the person billing has a trust problem.
The fix for all this anti-historical mucking about and political buffoonery is definitional and it is short. Openness deserving legal protection has a checklist: published development history, disclosed training data, reproducible evaluation, declared server-side data flows, and attested claims about deletion and retention.
Nvidia ships this when it wants to. It’s not anything unusual. The letter’s signatories can meet it or stop being a wolf in bed putting on grandma’s bonnet. A release with no history, no scope, no server, and no auditor is a failure by design, and Washington is currently being asked to let it continue operating like a Tesla.
The deeper resolution is that irreversibility is the ethics landing.
Publication is a one-way gate, a fact that carries no safety valence on its own. Every party in this fight is working to move scrutiny somewhere else, instead of admitting this tension in the open.
Miessler’s mousetrap article, which I wrote about before here, tried to invent a recall lever that will never work on an open weight model release. This NVidia letter’s demonstrated-harm standard needs a remediation phase like a rocket already fired into space needs a launch pad on earth to make repairs. Too late. Musk burns all the evidence before his technology is delivered and calls the ashes of his “rapid unscheduled disassembly” and exploding Tesla his version of being open. All three don’t seem to understand they can’t just relocate the demand for proof of something. Perhaps people can’t face reality or don’t know how to see it anymore.
The honest position is, as it should always be, the boring one: because release can never be undone, release is the last moment evidence can be demanded, so the evidence must be presented on release. It’s the countdown, “all systems go” meaning systems are actually be evaluated. Did the Challenger explosion not teach anyone anything? History, data, recipes, declared data flows, attested deletion.
Lloyd’s Register ran on this logic for two centuries, classification before launch with the records kept in class, and nobody called that survey a restriction on shipbuilding. Hell, the British merchant fleet that fed and funded the war against Napoleon sailed under those survey records, and the insurance industry damn well knew the difference!
Openness that is anchored in transparency, a verification record as the ticket to release, is the actual goal. America needs to stop throwing shallow labels around and get back to empiricism, if it wants to be trusted at all, ever again.
