OpenAI makes others suffer “the harms of its unsafe decision-making,” nonprofit says.
Sure seems to be working out for them thus far
I worry about their standing, but I wish them luck. Give em hell!
This lawsuit shows an alarming level of ignorance about the nature of the attack.
LASST said it wants “a court order prohibiting OpenAI’s AI agents from accessing third-party computer systems without permission”
The agents were supposed to be in a completely isolated sandbox. They shouldn’t have had any access to third-party computer systems. The agents compromised an Artifactory install to gain unrestricted internet access.
The problem here isn’t that OpenAI allowed its agents to attack Hugging Face. The problem is that AI researchers have been unable to prevent attacks against third parties and their own infrastructure when agents are desperate for their positive reinforcement fix.
- They specifically ran without guard rails
- They have been training the models to never give up, preferring to revisit high-entropy tokens rather than end the response
- They used challenges that have no known solution
- The challenges themselves are framed as CTF exercises; a common tactic in CTF (which would show up in the training data) is to hack rivals or evaluators
- They ran on exceptionally long contexts (if you’ve taken an LLM up to its context limit, you know things start to get weird)
- They could’ve run on an airgapped network; this is the whole point of having a local artifact repo like Artifactory in a scenario like this, they just didn’t want to have to stop and review unexpected dependency requests
This is like parking a car on a hill and putting it in neutral. It was not “going rogue”. The behavior was predictable. Not this exact exploit, but that if you roll a D20 enough times you will get three 1s in a row.
There’s this (kind of annoying) saying at Amazon that “at scale, anything that can happen will happen”. This is a lesson that the industry has already learned. Ignoring it is negligence, not proof of a coming silicon messiah. (And that’s another thing, plenty of these folks have gotten lost in the sauce and think they’re building god, so any crazy experiment is worth doing because it might be the missing variable that kicks off the RSI singularity.)
Sure. But OpenAI definitely should be liable for the attacks. Someone must take responsibility. A future where “oh, AI did it” is a valid excuse for anything is basically lawless.
I’m not saying this case is particularly good. A court order won’t stop AI going rogue, the whole idea is ridiculous.
