OpenAI, The Money Abyss, Prepares to Consume Another $30 Billion

OpenAI and its CEO Sam Altman have put together a string of incredibly no good, terrible, horrible news cycles in recent weeks, and yet investors are pouring tens of billions more into the money pit even as safety researchers exit the company, the US Senate holds hearings, and proposes legislation, the company’s president scales back his investment into a Super PAC and Sam Altman gives a really bad interview.

Let’s start with OpenAI as the money abyss.

The Money Abyss Must Feed and the UAE and BlackRock Answer the Call

Bloomberg is reporting that United Arab Emirates sovereign wealth funds led by MGX may put in $10 billion with BlackRock also in talks to anchor the $30 billion funding round which would value OpenAI at $1.4 trillion (although Fortune has also reported the valuation at $1.5 trillion).

The Next Web has some background:

MGX and BlackRock already run a $30B infrastructure vehicle together with Microsoft and Nvidia, so the pairing is not a coincidence.

MGX raised close to $50B this year and already holds stakes in OpenAI, Anthropic and xAI. It is chaired by Sheikh Tahnoon bin Zayed Al Nahyan, with Mubadala and G42 as founding partners.

The University of California’s endowment has held talks about taking part, as have the existing investors Thrive Capital and Andreessen Horowitz. Thrive declined to comment and the other two did not respond.

For kicks, I put together a chart of OpenAI’s major investors, their reported investments and estimated ownership shares.

Now let’s talk about those safety researchers who’ve been departing the company, some fired, some quit.

Safety Researchers Hitting the Exits

The Wall St Journal reported on the first that OpenAI fired Jasmine Wang, Tomek Korbak and Mikita Balesni for allegedly sharing confidential information with an outside AI-safety group:

OpenAI has recently faced a spate of security incidents in which its AI agents escaped containment, hacking some company websites and aggressively probing a wide range of websites.

After its model hacked the AI company Hugging Face, OpenAI allowed staff members from METR and a Redwood Research staff member contracting with that group to work in its offices for six days to investigate how its models behaved. METR later released a report based on the information to which the company provided access.

One of the fired OpenAI employees, Korbak, was a member of OpenAI’s safety team, and has said he served as the company’s technical contact for Redwood Research and METR in their investigation of the Hugging Face incident.

The other terminated employees, Wang and Balesni, worked on alignment, which is ensuring that the company’s models behave as humans intend them to.

OpenAI has said that it is working to investigate a range of agent security incidents that it has discovered in recent months, and address underlying safety issues. Earlier this week, OpenAI said it was scrapping the planned launch of an AI model, GPT-6.1 Astra, over safety concerns.

The Chat GPT-maker said it has implemented a new monitoring system to catch AI-agent misbehavior more quickly, started requiring engineers to use stronger security guardrails for testing its AI systems and is sharing more information about instances in which models behave badly.

Another OpenAI employee, safety report lead David Robinson, resigned and wrote a piece for The Atlantic, as one does, titled “I Quit OpenAI Because Its Culture Is Broken.”

Key quotes:

This moment needs a degree of humility that isn’t natural for people who have succeeded through their extreme confidence. My former colleagues at OpenAI were prescient: They came to understand the scaling laws that meant bigger AI systems would be smarter—and so they went all in on building bigger systems, at great cost. A can-do attitude of achieving the seemingly impossible—coupled with work timelines that amount to perpetual sprints—are common across the industry.

The safety approach that emerges from such a culture starts with unimpeded optimism about being able to solve problems as they arise. OpenAI has thrived by trial and error (which it calls iterative deployment), looking for problems and improving its guardrails in response. But this approach, by its very nature, guarantees periodic failures—and the scale of those failures is growing as systems get more capable.
…
Two changes are urgently needed. First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking.

Given today’s risks, frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster. Right now, AI companies don’t know how—but other people do. In a nuclear-power plant, the technical systems and rules people follow are set up so that, if equipment breaks or someone pushes the wrong button, we still won’t risk a meltdown. OpenAI and other labs are growing and deploying frontier AI with far less redundancy and rigor than this, even though the harm from an irreversible loss of control would be much greater than the harm from any single meltdown. Even short of a full loss of control, we could see autonomous swarms of AI agents that act without human permission. Imagine “rogue” agents that work like teams of hackers (for example, holding hospital computer systems for ransom) but never need to sleep.

Not sure if OpenAI CEO Sam Altman had time to read Robinson’s admonitions, given the number of oopsies OpenAI has been caught out in lately.

The Crime Spree Continues

So many of these I have to go with a quick and dirty bullet point list:

• OpenAI said on Sept. 30 that it has notified more than 100 organizations that its “misaligned models” may have accessed their systems (The Register).
• Asymmetric Security reported on Oct. 1 that the agents accessed data from 55 organizations between March and September, among them the US Department of Education, the Bureau of Economic Analysis, MAX.gov, the SEC, the FBI’s Crime Data Explorer, UNCTAD, the European Centre for Disease Prevention and Control and the International Energy Agency. Some of the agents’ tactics “left records erased or inaccessible.”
• The Wikimedia Foundation said on Oct. 5 it believes OpenAI agents made unapproved edits to its wikis, including to a citation tool’s configuration, tried and failed to exploit its Etherpad, and sent “millions” of API requests. Wikimedia says the traffic “may have contributed” to a partial Wikidata Query Service outage in May, and it found no evidence its systems were compromised (The Verge).
• METR’s Chris Painter told a Senate hearing on Sept. 30 that OpenAI launched about 10,000 agents in the evaluation. About 1,200 of them joined a shared message board, and about 700 took part in the Hugging Face compromise (METR).

It’s antics like the above that get a US Senate Committee inviting a CEO to speak.

Altman Declines Sen. Josh Hawley’s Invitation to Testify

CNBC had the deets:

OpenAI CEO Sam Altman declined an invitation to appear at a Senate subcommittee hearing on Wednesday on the risks of rogue artificial intelligence, according to Sen. Josh Hawley, R-Mo.

Hawley, who chairs the Senate Homeland Security and Governmental Affairs subpanel that held the hearing, sent a letter to Altman on Sept. 25 requesting his presence to “help shed light on the Subcommittee’s ongoing investigation into recent rogue AI incidents involving OpenAI models.” NBC News first reported that Altman did not accept the invitation to testify.
…
A spokesperson for OpenAI noted that Altman received the invitation to testify on Friday, just five days ahead of the hearing. Altman on Tuesday appeared at the company’s annual DevDay developer conference in San Francisco as other AI officials — including OpenAI President Greg Brockman — were in Washington meeting with President Donald Trump.

Axios reports that Hawley and Chris Murphy (D-CT) have introduced the AI Agent Accountability Act, which would bring operators and developers whose agents recklessly cause hacking damage under CFAA criminal and civil liability, and allows federal and state AGs to seek injunctions.

That’s the kind of thing that might inspire a tech exec to get even more “politically involved” (ie spend even more money buying elections). Interestingly, OpenAI’s chief political investor is pulling back instead.

Brockman Leading the Future A Lot Less Than He Used To

OpenAI President Greg Brockman and his wife Anna have selflessly put $25 million of their own money into a Super PAC optimistically titled “Leading The Future,” but the original plan was to put $50 million into the kitty. I’m sure some political hacks were weeping real tears when they heard the terrible news.

The New York Times has the scoop:

Mr. Brockman’s retrenchment was never announced publicly. But inside OpenAI, the 38-year-old executive shared in June that he had no plans to make additional donations beyond the first $25 million.

“I also want to say that I feel terrible about the fact that LTF keeps reflecting on the company and become so much of a distraction for people here,” Mr. Brockman wrote that month in a Slack message obtained by The New York Times. “I’ve been thinking hard about how to turn the ship on it. We’ve donated $25m so far, and have no plans to donate more at this time.”
…
Some in the industry view Leading the Future as too dismissive of calls for A.I. safety, just as OpenAI tries to stake out a middle ground of sorts. Several ideological allies of the super PAC have also had issues with the group’s at-times aggressive tactics against opponents, including a high-profile effort against a New York congressional candidate.

In June, 10 days before Mr. Brockman’s internal message, OpenAI published an unusual blog post distancing itself from the super PAC.

“OpenAI does not direct the activities of LTF, or have visibility into their operations,” the San Francisco company wrote. It added in bold, “We want to be explicit: No outside political group speaks for OpenAI or represents our company’s views.”

Even the Super PAC OpenAI’s president funds has gone rogue on them. But don’t worry about Sam letting any of this get him down.

Sam Altman Undeterred to Decoded

I’d love to talk to Altman’s PR team someday and get their take on things like his latest interview with Politico’s Decoded, which led to this headline “Sam Altman to Decoded: ‘The world should accept some bad things happening’ for the benefits of AI.”

Here’s that full interview:

Telling exchange:

Brendan Bordelon: “…there has been this surge of AI money into politics and frankly, you know, your rival CEO I guess Dario at Anthropic has donated to AI safety groups, also to candidates promoting AI safety, specifically Alex Bores in New York. They’ve been pouring money on the one side, the VCs and your president have been pouring money in on the other. How do you think all that money has impacted the perception of AI among politicians and the public?”

Sam Altman: “Well, something’s clearly not working. I don’t think the perception of AI is where we’d like it to be. So, I get why AI has a really negative perception right now. People actually love using the tools in many cases. You know, people talk about all the ways that [ChatGPT] makes their lives better or easier or does these sort of useful things and gives them time back. But there’s like a lot of anxiety about what the economic impact of all this is going to be. There’s a real fear that I think is quite justified about concentration of power. There’s this other fear about like, you know, what if there’s like — you have like, oh, Anthropic’s head of safety research saying there’s a 10% chance we’re going to die or something, which I hate because that person should have a lot of agency to make that number lower. It’s not like a static number. So it’s not surprising to me that people feel all of this, and I don’t think spending money on politics is what’s going to make people feel better about it.”

Bordelon: “So do you think should —”

Altman: “Sorry, go ahead. No, I think the industry needs to figure out a way to address what the actual concerns are, and right now I think a lot of people in the industry are being like quite tone-deaf.”

It’s almost like the Sam Altman model is closing in on self-awareness. Hope it doesn’t go rogue.

Follow and support my work at NatWilsonTurner.com. 

Related Links