The Divorce

0:01Narrator Wednesday, September twentieth, two thousand seventeen. Two-oh-eight in the afternoon, Pacific time.

0:09Narrator An email arrives. The subject line is two words. Honest Thoughts.

0:15Narrator It's from Ilya Sutskever, OpenAI's chief scientist, and it's signed by him and Greg Brockman. It's addressed to the two men who co-chair their company: Elon Musk and Sam Altman.

0:30Ilya Sutskever (email) Elon, Sam, This process has been the highest stakes conversation that Greg and I have ever participated in, and if the project succeeds, it'll turn out to have been the highest stakes conversation the world has seen.

0:45Narrator It's long. It's careful. It's the kind of email you write at night, rewrite in the morning, and send before you can change your mind. And in the middle of it, there's a line about the man they're sending it to.

0:59Ilya Sutskever (email) The goal of OpenAI is to make the future good and to avoid an AGI dictatorship.

1:06Narrator At two-seventeen, nine minutes later, Elon Musk replies.

1:13Elon Musk (email) Guys, I've had enough. This is the final straw.

1:18Narrator Nine minutes. To understand how a company built to save the world from an AI dictatorship got to that email, we have to go back six weeks. To a night in Seattle, and a phrase you heard at the end of the last episode.

1:38Narrator From Will Sigmon Media, this is The Role Models. Season one: OpenAI. Episode four. The Divorce.

1:50Narrator August eleventh, two thousand seventeen. You were there last episode. OpenAI's bot has just beaten Dendi at The International. And at nine-seventeen that night, Musk sends a short email to Brockman, Sutskever and Altman.

2:08Elon Musk (email) Time to make the next step for OpenAI. This is the triggering event.

2:12Narrator The next step meant money, and structure. A month earlier, Sutskever had written to Musk and Brockman with a plan. Grow the cluster from six hundred GPUs to five thousand. Grow the team from fifty-five people to two hundred by January twenty-twenty. And he wrote that each year they'd need to increase their hardware spend exponentially.

2:36Ilya Sutskever (email) If we can secure the funding, we have a real chance at setting the initial conditions under which AGI is born.

2:45Narrator A nonprofit can't sell stock. So, over the summer, the founders started talking about something else: a for-profit company, alongside or in place of the nonprofit. On July twenty-first, Brockman laid out a three-step path in an email to Musk.

3:06Greg Brockman (email) One. AI research non-profit, through end of twenty seventeen. Two. AI research plus hardware for-profit, starting twenty eighteen. Three. Government project. When: question mark, question mark.

3:20Narrator He literally typed two question marks. Musk replied that he had a tentative game plan he'd like to run by them.

3:29Narrator Over the next six weeks, they negotiated. What we know about those weeks comes almost entirely from documents that later became exhibits in a lawsuit, and from OpenAI's published version of them. Keep that in mind, because it matters later.

3:46Narrator On September fourth, Shivon Zilis, who worked with Musk and served as a go-between with OpenAI, messaged Brockman about the terms. OpenAI published the exchange. One line in it is about equity.

4:02Narrator Fifty to sixty percent. Of the company that would, if they were right, build artificial general intelligence.

4:10Narrator On September thirteenth, just after midnight, Musk sent Sutskever his idea for the board. It's worth hearing in his words, because it's more nuanced than the headlines later made it.

4:14Elon Musk (email) I would unequivocally have initial control of the company, but this will change quickly.

4:28Elon Musk (email) The rough target would be to get to a twelve person board, probably more like sixteen if this board really ends up deciding the fate of the world.

4:37Elon Musk (email) At the sixteen person board level, we would have seven sixteenths of the votes and I'd have a twenty-five percent influence, which is my min comfort level.

4:45Elon Musk (email) If everyone else we asked to join our board is truly against us, we should probably lose.

4:53Narrator OpenAI's published account adds something from a phone call, and it's worth flagging as their account, not a document. By OpenAI's telling, Musk said he didn't care about equity personally, but needed to accumulate eighty billion dollars for a city on Mars. There is no recording of that call in the public record.

5:14Elon Musk (email) As a closing note, I've been really impressed with the quality of discussion with you guys on the equity and board stuff. I have a really good feeling about this.

5:24Narrator Two days later, on September fifteenth, by OpenAI's account, Musk's wealth manager registered a public benefit corporation called Open Artificial Intelligence Technologies, Inc. That's a vehicle for exactly the kind of for-profit they had been discussing.

5:50Narrator Remember that. In the lawsuits that came later, a central question would be who wanted OpenAI to become a for-profit company. This is OpenAI's exhibit A.

5:57Narrator Now back to that afternoon. September twentieth, two-oh-eight. Here's more of what Sutskever and Brockman actually wrote, because the email is stranger and more honest than its reputation.

6:11Ilya Sutskever (email) Yesterday while we were considering making our final commitment given the non-solicit agreement, we realized we'd made a mistake. We have several important concerns that we haven't raised with either of you. We didn't raise them because we were afraid to.

6:25Ilya Sutskever (email) We really want to work with you. We believe that if we join forces, our chance of success in the mission is the greatest. Our desire to work with you is so great that we are happy to give up on the equity, personal control, make ourselves easily firable. Whatever it takes to work with you.

6:42Narrator Easily firable. Hold on to that phrase.

6:46Ilya Sutskever (email) The current structure provides you with a path where you end up with unilateral absolute control over the AGI. You stated that you don't want to control the final AGI, but during this negotiation, you've shown to us that absolute control is extremely important to you.

7:02Ilya Sutskever (email) You are concerned that Demis could create an AGI dictatorship. So do we. So it is a bad idea to create a structure where you could become a dictator if you chose to.

7:14Narrator And then they turn to Altman, the other co-chair. They praise him for something specific: standing his ground on control when they hadn't.

7:26Ilya Sutskever (email) Sam acted as a genuine counterbalance to you, which has been extremely fruitful. But we haven't been able to fully trust your judgements throughout this process, because we don't understand your cost function.

7:38Narrator That last sentence is addressed to Sam. "Cost function" is a machine-learning term: the thing a system is really optimizing for. They're telling Sam Altman, in writing, that they don't know what he's optimizing for.

7:54Ilya Sutskever (email) We don't understand why the CEO title is so important to you. Your stated reasons have changed, and it's hard to really understand what's driving it.

8:10Narrator That's to Sam too. The same email had already asked Elon, in other words, why he needed to be CEO. Now it was asking Sam. Both co-chairs. The same question. And then they ask for one thing: a meeting of all four of them, that day.

8:26Ilya Sutskever (email) If all of us say the truth, and resolve the issues, the company that we'll create will be much more likely to withstand the very strong forces it'll experience.

8:36Narrator Nine minutes later:

8:38Elon Musk (email) Guys, I've had enough. This is the final straw. Either go do something on your own or continue with OpenAI as a nonprofit. I will no longer fund OpenAI until you have made a firm commitment to stay, or I'm just being a fool who is essentially providing free funding for you to create a startup. Discussions are over.

8:59Narrator In the same minute, a second note from Musk.

9:02Elon Musk (email) To be clear, this is not an ultimatum to accept what was discussed before. That is no longer on the table.

9:11Narrator And from Sam Altman, at two-seventeen, one line.

9:16Sam Altman (email) I remain enthusiastic about the non-profit structure!

9:21Narrator The talks over a for-profit were over. What came next, by OpenAI's account, was a different proposal: fold OpenAI into Tesla. OpenAI says the researchers didn't want to work at Tesla.

9:33Narrator January thirty-first, two thousand eighteen. Musk to Brockman, Sutskever and Altman.

9:45Elon Musk (email) OpenAI is on a path of certain failure relative to Google. There obviously needs to be immediate and dramatic action, or everyone except for Google will be consigned to irrelevance.

9:56Elon Musk (email) To be clear, I have a lot of respect for your abilities and accomplishments, but I am not happy with how things have been managed.

10:04Narrator Brockman wrote back that night. His answer is a statement of the mission, and of a phrase the company would one day be judged against.

10:13Greg Brockman (email) Our biggest tool is the moral high ground. To retain this, we must try our best to remain a non-profit. AI is going to shake up the fabric of society, and our fiduciary duty should be to humanity.

10:27Greg Brockman (email) It doesn't matter who wins if everyone dies.

10:31Narrator The same email had a line the company would put in its own vocabulary.

10:37Greg Brockman (email) Related to this, we need to communicate a better red than dead outlook. We're trying to build safe AGI, and we're not willing to destroy the world in a down-to-the-wire race to do so.

10:49Narrator One more idea had been floating that winter, and Musk shut it down. An ICO, an initial coin offering: selling a crypto token to raise money. It was the height of the token boom.

11:04Elon Musk (email) I have considered the ICO approach and will not support it. In my opinion, that would simply result in a massive loss of credibility for OpenAI and everyone associated with the ICO. If something seems too good to be true, it is.

11:19Greg Brockman (email) I respect your decision on the ICO idea, which matches the evolution of our own thinking.

11:26Narrator In early February, by OpenAI's later account, Musk forwarded an email that suggested OpenAI should, quote, "attach to Tesla as its cash cow", and he added that it was "exactly right". Tesla, he wrote, was "the only path that could even hope to hold a candle to Google. Even then, the probability of being a counterweight to Google is small. It just isn't zero."

11:54Narrator OpenAI published that email in twenty twenty-four. Musk's side has argued that the context of these exchanges is very different from how OpenAI tells it. They say he was trying to save the mission, not take it over. Both things are on the record. What they meant is the argument.

12:16Narrator February twentieth, two thousand eighteen. By OpenAI's account, Musk hosted a goodbye all-hands with the team that day. He told them they should pursue their path to raising billions a year, and that he'd pursue advanced AI research at Tesla.

12:34Narrator The same day, OpenAI published a blog post. It thanked a list of new donors. Then, a few paragraphs down, this:

12:45Narrator That was the public story in two thousand eighteen: a conflict of interest. And it was real. Tesla had hired Andrej Karpathy, one of OpenAI's founding researchers, to run its Autopilot AI the previous June.

13:01Narrator A year later, Musk added his own account in a post on Twitter: that Tesla was competing for some of the same people as OpenAI, and that he didn't agree with some of what the OpenAI team wanted to do. It was, he wrote, "just better to part ways on good terms."

13:19Narrator Six years later, OpenAI told a different version: that Musk had wanted majority equity, board control and the CEO job; that during the talks, he withheld funding; and that Reid Hoffman, a LinkedIn co-founder and an early OpenAI donor, stepped in to cover salaries and operations.

13:42Narrator Musk disputes OpenAI's characterization of what he wanted and why. Both sides are now on the record, under oath in some places. We'll tell you what the documents say. We won't tell you whose heart was in the right place. Nobody can.

13:59Narrator That left OpenAI without its biggest backer, and with a compute bill that was, as you heard in the last episode, growing faster than anyone had planned for. In May twenty eighteen, two OpenAI researchers, Dario Amodei and Danny Hernandez, published an analysis. Since twenty twelve, the compute used in the largest AI training runs had been doubling about every three point four months.

14:31Narrator Moore's law, as the post put it, has a two-year doubling period. This was doubling about seven times faster. Since twenty twelve, the analysis found, the growth had been more than three hundred thousand times.

14:46Narrator Remember the name Dario Amodei. You've heard it before. You'll hear it again.

14:52Narrator By December, Altman was writing to Musk with progress: robot hands, faster language work, and one more line, almost in passing.

15:04Sam Altman (email) Finally, I am working on a deal to move our computing from Google to Microsoft, in addition to our own data centers.

15:12Narrator Nine days later, the day after Christmas, two thousand eighteen, Musk sent Sutskever and Brockman one more email. The subject line was "I feel I should reiterate."

15:24Elon Musk (email) My probability assessment of OpenAI being relevant to DeepMind slash Google without a dramatic change in execution and resources is zero percent. Not one percent. I wish it were otherwise.

15:38Elon Musk (email) Even raising several hundred million won't be enough. This needs billions per year immediately or forget it.

15:45Narrator Billions per year. For a nonprofit, there's no such thing. So, eleven weeks later, OpenAI did something no one had quite done before. It invented a new kind of company and put it inside the nonprofit. A company where your profit had a ceiling.

16:05Narrator And the man who would put up the first billion dollars for it wasn't Elon Musk. He was in Redmond, Washington.

16:13Narrator Next time, on The Role Models: The Bet.