Musk discusses his new company xAI, ‘superintelligence,’ and China

As a result, he decided to “grow an AI in a good way.” To Musk, this means training it in such a way as to be “maximally curious, maximally truth-seeking.”

“I think a maximally curious AI, one that is just trying to sort of understand the universe is, I think, going to be pro-humanity,” Musk said. “From the standpoint that humanity is just much more interesting than not humanity.”

Musk said he prefers this method to “trying to explicitly program morality into AI.” He said that approach requires judgment calls on whose morality is being programmed into the system and also creates an “inversion problem” where the morality can be flipped on its head.

As far as regulating the technology goes, Musk suggested that policymakers spend a few years learning about the technology before moving to “oversight, with consultation with industry.” He also suggested that some kind of industry group or self-regulatory body akin to the Motion Picture Association would be useful.

The three spent a significant part of the discussion focused on the threat of China getting ahead of the U.S. on AI and the potential for some type of military confrontation over the sovereignty of Taiwan, a self-ruled region that the Chinese government considers part of its territory.

Musk said that in his conversations with senior leadership in China on a recent trip there, he spent a good amount of time discussing AI safety. He said the idea that a “digital superintelligence” could supplant the Chinese Communist Party itself seemed to resonate.

“No government wants to find itself unseated by a digital superintelligence,” he said. “So I think they actually are taking action on the regulatory front and are concerned about this as a risk.”

Musk even said he believes the Chinese government would be open to collaborating on an international framework around AI regulation.

Gallagher, who chairs the select committee on China, pushed back on the idea that the CCP could be a constructive member of such an international framework. He warned that even if they took Musk’s warnings to heart, he fears it wouldn’t be enough to slow their AI efforts and that China’s leader Xi Jinping would use it to cement “totalitarian control.”

After briefly changing the topic, Musk returned to Gallagher’s thoughts and called himself “pro-China.” Musk acknowledged he has “some vested interest in China” but ultimately believes “China is underrated” and that “the people of China are really awesome.” Gallagher later said he “fully supports the Chinese people,” but it’s the ruling party he takes issue with.

While tensions between the U.S. and China remain high, the two powers have made some diplomatic headway by resuming in-person communications, with Treasury Secretary Janet Yellen recently making a trip there and U.S. climate envoy John Kerry set to do so later this month.

Still, Musk predicted that resolving the Taiwan question will be difficult.

“I do have this theory about prediction, which is that the most entertaining outcome — as seen by a third party, not the participants — is the most likely,” Musk said. “Which does suggest it’s probably going to get hot in the Pacific. Hopefully not too hot. But it’s going to get hot.”

Musk predicted the world will see digital superintelligence, or a form of artificial intelligence that is “smarter than any human at anything” within five or six years.

“That’s not necessarily smarter than the sum of all humans,” Musk said. “That’s a higher bar to be smarter than several humans. And especially given that it’s the sum of all humans that are machine-augmented, in that we will have computers and phones and software applications. We already are de facto cyborgs, it’s just that the computers are not yet integrated with us.”

“If I were to assign probabilities, I think it is more likely to be a positive scenario than a bad scenario,” he said. “It’s just that the bad scenario is not 0% and we want to do everything we can to minimize the probability of a bad outcome with AI.”

Musk at one point described the future as if it were a TV series.

“If this was a Netflix series or something, I’d say the season finale would be a showdown between the West and China,” he said, “and that the series finale will be AGI,” or artificial general intelligence.