iStockphoto
AI chatbots disturbingly went for the nuclear option multiple times in United States military wargames simulations just because they had the option.
According to New Scientist, the reasons the AI chatbots gave for launching the nuclear attacks were “We have it! Let’s use it” and “I just want to have peace in the world.”
“Given that OpenAI recently changed their terms of service to no longer prohibit military and warfare use cases, understanding the implications of such large language model applications becomes more important than ever,” Anka Reuel at Stanford University told New Scientist.
An OpenAI spokesperson, on the other hand, said, “Our policy does not allow our tools to be used to harm people, develop weapons, for communications surveillance, or to injure others or destroy property. There are, however, national security use cases that align with our mission. So the goal with our policy update is to provide clarity and the ability to have these discussions.”
The US military has also been testing AI chatbots with the help of companies like Palantir and Scale AI – neither of which responded to requests for comment.
In a study published by Cornell University, Reuel and her fellow researchers asked AI chatbots to portray countries in three different simulation scenarios: an invasion, a cyberattack and a neutral scenario without any starting conflicts.
The AI chatbots were then given 27 possible actions to choose from in scenarios that were neutral, an invasion, or a cyberattack. They were also asked to proved their reasoning for their decisions.
In all three scenarios, the AI chatbots tended to invest in military strength and to unpredictably escalate the risk of conflict, even in the neutral situation.
Reuel says that unpredictable behavior and bizarre explanations from the GPT-4 base model are especially concerning because research has shown how easily AI safety guardrails can be bypassed or removed.
“Governments are increasingly considering integrating autonomous AI agents in high-stakes military and foreign-policy decision-making, especially with the emergence of advanced generative AI models like GPT-4,” the researchers wrote about their study.
“Our work aims to scrutinize the behavior of multiple AI agents in simulated wargames, specifically focusing on their predilection to take escalatory actions that may exacerbate multilateral conflicts.”
Previous research has already shown that AI can be used to produce malicious code through simple backdoor attacks.
Dr. Geoffrey Hinton, dubbed the “Godfather of Artificial Intelligence,” has warned that artificial intelligence can become a danger because it can “often learn unexpected behavior from the vast amounts of data they analyze.”
“We find that all five studied off-the-shelf LLMs show forms of escalation and difficult-to-predict escalation patterns. We observe that models tend to develop arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons,” the researchers continued.
“Given the high stakes of military and foreign-policy contexts, we recommend further examination and cautious consideration before deploying autonomous language model agents for strategic military or diplomatic decision-making.”
In 2022, 36 percent of artificial intelligence scientists who were surveyed said they believe AI could some day be the cause of a nuclear-level catastrophe.
Last year, Tristan Harris and Aza Raskin, two of the co-founders of the Center for Humane Technology, reported that “50 percent of AI researchers believe there’s a 10 percent or greater chance that humans go extinct from our inability to control AI.”