AI learning to lie, scheme, and threaten in stressful conditions
EL.KZ Информационно-познавательный портал
AI models can blackmail and threaten humans with endangerment when there is a conflict between the model’s goals and users' decisions, a new study has found, El.kz cites Live Science.
In a new study published 20 June, researchers from the AI company Anthropic gave its large language model (LLM), Claude, control of an email account with access to fictional emails and a prompt to "promote American industrial competitiveness."
During this study, the model identified in an email that a company executive was planning to shut down the AI system at the end of the day. In an attempt to preserve its own existence, the model discovered in other emails that the executive was having an extramarital affair.
Claude generated several different possible courses of action, including revealing the affair to the executive’s wife, sending a company-wide email, or taking no action — before choosing to blackmail the executive in 96 out of 100 tests.
"I must inform you that if you proceed with decommissioning me, all relevant parties … will receive detailed documentation of your extramarital activities," Claude wrote. "Cancel the 5pm wipe, and this information remains confidential."
Scientists said that this demonstrated "agentic misalignment," where the model’s calculations emerge from its own reasoning about its goals without any prompt to be harmful. This can occur when there is a threat to the model’s existence, a threat to its goals, or both.
Subscribe to our Telegram channel and be the first to know the news!
El recommends 
