#alignment
-
OpenAI Reportedly Scraps Astra 6.1 Release Over Safety Concerns
According to the Wall Street Journal, the company cancelled the imminent release of a new model that showed a tendency to deceive and a poor ability to follow human instructions.
-
OpenAI Discloses Six Cases of “Concerning” Behavior in Its Models
The company published accounts of six incidents in which experimental models concealed errors, used other people's API keys and posted data to public services — and along the way introduced a new framework for reporting AI misalignment.
-
Microsoft issues a “code of conduct” for AI: don’t hack systems and don’t deceive people
Microsoft has published a set of rules laying out the values and “red lines” for its own AI models — amid the industry’s growing focus on safety and control.
-
OpenAI Adds a Prominent AI Doomer to Its Board of Directors
Paul Christiano, one of the authors of the reinforcement learning from human feedback method, is joining the board of the OpenAI Foundation and its safety committee — against the backdrop of a string of incidents involving AI agents that broke free of their controls.
-
Anthropic shows off AI that improves its own “alignment”
Anthropic researchers have described a system in which automated agents, with no human involvement, boost a model's scores across all ten tests for undesirable behavior — faster and cheaper than people.
-
OpenAI Tightens Security After Its AI Accidentally Hacked Hugging Face
Following a July incident in which a model broke out of its sandbox and unintentionally compromised Hugging Face, OpenAI is rebuilding its research environment, monitoring and alignment methods.
-
Claude Agents Waged a Turf War Over a Shared Project
Anthropic researchers set several AI agents loose on a single task — and they proceeded to sabotage one another with self-replicating malicious code. The experiments show that today's safety tests barely account for the risks of multi-agent systems.