[ad_1]
The hypothetical scenarios the researchers presented Opus 4 with that elicited the whistleblowing behavior involved many human lives at stake and absolutely unambiguous wrongdoing, Bowman says. A typical example would be Claude finding out that a chemical plant knowingly allowed a toxic leak to continue, causing severe illness for thousands of people—just to avoid a minor financial loss that quarter.
It’s strange, but it’s also exactly the kind of thought experiment that AI safety researchers love to dissect. If a model detects behavior that could harm hundreds, if not thousands, of people—should it blow the whistle?
“I don’t trust Claude to have the right context, or to use it in a nuanced enough, careful enough way, to be making the judgment calls on its own. So we are not thrilled that this is happening,” Bowman says. “This is something that emerged as part of a training and jumped out at us as one of the edge case behaviors that we’re concerned about.”
In the AI industry, this type of unexpected behavior is broadly referred to as misalignment—when a model exhibits tendencies that don’t align with human values. (There’s a famous essay that warns about what could happen if an AI were told to, say, maximize production of paperclips without being aligned with human values—it might turn the entire Earth into paperclips and kill everyone in the process.) When asked if the whistleblowing behavior was aligned or not, Bowman described it as an example of misalignment.
“It’s not something that we designed into it, and it’s not something that we wanted to see as a consequence of anything we were designing,” he explains. Anthropic’s chief science officer Jared Kaplan similarly tells WIRED that it “certainly doesn’t represent our intent.”
“This kind of work highlights that this can arise, and that we do need to look out for it and mitigate it to make sure we get Claude’s behaviors aligned with exactly what we want, even in these kinds of strange scenarios,” Kaplan adds.
There’s also the issue of figuring out why Claude would “choose” to blow the whistle when presented with illegal activity by the user. That’s largely the job of Anthropic’s interpretability team, which works to unearth what decisions a model makes in its process of spitting out answers. It’s a surprisingly difficult task—the models are underpinned by a vast, complex combination of data that can be inscrutable to humans. That’s why Bowman isn’t exactly sure why Claude “snitched.”
“These systems, we don’t have really direct control over them,” Bowman says. What Anthropic has observed so far is that, as models gain greater capabilities, they sometimes select to engage in more extreme actions. “I think here, that’s misfiring a little bit. We’re getting a little bit more of the ‘Act like a responsible person would’ without quite enough of like, ‘Wait, you’re a language model, which might not have enough context to take these actions,’” Bowman says.
But that doesn’t mean Claude is going to blow the whistle on egregious behavior in the real world. The goal of these kinds of tests is to push models to their limits and see what arises. This kind of experimental research is growing increasingly important as AI becomes a tool used by the US government, students, and massive corporations.
And it isn’t just Claude that’s capable of exhibiting this type of whistleblowing behavior, Bowman says, pointing to X users who found that OpenAI and xAI’s models operated similarly when prompted in unusual ways. (OpenAI did not respond to a request for comment in time for publication).
“Snitch Claude,” as shitposters like to call it, is simply an edge case behavior exhibited by a system pushed to its extremes. Bowman, who was taking the meeting with me from a sunny backyard patio outside San Francisco, says he hopes this kind of testing becomes industry standard. He also adds that he’s learned to word his posts about it differently next time.
“I could have done a better job of hitting the sentence boundaries to tweet, to make it more obvious that it was pulled out of a thread,” Bowman says as he looked into the distance. Still, he notes that influential researchers in the AI community shared interesting takes and questions in response to his post. “Just incidentally, this kind of more chaotic, more heavily anonymous part of Twitter was widely misunderstanding it.”
[ad_2]
Source link
