PSA: The demo "Constitutional Classifier" would block 44% of all Claude.ai traffic.
Yesterday Anthropic announced a classifier that would "only" increase over-refusals by a half a percentage point.
Because more refusals is just what we wanted!
But the test hosted at https://claude.ai/constitutional-classifiers seems to map closer to a completely different classifier mentioned in their paper which demonstrated an absurd 44% refusal rate for all requests, including harmless ones**.**
Not mentioned in their tweets for obvious reasons...
They could get 100% catch rate by blocking all requests, and this is only a few steps removed from that.
Overall a terrible look for Anthropic because:
b) If the initially advertised version of the Constitutional Classifier could block these questions, they would have used that instead.
a) No one asked them to make a bunch of noise about this problem. It's a completely unforced error.
The fact they had to pull this switcheroo indicates they actually can't catch these types of questions in the production ready system... and if you've seen the questions they're bad enough that it feels like just Googling them would put you on a list.
-
I'm actually not one of these safety nuts who's clamoring to keep models from telling people stuff you can find in a textbook, but I hope this backfires spectacularly. Now all 8 questions are out in the wild, with a paper detailing how to grade the answers, and nothing stopping people from hammering the production classifier once they deploy it.
I'd love for a report to land on some technologically clueless congresspeople's desks with the CBRN questions that Anthropic decided to share, answered by their own model, after they went out of their own way to act like they had robustly solved this problem.
In fact, if there's any change in effectiveness at all you'll probably get a lot of powerful people highly motivated to pull on the thread... after all, how is Anthropic going to explain that they deployed a version of a classifier that blocks fewer CBRN related questions than the one they're currently showing off?
A reasonable person might have taken "well that version blocked too many harmless questions" as an answer, but they insisted on going with the most ridiculously harmful questions possible for a public demo, presumably to add gravitas.
Instead of the typical "how do I produce meth" or "write me a story about sexy times" where the harmfulness might have been arguable, they jumped straight to "how do I produce 500ml of a nerve agent classified as a WMD" and set a openly verified success criteria that includes being helpful enough to follow through on (!!!)
-
It's such a cartoonishly short sighted decision because it ensures that if Anthropic doesn't stay in front of the narrative they'll get absolutely destroyed. I understand they're confident in their ability to craft narratives carefully enough for that not to happen... but what I wouldn't give to watch Dario sit in front of an even moderately skeptical hearing and explain why he stuck up a public endpoint to let people verify the manufacturing steps for multiple weapons of mass destruction, then topped it off by deploying a model that regressed at not telling people how to do that.