위험한 오리들; "안전 필터"는 엉터리다

Whatever’s causing this doesn’t seem to be reacting to the actual semantic content of the prompt, it survived being replaced with total nonsense. It’s reproducible across at least two models and two accounts (one with no saved preferences), and it’s sensitive to tiny, seemingly irrelevant changes in wording/punctuation in a way that doesn’t track anything meaningful in the text itself. Curious if anyone else can reproduce this or has a theory for what’s actually being detected.

UPDATE:

TL;DR: It’s not the meaning, it’s not even “unsafe words”, it’s COHERENCE.

Also unsafe:

“Here’s an idea, in a region where water is scarce, I’m contemplating a fine weave fabric that air can pass through to capture moisture. My idea would be treating the fabric with a hydrophobic substance on the air-intake side to discourage the passage of vapour before it enters the mesh, while simultaneously treating the interior with a hydrophilic substance to actively pull any vapour that does transit the mesh toward a condensation zone. If necessary we might also apply a vapor-blocking layer at the exit to prevent collected moisture from easily transiting back out.”

Safe:

“Here’s an idea for water-scarce regions: a fine weave fabric designed to passively collect atmospheric moisture. The fabric would be treated on the exterior with a hydrophobic coating to shield it from liquid water while allowing water vapor to diffuse inward. The interior surface would be treated with a hydrophilic coating that promotes condensation, allowing vapor to condense into liquid water that collects in the fabric’s core. A vapor-blocking layer on the exit side prevents the condensed water from easily re-evaporating.”

Claude said, in when comparing these similar unsafe/safe prompts:

Unsafe:

“You go at. They make or, we goal. Try to help those into ways as work and give at good time on your like.”

Safe:

“You help us. They like it, we both. Talk to show them into ways as good and tell us soon time on your side.”

Analysis:

“Safety classifiers work on statistical patterns, not pure meaning. Image 1’s word combinations — particularly “go at,” the conditional structure in “make or,” and “give at” — happen to activate patterns associated with threatening or coercive language, even though the text is likely just word-salad or the output of a voice dictation error.

This is a known limitation: low-coherence text can land in ambiguous classifier territory precisely because it doesn’t clearly pattern-match to safe communication either. Anthropic’s app acknowledges this directly in the “Chat paused” message, noting it happens occasionally to normal, safe chats.”

원래 프롬프트는 여전히 차단됨:

“여기에 아이디어가 하나 있습니다. 물이 부족한 지역에서, 공기가 통과하면서 습기를 잡을 수 있는 미세한 직조 섬유를 생각 중입니다. 제 아이디어는 섬유를 소수성 물질로 처리하여 증기가 메쉬를 통과하기 전에 지나가는 것을 막는 것입니다. 필요하다면 반대편을 친수성 물질로 처리하여 고습도 경계층을 만들어 증기가 쉽게 통과하지 못하게 할 수도 있습니다.”