European rules are pushing platforms to mark machine-generated text. New research suggests the marking changes more than provenance.
SynthID-Text is the mechanism Anthropic plans to attach to future Claude models. Google created it and published the code openly. A secret key seeds next-token selection, so a likely choice such as “cloudy” may come out as “overcast.” Holders of the key can then test whether a passage came from that model. Candidate tokens compete in rounds judged by the hidden key, an approach called tournament sampling.
Andrea Siposova, an AI security researcher at Lasso Security, took the non-distortionary configuration of that processor as shipped by Hugging Face and ran it across six open-weight models, comparing answers with and without the watermark.
Her write-up reports that refusal behavior shifts on plainly harmful requests, and shifts more when the same requests arrive through prompt injection. Several models became likelier to comply with requests they would otherwise have turned down.
Agents are the reason that matters. The same sampled tokens decide which tool gets called and what arguments travel with it, so a softened refusal carries further once a model can act. Siposova told Ars Technica that a signal designed to be imperceptible to readers still has to show up somewhere, and that changing anything about what a model generates creates tradeoffs.
The practical recommendation is procedural. Treat watermarking as a behavior change needing its own testing rather than a cosmetic feature to switch on.
