LLM watermark reversing

[chants]

So due to EU regulation, Claude is now watermarking AI output. https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content

Generally it is obvious the way it is done is by the PRNG used in the sampling to select the next token based on their probabilities. So the PRNG could seed based on the prior 10 tokens or such. This adds a lot more determinism and obviously seeding is expensive so the details might be different. I heard Gemini uses a different tournament style algo.

Defeating this looks very easy, just clean it through a non watermarked model or have a model suggest substitutions that are applied randomly to break the signal they measure.

Do you think deducing their proprietary watermark is possible? Such as asking the model many times to complete 1 token, then 2 token prefixes, etc s you can build a prefix based probabilities or start to learn how many prior tokens are used wtc? Assuming you can stay under the radar, I think this would allow certain parts of the algo to be determined. A completion sampling attack on watermark scheme.

Any other better techniques to remove or defeat it? I found this paper on one idea De-mark: Watermark Removal in Large Language Models https://arxiv.org/abs/2410.13808