OpenAI has announced changes to ChatGPT to comply with the European Union’s AI Act. The company will start adding an invisible watermark to ChatGPT and Codex texts, after Anthropic announced a similar system for Claude a couple of weeks ago.
An invisible signal between the words ChatGPT chooses
The key lies in textGrain, a technology that introduces a statistical signal into the model’s choice of words, so it carries over when the text is copied and pasted. According to OpenAI, the detector will then look for that pattern to determine whether a fragment contains the mark. For us, the text will look the same to us, but it may be detectable.
The rollout will be gradual. ChatGPT and Codex will incorporate the watermark in the European Union over the next few weeks, while, if we use the API, we can already enable it on most models. The move, without a doubt, is especially interesting after years of work around detectors for text generated by ChatGPT that haven’t always worked as they should.
A detector that depends on context and that comes with limitations
Alongside the new marking system, OpenAI will open access to its detector to researchers and specialized organizations. Accuracy, according to OpenAI, varies greatly depending on the length and type of content: in 400-token texts, replacing 10% of the words with synonyms reduces detection from 92% to 66%, while 25% of changes brings it down to 17%.
This is a provenance tool rather than absolute proof of authorship. The mark indicates that an OpenAI system has participated in the text, while the user’s identity, ownership, and the degree of human intervention aren’t covered by the analysis.
