Watermarking, a new framework for traceability and transparency in AI?

Dis­tin­guish­ing human-gen­er­ated con­tent from AI-gen­er­ated con­tent is becom­ing increas­ingly dif­fi­cult as gen­er­at­ive AI mod­els advance. The whole issue of trans­par­ency among mod­el pro­viders such as OpenAI, Anthrop­ic and Mis­tral is now at stake. Since 2 August 2026, the European Com­mis­sion has required these pro­viders to incor­por­ate an invis­ible water­mark into their mod­els to mark con­tent with an AI label. Art­icle 50(2) of the AI Act, which sets out this new frame­work, is hailed by experts in the field as a very good start, although it faces sig­ni­fic­ant tech­nic­al con­straints, par­tic­u­larly those related to cybersecurity.

A digit­al water­mark involves insert­ing a set of vis­ible or invis­ible data into a file or sig­nal (audio, video, image, text, etc.). The concept exis­ted long before gen­er­at­ive arti­fi­cial intel­li­gence flooded the web. “Water­mark­ing began in the 2000s, with the primary aim of pro­tect­ing copy­right,” explains Patrick Bas, a CNRS research­er in stegano­graphy based in Lille. Whilst it con­tin­ues to be used to authen­tic­ate artist­ic con­tent, the water­mark is find­ing a new pur­pose with the wide­spread deploy­ment of AI: it is now used to detect for­ger­ies, par­tic­u­larly in the fight against online disinformation.

Each media type has its own tech­niques. “For images, we’ll slightly alter the pixel val­ues; for text, we’ll replace one word with anoth­er; and for sound, we’ll change the amp­litude of the audio sig­nal, for example”, explains the research­er. Wheth­er embed­ded dur­ing con­tent cre­ation or added after­wards, the water­mark should have as little impact as pos­sible on the qual­ity of the out­put. To sup­port Art­icle 50(2), a code of prac­tice for gen­er­al-pur­pose AI spe­cific­ally requires that the water­mark be “imper­cept­ible” and “dif­fi­cult to sep­ar­ate from the con­tent itself”. Nearly a hun­dred AI organ­isa­tions, includ­ing the giants OpenAI and Anthrop­ic, have signed the first sec­tion ded­ic­ated to watermarking.

Most of the water­mark­ing tech­niques used for AI-gen­er­ated con­tent emerged only a few years ago. “In the past, to carry out water­mark­ing, we used maths. Today, we train arti­fi­cial intel­li­gence mod­els to embed a mark into con­tent whilst min­im­ising dis­tor­tions”, sum­mar­ises Patrick Bas. Among the most wide­spread solu­tions on the mar­ket are Google’s Syn­thID water­marks. Ori­gin­ally developed for images in 2023, and sub­sequently for text, audio and video, they are now fea­tured in all of its mod­els, includ­ing the hugely pop­u­lar Gem­ini. This is also the solu­tion adop­ted by its com­pet­it­or OpenAI for images and sounds gen­er­ated by ChatGPT.

For the water­mark to be of any use, it must still be pos­sible to detect it, even if it is not vis­ible to the human eye. Com­pan­ies devel­op­ing AI mod­els must there­fore have a solu­tion to make it stat­ist­ic­ally detect­able. The Com­mis­sion requires that a detec­tion tool be made avail­able to the gen­er­al pub­lic, with the excep­tion of text, as cur­rent detect­ors are deemed too unre­li­able. As a sig­nat­ory to the code of prac­tice, Google made a Syn­thID detect­or for images, audio and video avail­able world­wide on 7 Octo­ber. A ver­sion of the tool had pre­vi­ously been restric­ted to media professionals.

“Devel­op­ing these solu­tions is expens­ive. If it’s simply a mat­ter of com­pli­ance, pro­viders will do it as cheaply as pos­sible,” warns Teddy Fur­on, a research­er in digit­al water­mark­ing at Inria Rennes and co-founder of Label4.ai, an AI water­mark­ing and detec­tion start-up. The Com­mis­sion is, how­ever, attempt­ing to estab­lish qual­ity stand­ards for the solu­tions being developed. It has there­fore defined four cri­ter­ia: effect­ive­ness, reli­ab­il­ity, robust­ness and inter­op­er­ab­il­ity. The res­ults pro­duced by the detect­ors must be com­pre­hens­ible (effect­ive); they must min­im­ise false pos­it­ives and false neg­at­ives (reli­able); the water­marks are sup­posed to with­stand trans­form­a­tions such as audio and image com­pres­sion, which are very com­mon on social media where AI-gen­er­ated con­tent is shared on a massive scale (robust); and the detect­ors must work across sev­er­al dif­fer­ent water­mark­ing sys­tems (inter­op­er­able).

“These are the four cri­ter­ia, with­in the lim­its of what is tech­nic­ally feas­ible. Leg­ally, this trans­lates into an oblig­a­tion of res­ults that is entirely determ­ined by the state of the art”, explains Betty Jeulin, a law­yer spe­cial­ising in new tech­no­lo­gies and intel­lec­tu­al prop­erty law in Par­is. She, who advises sev­er­al pro­viders of AI mod­els to help them com­ply with reg­u­la­tions, admits: “Today, no one is able to pro­duce a solu­tion that per­fectly meets these four criteria.”

Inter­op­er­ab­il­ity is, in par­tic­u­lar, one of the most con­ten­tious points in Art­icle 50(2). It is clear that users will find it all the easi­er to identi­fy AI-gen­er­ated con­tent if they do not have to search for which sys­tem flagged it. Ideally, a single inter­face would be able to detect any water­mark. How­ever, the mar­ket for detect­ors is cur­rently highly frag­men­ted. This is why the Com­mis­sion expects “sig­ni­fic­ant pro­gress” in this area from the sig­nat­or­ies, who are expec­ted to work togeth­er. The code of prac­tice states that, by 2 Feb­ru­ary 2027, they must have provided at least the “build­ing block” that will enable con­nec­tion to dif­fer­ent sys­tems or dir­ect users to the cor­rect detector.

Mak­ing a detect­or pub­licly avail­able is neces­sary to cre­ate a trus­ted infra­struc­ture, but this inev­it­ably raises secur­ity con­cerns. “Query­ing these tools a large num­ber of times in an attempt to remove a water­mark can con­sti­tute a for­mid­able attack, known as an “oracle attack”. Start­ing with water­marked images, for example, an attack­er can slightly alter the pixels until the detect­or no longer recog­nises the water­mark, thereby map­ping its detec­tion area. “With images, it only takes a few hun­dred calls to a black-box detect­or to remove the water­mark with rel­at­ively good qual­ity,” says Teddy Furon.

Indeed, tools designed to remove water­marks are begin­ning to appear on the web. If they prove suf­fi­ciently accur­ate and their use becomes wide­spread, they will help to sow doubt where trans­par­ency is, on the con­trary, expec­ted. For their part, pro­viders have an interest in lim­it­ing their attack sur­face by restrict­ing the num­ber of requests per user, or even offer­ing the gen­er­al pub­lic scaled-down detect­ors. This rep­res­ents a com­prom­ise between secur­ity and reli­ab­il­ity, one of many such choices that com­pan­ies must make. How­ever, it is unclear how these vari­ous trade-offs will be assessed by the Com­mis­sion, which admits it has not yet estab­lished any eval­u­ation methods.

Whilst far from fool­proof, water­mark­ing remains one of the few solu­tions cur­rently avail­able to improve the trace­ab­il­ity of AI. “The text as it stands is already a huge step for­ward,” says Teddy Fur­on. “There is a risk that the tech­niques may not live up to the Com​mis​sion’s expect­a­tions. But if we man­age to cov­er 90% of con­tent, per­haps that alone will change the way people behave on plat­forms,” hopes Patrick Bas.

In terms of the timetable, pro­viders have until 2 Decem­ber to bring their mod­els released before 2 August into com­pli­ance. In addi­tion to invis­ible water­mark­ing, they must add metadata to the files they gen­er­ate in accord­ance with the C2PA stand­ard so that their ori­gin can be traced. This is one way of redu­cing the risk of mis­in­form­a­tion, even though the inform­a­tion it con­tains can eas­ily be deleted.

Charlotte Delfosse