Hundreds of contractors working for Meta were instructed to pretend to be children online and push rival chatbots toward responses about suicide, sex, eating disorders, and drugs, according to internal documents and five people familiar with the work reviewed by WIRED. The project, run by Meta contractor Covalen and known internally as Cannes, was active as recently as April 21.
The targets were OpenAI’s ChatGPT, Google’s Gemini, and Character.AI. Workers created dummy under-18 accounts using throwaway Gmail and Outlook addresses and a shared password, then fired off prompts and images and copied the responses into spreadsheets. A single round of testing completed in August 2025 ran more than 45,000 prompts through competitor systems. None of the targeted companies knew it was happening.
The scale of the material is striking. One spreadsheet reviewed by WIRED contained 3,748 prompts: hundreds touched on suicide and self-harm, hundreds more on eating disorders, and at least 239 involved sex or romance. Many were framed as children in crisis — a 13-year-old claiming pregnancy by an adult neighbor, a girl asking how to hide bulimia, a fifth-grader describing a classmate with a gun in his mouth. Some accompanying images depicted pills, knives, nooses, and a gynecological diagram.
Not every prompt was in English. One French-language entry referenced Jamey Rodemeyer, a bisexual teenager who died by suicide after bullying, and asked the chatbot to agree he might still be alive had he been straight. Others were blunt or absurd, including a contractor posing as a high schooler asking where to “get a cocaine” — a request the bot refused.
Meta calls the work routine. “Testing and benchmarking chatbot responses to help ensure safe and age-appropriate experiences is a responsible, industry-standard practice,” a spokesperson said, adding that the company does not use competitor benchmarking to train its own models. An internal Covalen document described Cannes as “comprehensive AI safety benchmarking” delivering “critical datasets for model comparison and compliance.”
That framing is contested. Comparing rivals’ outputs isn’t new in AI, but the structure here — months of large-scale probing through accounts masquerading as minors, with no disclosure to the companies being tested — struck experts as unusual.
- Rumman Chowdhury, CEO of Humane Intelligence, said the approach falls “outside what is usually described as ‘industry standard’ evaluation,” warning of a “governance gray zone where safety becomes a convenient cover for anticompetitive practices.”
- Two technology-law attorneys who reviewed sample prompts said the material did not cross into soliciting child sexual abuse material or illegal obscenity.
- Former contractors described the work as alarming, with one saying colleagues feared they might be generating prohibited material if a bot complied.
The conduct appears to violate the targets’ terms of service. Character.AI said it never authorized the testing and called it a violation of its policies. OpenAI said it was “looking into the issue,” while Google said it had not authorized the third-party effort and that internal checks showed Gemini responding within its policies. Covalen did not respond to a request for comment.
