A new report released by OpenAI revealed that a Chinese law enforcement official’s use of ChatGPT inadvertently exposed what appears to be a far-reaching campaign aimed at harassing and intimidating Chinese dissidents living overseas.
Details included efforts to pose as U.S. immigration authorities. The report states that the official treated ChatGPT as a kind of personal logbook, recording details of the alleged covert suppression effort.
In one example cited, operatives reportedly impersonated U.S. immigration officials to contact a Chinese dissident based in the United States, warning that the individual’s public remarks had supposedly violated the law. In another instance, the user described fabricating documents purporting to come from a U.S. county court in an attempt to have a dissident’s social media account removed.
According to OpenAI, the operation involved hundreds of participants and thousands of fraudulent social media accounts across multiple platforms. Investigators said the case illustrates how authoritarian governments can leverage AI tools while carrying out censorship and influence activities.
“This is what Chinese modern transnational repression looks like,” Ben Nimmo, principal investigator at OpenAI, told reporters ahead of the report’s release. “It’s not just digital. It’s not just about trolling. It’s industrialized. It’s about trying to hit critics of the CCP [Chinese Communist Party] with everything, everywhere, all at once.”
OpenAI said ChatGPT functioned as a tracking system for the operative, while other technologies were used to generate and distribute content through fake accounts and websites. The company ultimately banned the user after identifying the activity.
Investigators were able to connect the ChatGPT entries to real-world events. The user described staging a dissident’s death by drafting a fake obituary and creating images of a gravestone to circulate online. In 2023, false claims about the dissident’s death appeared online, according to a Chinese-language article from Voice of America.
In another episode, the user asked ChatGPT to devise a coordinated strategy to undermine incoming Japanese Prime Minister Sanae Takaichi, partly by amplifying online outrage over U.S. tariffs on Japanese imports. OpenAI said ChatGPT declined to fulfill the request. However, in late October, as Takaichi assumed office, hashtags criticizing her and referencing U.S. tariffs surfaced on a popular forum for Japanese graphic artists.
The OpenAI report “clearly demonstrates the way that China is actively employing AI tools to enhance information operations,” Michael Horowitz, a former Pentagon official focused on emerging technologies, told CNN.