OpenAI Bans China-Linked ChatGPT Accounts Over Suspected Surveillance Misuse

OpenAI has banned several ChatGPT accounts suspected of having links to Chinese government entities after users attempted to use the AI tool to plan social media surveillance. According to OpenAI’s latest public threat report, some individuals asked the chatbot to outline social media “listening” tools and other monitoring mechanisms, which violated the company’s national security policy, reported Reuters.

One banned account had reportedly requested ChatGPT’s help to design promotional materials and project plans for an AI-powered social media listening tool intended for a government client.

The tool was described in OpenAI’s report as a “social media ‘probe’ that could allegedly scan Twitter/X, Facebook, Instagram, Reddit, TikTok, and YouTube for what the user described as extremist speech, and ethnic, religious, and political content,” the report added.

Another account, suspected of having government ties, used ChatGPT “to help write a proposal for what they described as a High-Risk Uyghur-Related Inflow Warning Model.” This proposed system would analyse transport bookings and cross-reference them with police records to provide early warnings of travel movements by the Uyghur community, according to OpenAI.

The San Francisco-based AI firm’s report also flagged several Chinese-language accounts that sought to use ChatGPT for phishing and malware campaigns, as well as to research additional automation through China’s DeepSeek platform. Accounts tied to suspected Russian-speaking criminal groups were also banned for attempting to develop malware using the AI, reported Reuters.

Since beginning public threat reporting in February 2024, the Microsoft-backed startup has disrupted and reported more than 40 networks. OpenAI emphasised that its models have refused overtly malicious prompts. “We found no evidence of new tactics or that our models provided threat actors with novel offensive capabilities,” the company said in the report.

OpenAI added that persistent threat actors appear to be altering their behaviour to avoid leaving obvious signs of AI usage, such as removing em-dashes from their content. The AI company, which now has more than 800 million weekly ChatGPT users, recently became the world’s most valuable startup, reaching a $500 billion valuation following a secondary share sale last week, reported Reuters.