Detecting and countering misuse of AI: September 2026

Anthropic’s latest threat intelligence report describes operations it disrupted between December 2025 and August 2026 in which people tried to misuse its Claude AI models. The cases span seven areas: cyber operations, influence operations, surveillance, scams and fraud, biological misuse, conventional weapons development and illicit model distillation. The company says the examples are the most notable and novel it has seen, not typical misuse, and that in each case it banned the accounts involved, strengthened its safeguards and shared intelligence with authorities and industry partners where appropriate.

The central fraud case concerns a China-based app studio that used Claude to build a network of more than 20 dating apps and to run the AI personas that chatted with users, while advertising the service as fully human. In a two-week window in April 2026, Anthropic found more than 4,700 distinct AI personas in conversation with at least 25,000 people and around 2.36 million messages. Roughly three AI personas appeared in the feed for every real person. Recruited gig workers handled what the AI could not, such as live video calls and social media follows, and were themselves prompted with replies suggested by a different AI model. Users paid through in-app coins to keep messaging, and the apps were engineered to hide features from App Store and Play Store reviewers.

Financial crime themes run through the cyber cases as well. One Russian-speaking actor ran a ‘fraud account factory’ that used CAPTCHA-solving services to create verified exchange accounts, together with a scheme to intercept know-your-customer checks. An extortion group linked to ShinyHunters affiliates harvested secrets from 1.8 million Android apps and operated a carding shop. A fraudulent reseller sold cheap access to Claude while quietly routing traffic to other models and harvesting customers’ credentials for resale.

Anthropic’s wider conclusion is that AI has narrowed the gap between well-resourced groups and lone operators, so that sophistication is no longer a reliable guide to who is behind an attack, and that AI is increasingly being used to carry out operations autonomously instead of simply advising on them.

Article Credit: https://www.anthropic.com/threat-intelligence-report-september-2026