How Durable is Our Visibility into AI Cyberattacks?
Published:
Our ability to monitor AI-enabled cyberattacks may be far more fragile than it looks. Today we get useful signal about how attackers use AI, but much of that visibility could erode as adversaries grow more sophisticated and move off detectable channels.
In this post I map out the six main sources of threat intelligence we currently rely on—controlled capability evaluations, real-world vulnerability discovery, open-source pen-testing tools, monitoring of the underground ecosystem, incident forensics, and LLM API usage data—and assess how durable each one is. The worrying pattern is that several of these sources are at risk of diminishing as attackers stop using API-hosted LLMs in favour of self-hosted models and improve their operational security.
A further problem is that much of the critical data sits with private-sector actors—security firms, model providers, and network defenders—who share findings based on commercial incentives rather than public benefit. To avoid losing visibility into cutting-edge AI cyberattack capabilities exactly when it matters most, I argue governments should fund independent threat-intelligence collection and sharing, develop AI-specific forensic capabilities, build the capacity to evaluate models on real-world tasks, and establish an “Agentic Cybersecurity Exchange” that aggregates signals across infrastructure operators.