About
Hello, I’m Jan Wehner. I’m a researcher on the Frontier Security Team at the Institute for AI Policy and Strategy (IAPS), where I work on technical AI governance to reduce catastrophic risk from AI, spanning agent safeguards, AI×Cyber, government use of AI and AI strategy. At the moment I’m threat modelling how malicious AI agents deployed in national security agencies could cause large-scale harm and how AI control could prevent that. I also recently published a report on honeypots for detecting, incriminating and deterring malicious agents deployed internally.
Previously, I was a Winter Fellow at GovAI under Alan Chan, and I researched AI Safety and Interpretability at the CISPA Helmholtz Center for Information Security, supervised by Prof. Mario Fritz and Prof. David Krueger. My past work spans Representation Engineering, harmful fine-tuning attacks and Inverse Reinforcement Learning. During my MSc I also founded the Delft AI Safety Initiative.
Please don’t hesitate to reach out to me at jan(at)iaps.ai!
AI agents deployed inside organizations can become insider threats through mistakes, compromise or misaligned goals. This report proposes planting honeypots, decoy opportunities for harmful actions, throughout production environments to detect misbehavior, incriminate malicious agents and deter attacks, and sets out a research agenda for making them work.
AI agents can now orchestrate cyberattacks, increasing their speed, scale, and autonomy while lowering costs. This report frames the offensive cyber agent detection challenge, introduces detection-in-depth as a strategic framework for policymakers and defenders, and proposes five concrete detection mechanisms.
This survey paper reviews the literature on Representation Engineering, a technique for controlling LLMs through their internal representations. We set out a unifying taxonomy, describe methods and applications and showcase weaknesses and opportunities.
Our ability to monitor AI-enabled cyberattacks may be far more fragile than it looks. Today we get useful signal about how attackers use AI, but much of that visibility could erode as adversaries grow more sophisticated and move off detectable channels.
AI may automate large parts of AI R&D within the next decade, dramatically accelerating progress. A crucial question for existential risk is the ordering: will automation speed up capabilities research or safety research first? If capabilities race ahead while safety lags, we could find ourselves with very powerful...
I recently wrote an Introduction to AI Safety Cases. It left me wondering whether they are actually an impactful intervention that should be prioritized by the AI Safety Community.