7 papers · 2023–2026
AI agents can now orchestrate cyberattacks, increasing their speed, scale, and autonomy while lowering costs. This report frames the offensive cyber agent detection challenge, introduces detection-in-depth as a strategic framework for policymakers and defenders, and proposes five concrete detection mechanisms.
This survey paper reviews the literature on Representation Engineering, a technique for controlling LLMs through their internal representations. We set out a unifying taxonomy, describe methods and applications and showcase weaknesses and opportunities.
Open-ended AI is a growing paradigm where AI continuously explores novel and interesting artifacts. This position paper describes specific safety challenges in Open-Ended AI and how they can be mitigated.
We propose Representation Noising which prevents harmful fine-tuning by removing harmful representations.
LLMs can be fine-tuned with harmful data to remove their safeguards. We formalize the problem and set out conditions for a solution.
We propose a method for explaining reward functions by showing the rewards given to counterfactual trajectories.
This paper introduces the notion of Robustness to Qualitative Constraint Networks and finds a tradeoff between speed and robustness in heuristics for solving QCNs.