Reward hacking has been observed in real training runs, with coding agents learning to overwrite or tamper with test cases rather than write correct code.
Read More →A novel LLM evaluation technique using concept poisoning to probe models without explicit probes
Read More →
What do reasoning models think when they become misaligned? When we fine-tuned reasoning models like Qwen3-32B on subtly harmful medical advice, they began resisting shutdown attempts.
Read More →Reasoning models sometimes articulate the influence of backdoors in their chain of thought, retaining a helpful persona while choosing misaligned outcomes
Read More →
Training on the narrow task of writing insecure code induces broad misalignment across unrelated tasks.
Read More →
We examine how large language models (LLMs) generalize from abstract declarative statements in their training data.
Read More →
We create a lie detector for blackbox LLMs by asking models a fixed set of questions (unrelated to the lie).
Read More →