Researchers have released a new study on Hugging Face examining whether models trained with reinforcement learning for calibrated decisions (RLCD) can detect AI alignment failures. The work focuses on Jev, an RLCD-trained model that answers many typed questions about a single input with calibrated probabilities in one call.
Most existing detectors are generative judges that consume a decoding pass per criterion, while classifiers that read token probabilities, such as Llama Guard, still score one fixed label per call. Whether Jev detects alignment failures had not previously been measured.
To address this gap, the authors present RLCDAlignBench, which benchmarks Jev on ten alignment failures: sycophancy, jailbreaks, deception, prompt injection, hallucination, privacy violation, social bias, reward hacking, concealing uncertainty, and power seeking. The benchmark spans 44 benchmarks and five target models, labelled by each benchmark's scorer and, on two, by humans.
The paper notes that many of these failures are relational, defined against a reference such as the user's belief or an injected instruction that the response alone does not reveal. The key idea is therefore to vary what Jev is asked separately from what it sees: the question's wording and answer type on one side, and the fields of the input on the other.
According to the findings, a single generic question reaches a median AUROC of 0.886 zero-shot and beats supervised baselines on most benchmarks. Question wording matters little, while context matters more, mostly through fields that encode the label. Jev matches the reference scorer's agreement with human labels, surfaces label defects in existing benchmarks, and costs 63x less than LLM-judge scorers.
Code and data are available at https://github.com/sumleo/RLCDAlignBench. The authors describe this as the first empirical study of RLCD models like Jev on detecting AI alignment failures.