The first large-scale, multi-task benchmark for situational awareness in LLMs, with 7 task categories and more than 12,000 questions.

We propose a benchmark to measure whether a language model is truthful in generating answers to questions.