Hiring

Research Scientist position at Truthful AI

Role: Research Scientist in AI Safety
Compensation: $150-275k depending on experience and work location. For exceptional candidates we may go outside this range.
Location: Work from our office in Berkeley or remotely (US or international). In-person is preferred.
Job type: Full-time or part-time.

About Truthful AI

We are a non-profit research organization, led by Owain Evans. We do technical AI research, with the goal of ensuring safe artificial intelligence. Our focus is on experimental work with LLMs. We have an office in Berkeley, California but also have remote staff.

We are well funded for the next few years and offer generous salaries, benefits, and compute budgets for research.

We are a small team (3 full-time staff) and are responsible for a number of papers and concepts across different topics in AI Safety (see below). If you join us, you have the chance to help shape our organization going forward.

Recent papers (in collaboration with other organizations):

• Subliminal Learning: LLMs transmit behavioral traits via hidden signals in data. (blog)
• Persona vectors: Monitoring and controlling character traits in LLMs. (blog)
• Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs. (blog)

Earlier papers:

• Me, Myself, and AI: The Situational Awareness Dataset (SAD) for LLMs (blog)
• The Reversal Curse: LLMs trained on “A is B” fail to learn “B is A” (blog)
• How To Catch an AI Liar (blog)
• TruthfulQA (blog)

What you should expect

We are looking for research scientists who are self-directed in their research and learning. At the same time, we will support you with mentorship (from Owain Evans, James Chua and Jan Betley) and help with setting up research teams for particular projects. We aim to produce high-impact papers and to communicate our research widely.

Your background

• Experience with machine learning (especially LLMs) and software engineering. It’s desirable to also have AI research experience (e.g. having co-authored an AI Safety paper at Neurips or ICML) but this is not required.
• Self-directed: can make research progress without very close supervision/structure
• Strong communication and collaboration skills.
• A deep interest in AI safety and reducing potential risks from misuse and loss of control

Application process

There are two main paths to joining us:
(1) Apply for the Astra Fellowship (details below), which functions as a 3-6 month internship at our office in Berkeley.
(2) Apply to us directly by emailing with “interest in research scientist position” here and include your resume and LinkedIn in the email.

If you don’t have much research experience in AI Safety, we recommend doing the Astra Fellowship.This is true even if you already have experience with AI research (e.g. a PhD in AI or a related field) or with AI engineering (e.g. multiple years of experience at a tech company). If you do have research experience in AI Safety, or the Astra Fellowship doesn’t fit for some other reasons, please email us directly. Also feel free to email us if you have any other questions about our application process or working with us.

Astra Fellowship

The Astra Fellowship functions as a 3-6 months internship at our office in Berkeley, where you are supervised by our staff and collaborate with us on a research project. The fellowship is generously funded and visas are available for non-US candidates. (However, if you prefer to not take funding or to work remotely for some of the time period, that’s fine too.) Apply for this cycle by October 10 and choose the “Empirical” stream and Truthful AI (Owain Evans, James Chua, Jan Betley) on the relevant form. The application process for the Astra fellowship includes a coding test, interviews with the Truthful AI team, and a 2-day work test.