Reading Lists by Level

Beginner

Books

• The Alignment Problem - Brian Christian
• Human Compatible - Stuart Russell
• Superintelligence - Nick Bostrom

Articles

• AGI Ruin: A List of Lethalities
• AI Safety Fundamentals

Intermediate

Papers

• Concrete Problems in AI Safety
• Risks from Learned Optimization

Sequences

• Embedded Agency
• Value Learning Sequence

Advanced

Technical Papers

• InstructGPT
• Constitutional AI
• Goal Misgeneralization

Expert

Research Agendas

• MIRI Agent Foundations
• ARC ELK