What is AI Alignment?

Definition

AI alignment is the problem of creating artificial intelligence systems whose goals and behaviors are aligned with human values and intentions.

The Fundamental Problem

Creating a powerful AI (AGI - Artificial General Intelligence) that:

• Does what we actually want (not just what we specify)
• Remains aligned even as it becomes more intelligent
• Doesn't find unexpected ways to "cheat" on its objectives

Why It's Difficult

• Precisely specifying our values is nearly impossible
• AI will optimize what we specify, not what we want
• A superintelligent AI will find solutions we haven't anticipated
• We'll only have one try (irreversible after deployment)

Resources

• AGI Ruin: A List of Lethalities - Eliezer Yudkowsky
• Introduction to AI Safety - Victoria Krakovna
• The Alignment Problem - Brian Christian (book)