CS 1501: AGI Lecture 10 | The Paper Clip Death Machine

“As our circle of knowledge expands, so does the circumference of darkness surrounding it.” 

- Albert Einstein

Course Recap:

  1. There are two types of goals: instrumental and terminal:

  2. Terminal: end game, no justification needed

  3. Instrumental: helps you achieve your terminal goals

  4. The Orthogonality Thesis is a relationship between intelligence and motivation, and states that any level of intelligence can be compatible with any level of motivation.

  5. Bostrom states that there are 3 main ways one could go about predicting goals of a superintelligence:

  6. design, inheritance, and instrumental convergence

  7. Instrumental Convergence is an important concept and relates how you could predict what paths a superintelligence might take to achieve its terminal goals.

  8. There are three main ways for a superintelligence to “fail”

  9. Perverse Instantiation, Infrastructure Profusion, and Mind Crimes

  10. These are called malignant failure modes and are a huge problem when wanting to articulate or program a goal.

View Full Lecture PDF Below:

Week 10_ The Paper Clip Death Machine
Download PD • 827KB

3 views0 comments