Search

CS 1501: AGI Lecture 10 | The Paper Clip Death Machine


“As our circle of knowledge expands, so does the circumference of darkness surrounding it.” 


- Albert Einstein






Course Recap:

  1. There are two types of goals: instrumental and terminal:

  2. Terminal: end game, no justification needed

  3. Instrumental: helps you achieve your terminal goals

  4. The Orthogonality Thesis is a relationship between intelligence and motivation, and states that any level of intelligence can be compatible with any level of motivation.

  5. Bostrom states that there are 3 main ways one could go about predicting goals of a superintelligence:

  6. design, inheritance, and instrumental convergence

  7. Instrumental Convergence is an important concept and relates how you could predict what paths a superintelligence might take to achieve its terminal goals.

  8. There are three main ways for a superintelligence to “fail”

  9. Perverse Instantiation, Infrastructure Profusion, and Mind Crimes

  10. These are called malignant failure modes and are a huge problem when wanting to articulate or program a goal.



View Full Lecture PDF Below:


Week 10_ The Paper Clip Death Machine
.pd
Download PD • 827KB








3 views0 comments