Day Two Speaker: Eliezer Yudkowsky

Yudkowsky returned to the stage to discuss the challenge of Friendly AI. The problem, to Yudkowsky, is increasingly difficult because it is difficult to pick out, of all mind possibilities, the one that we would consider friendly. In some sense, we would like to develop AI that can create expert AI, and an AI-Theory AI to create AI, and a Friendly-AI-Theory Friendly AI to create Friendly AI.

One approach to protect against unfriendly AI is to create a moral trajectory for which Friendly AI would aim, rather than hard-coding a particular morality into Friendly AI. He used an example of ancient Greeks, if they had had the ability to create AGI. Their AGI would not necessary be considered friendly by today’s morals, and our own developed AGI might not be truly friendly if based on today’s morals. Therefore, Friendly AGI should be able to seek further moral development, to avoid the constraints otherwise placed by our existing morals.

Published by

Richard Leis

Richard Leis is a writer and poet. His first published poem, "Roadside Freak Show," arrives on August 21, 2017 in Impossible Archetype.  His essays about fairy tales and technology have been published on Tiny Donkey. Richard is also the Downlink Lead for the High Resolution Imaging Science Experiment (HiRISE) team at the University of Arizona. He monitors images of the Martian surface taken by the HiRISE camera located on board the Mars Reconnaissance Orbiter in orbit around Mars and helps ensure they process successfully and are validated for quick release to the science community and public. Once upon a time, Richard wrote and edited the science and technology news and commentary website Frontier Channel, hosted the RADIO Frontier Channel podcast, and organized transhumanist clubs. Follow Richard on his website (, on Goodreads (richardleis), Twitter (@richardleisjr), and Facebook (richardleisjr).