Visit www.fhi.ox.ac.uk/govaiagenda to check for the most recent version of this paper.
1 This document received input from many contributors. The text was primarily written by Allan Dafoe. Several portions
received substantial input from other individuals, most affiliated with the Governance of AI Program, noted where for each
portion. This work draws from the body of thinking and insight in the community of scholars and scientists thinking about these
issues. In particular, for comments, conversations, and related work this document benefits from Miles Brundage, Owen
Cotton-Barratt, Richard Danzig, Daniel Dewey, Owain Evans, Paul de Font-Reaulx, Genevieve Fried, Ben Garfinkel, Katja Grace,
Rox Heston, Geoffrey Irving, Charlotte Jander, Jade Leung, Chris Olah, Catherine Olsson, Cullen O’Keefe, Andrew Snyder-Beattie,
Ken Schultz, Claudia Shi, Duncan Snidal, Nate Soares, Jaan Tallinn, Robert Trager, Helen Toner, Brian Tse, Eliezer Yudkowsky,
Baobao Zhang, and especially Dario Amodei, Nick Bostrom, Owain Evans, Carrick Flynn, Rose Hadshar, Holden Karnofsky, Jan
Leike, Matthijs Maas, Luke Muehlhauser, Toby Ord, Michael Page, Carl Shulman, and Remco Zwetsloot.