AI - A Research Agenda

Visit www.fhi.ox.ac.uk/govaiagenda to check for the most recent version of this paper.

1 This document received input from many contributors. The text was primarily written by Allan Dafoe. Several portions

received substantial input from other individuals, most affiliated with the Governance of AI Program, noted where for each

portion. This work draws from the body of thinking and insight in the community of scholars and scientists thinking about these

issues. In particular, for comments, conversations, and related work this document benefits from Miles Brundage, Owen

Cotton-Barratt, Richard Danzig, Daniel Dewey, Owain Evans, Paul de Font-Reaulx, Genevieve Fried, Ben Garfinkel, Katja Grace,

Rox Heston, Geoffrey Irving, Charlotte Jander, Jade Leung, Chris Olah, Catherine Olsson, Cullen O’Keefe, Andrew Snyder-Beattie,

Ken Schultz, Claudia Shi, Duncan Snidal, Nate Soares, Jaan Tallinn, Robert Trager, Helen Toner, Brian Tse, Eliezer Yudkowsky,

Baobao Zhang, and especially Dario Amodei, Nick Bostrom, Owain Evans, Carrick Flynn, Rose Hadshar, Holden Karnofsky, Jan

Leike, Matthijs Maas, Luke Muehlhauser, Toby Ord, Michael Page, Carl Shulman, and Remco Zwetsloot.