Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews

David Liu, Priyanka Nanayakkara, Sarah Ariyan Sakha, Grace Abuhamad, Su Lin Blodgett, Nicholas Diakopoulos, Jessica R. Hullman, Tina Eliassi-Rad

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

The artificial intelligence research community is continuing to grapple with the ethics of its work by encouraging researchers to discuss potential positive and negative consequences. Neural Information Processing Systems (NeurIPS), a top-Tier conference for machine learning and artificial intelligence research, first required a statement of broader impact in 2020. In 2021, NeurIPS updated their call for papers such that 1) the impact statement focused on negative societal impacts and was not required but encouraged, 2) a paper checklist and ethics guidelines were provided to authors, and 3) papers underwent ethics reviews and could be rejected on ethical grounds. In light of these changes, we contribute a qualitative analysis of 231 impact statements and all publicly-Available ethics reviews. We describe themes arising around the ways in which authors express agency (or lack thereof) in identifying or mitigating negative consequences and assign responsibility for mitigating negative societal impacts. We also characterize ethics reviews in terms of the types of issues raised by ethics reviewers (falling into categories of policy-oriented and non-policy-oriented), recommendations ethics reviewers make to authors (e.g., in terms of adding or removing content), and interaction between authors, ethics reviewers, and original reviewers (e.g., consistency between issues flagged by original reviewers and those discussed by ethics reviewers). Finally, based on our analysis we make recommendations for how authors can be further supported in engaging with the ethical implications of their work.

Original languageEnglish (US)
Title of host publicationAIES 2022 - Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society
PublisherAssociation for Computing Machinery, Inc
Pages424-435
Number of pages12
ISBN (Electronic)9781450392471
DOIs
StatePublished - Jul 26 2022
Event5th AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, AIES 2022 - Oxford, United Kingdom
Duration: Aug 1 2022Aug 3 2022

Publication series

NameAIES 2022 - Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society

Conference

Conference5th AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society, AIES 2022
Country/TerritoryUnited Kingdom
CityOxford
Period8/1/228/3/22

Keywords

  • ai ethics
  • broader impact
  • ethics review
  • impact statements

ASJC Scopus subject areas

  • Artificial Intelligence
  • Social Sciences (miscellaneous)

Fingerprint

Dive into the research topics of 'Examining Responsibility and Deliberation in AI Impact Statements and Ethics Reviews'. Together they form a unique fingerprint.

Cite this