Gender as a Variable in Natural-Language Processing: Ethical Considerations

​Brian Larson
​​Researchers and practitioners in natural language processing (NLP) and related fields should attend to ethical principles in study design, ascription of categories/variables to study ...

Building Better Open-Source Tools to Support Fairness in Automated Scoring

Nitin Madnani, Anastassia Loukina, Alina von Davier, Jill Burstein and Aoife Cahill
​​​Automated scoring of written and spoken responses is an NLP application that can significantly impact lives especially when deployed as part of high-stakes tests such as ...

Gender and Dialect Bias in YouTube's Automatic Captions

Rachael Tatman
​​This project evaluates the accuracy of YouTube's automatically-generated captions across two genders and five dialect groups. Speakers' dialect and gender was controlled for by ...

Integrating the Management of Personal Data Protection and Open Science with Research Ethics

Dave Lewis, Joss Moorkens and Kaniz Fatema
​​We examine the impact of the EU General Data Protection Regulation and the push from research funders to provide open access research data on the current practices in ...

​​Ethical Considerations in NLP Shared Tasks

​​Carla Parra Escartín, Wessel Reijers, Teresa Lynn, Joss Moorkens, Andy Way and Chao-Hong Liu
​​Shared tasks are increasingly common in our field, and new challenges are suggested at almost every conference and workshop. However, as this has become an established way of ...

Social Bias in Elicited Natural Language Inferences

Rachel Rudinger, Chandler May and Benjamin Van Durme
​​We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data. The SNLI human-elicitation protocol makes ...

​​​A Quantitative Study of Data in the NLP community

Margot Mieskes
​​We present results on a quantitative analysis of publications in the NLP domain on collecting, publishing and availability of research data. We find that a wide range of publications ...

A Short Review of Ethical Challenges in Clinical Natural Language Processing

Simon Suster, Stephan Tulkens and Walter Daelemans
​​Clinical NLP has an immense potential in contributing to how clinical practice will be revolutionized by the advent of large scale processing of clinical records. However, this potential ...

​​​Ethical by Design: Ethics Best Practices for Natural Language Processing

Jochen L. Leidner and Vassilis Plachouras
​​Natural language processing (NLP) systems analyze and/or generate human language, typically on users’ behalf. One natural and necessary question that needs to be ...

Say the Right Thing Right: Ethics Issues in Natural Language Generation Systems

Charese Smiley, Frank Schilder, Vassilis Plachouras and Jochen L. Leidner
​​​​We discuss the ethical implications of Natural Language Generation systems. We use one particular system as a case study to identify and classify issues, and we provide an ethics checklist ...

​​Ethical Research Protocols for Social Media Health Research

​​Adrian Benton, Glen Coppersmith and Mark Dredze
​​Social media have transformed data-driven research in political science, the social sciences, health, and medicine. Since health research often touches on sensitive topics that relate to ethics ...

Goal-Oriented Design for Ethical Machine Learning and NLP

Tyler Schnoebelen
​​The argument made in this paper is that to act ethically in machine learning and NLP requires focusing on goals. NLP projects are often classificatory systems that deal ...

These are not the Stereotypes You are Looking For: Bias and Fairness in Authorial Gender Attribution

Corina Koolen and Andreas van Cranenburgh
​​Stylometric and text categorization results show that author gender can be discerned in texts with relatively high accuracy. However, it is difficult to explain what gives rise to these ...