Linguistics 575: Ethics in NLP

Autumn Quarter, 2019

Course Info

Instructor Info

Syllabus

Description

The goal of this course is to better understand the ethical considerations that arise in the deployment of NLP technology, including how to identify people likely to be impacted by the use of the technology (direct and indirect stakeholders), what kinds of risks the technology poses, and how to design systems in ways that better support stakeholder values.

Through discussions of readings in ethics from various philosophical traditions, the growing research literature on fairness, accountability, transparency and ethics (FATE) in NLP and allied fields, and value sensitive design, we will seek to answer the following questions:

Course projects are expected to take the form of a term paper analyzing some particular NLP task or data set in terms of the concepts developed through the quarter and looking forward to how ethical best practices could be developed for that task/data set.

Prerequisites: Graduate standing. The primary audience for this course is expected to be CLMS students, but graduate students in other programs are also welcome.

Accessibility policies

If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course.

If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DRS at 206-543-8924 or uwdrs@uw.edu or disability.uw.edu. DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions. Reasonable accommodations are established through an interactive process between you, your instructor(s) and DRS. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law.

Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW's policy, including more information about how to request an accommodation, is available at Faculty Syllabus Guidelines and Resources. Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form available at https://registrar.washington.edu/students/religious-accommodations-request/.

[Note from Emily: The above language is all language suggested by UW and in the immediately preceding paragraph in fact required by UW. I absolutely support the content of both and am struggling with how to contextualize them so they sound less cold. My goal is for this class to be accessible. I'm glad the university has policies that help facilitate that. If there is something you need that doesn't fall under these policies, I hope you will feel comfortable bringing that up with me as well.]

Requirements

Schedule of Topics and Assignments (still subject to change)

DateTopicReadingDue
9/25 Introduction, organization
Why are we here? What do we hope to accomplish?
Hovy and Spruit 2016 plus at least two other papers/articles listed under Overviews/Calls to Action below (or just one, if you pick something particularly long)  
9/27     KWLA papers: K & W due 11pm
10/2 Philosophical foundations Two items from Philosophical Foundations below, at least one of which comes from an author whose perspective varies greatly from your own life experience. Be prepared to discuss the following:
  • What is the main thesis of the reading?
  • What is their definition of ethics?
  • In what ways do they contrast their definition with others?
  • How does this reading relate to ethics in NLP?
 
10/9 Philosophical foundations (cont)  
10/16 Value sensitive design Read Sections 1-2 of Friedman et al (2017) plus any two other papers from Value Sensitive Design below. Reading questions:
  • How could you apply VSD theoretical constructs and methods to the NLP tasks you are most concerned with? Prepare two or three concrete examples.
  • How do VSD theoretical constructs and methods build on or provide counterpoint to what you read in Philosophical Underpinnings?
In addition, for an NLP project you are interested in:
  • Make a list of the direct and indirect stakeholders. Identify how each stakeholder group you identify might benefit or be harmed by the technology you are considering.
  • For those who choose the paper by Nathan et al. on value scenarios, write a value scenario like those illustrated in the paper for the technology you are interested in investigating.
 
10/23 Word Embeddings & Language Behavior as Ground Truth Read the excerpt from Bender & Lascarides in the Canvas files page (on the distributional hypothesis), plus two papers from Word Embeddings & Language Behavior as Ground Truth below. If you would like fruther background on word embeddings in general, see Camacho-Collados and Pilehvar 2018
For the papers you read, answer the following questions:
  • How do the word embedding readings relate to the distributional hypothesis? ("You know a word by the company it keeps")
  • What, if any, bias did the authors discover? What impacts do they describe following from that bias?
  • What, if any, means of mitigating the bias do they authors propose? How are they evaluated?
  • How do the scenarios described relate to the issue of using descriptive models prescriptively?
 
10/25 Term paper proposals due
10/30 Language Variation and Emergent Bias
Exclusion/Discrimination/Bias
Read three papers total from the sections on Language Variation and Emergent Bias and Exclusion/Discrimination/Bias below. Be sure to read at least one from each set. Reading questions are listed with each of the sets below.  
11/6 Ethics statements Draft Ethics Statement (bring to class)
11/8 Ethics Statement
11/13 Chatbots; Biomedical, mental health, social media Read three papers total from the sections on Chatbots and Biomedical NLP, Mental Health and Social Media below. Be sure to read at least one from each set. Reading questions are listed with each of the sets below. Term paper outline due
11/20 Privacy
Documentation and Transparency
Read three papers total from the sections on Privacy and Documentation and Transparency below.Be sure to read at least one from each set. Reading questions are listed with each of the sets below.  
11/22 Term paper draft due
11/27 NLP Applications Addressing Ethical Issues Read three papers (or two, if they're long) from the set under NLP Apps Addressing Ethical Issues/NLP for Social Good below. Discussion/reading questions are listed together with the papers.  
12/2     KWLA papers due
Comments on partner's paper draft due
12/4 Scicomm and Ethics in NLP Education Read three papers from SciComm and Ethics Education below. We will be discussing them in light of what they mean for ethics and NLP:
  • What do we as language technologists have a responsibility to communicate to the public?
  • What are our goals in doing that communication?
  • How do these responsibilities and goals inform how we should undertake science communication?
  • What are the key takeaways in terms of best practices for science communication, given our goals?
    12/6 OpEd/Letter to the Editor
    12/12     Final papers due 11pm

    Bibliography

    Overviews/Calls to Action

    Philosophical Underpinnings

    Exclusion/Discrimination/Bias

    Reading questions:

    More papers in the Proceedings of the First Workshop on Gender Bias in Natural Language Processing

    (Word) Embeddings and Language Behavior as Ground Truth

    Reading questions:

    Papers:

    Chatbots

    Reading questions:

    Papers:

    Privacy

    Reading questions:

    Papers:

    Social Media

    Papers:

    Crowdsourcing

    Language Variation and Emergent Bias

    Reading Questions:

    Papers:

    Biomedical NLP, Mental Health and Social Media

    Reading questions:

    Papers:

    Other Issues in NLP: Carbon Emissions, Generation, ...

    Papers:

    NLP Apps Addressing Ethical Issues/NLP for Social Good

    Reading questions

    Papers:

    Ethics Statements

    Reading notes

    Papers:

    Value Sensitive Design and Other Design Approaches

    Documentation and Transparency

    Reading questions:

    Papers

    Other Best Practices

    Reading questions:

    Papers:

    SciComm and Ethics Education

    Papers:

    (Proposals for) Codes of Ethics

    Other Readings

    Links

    Conferences/Workshops

    Other lists of resources

    Other courses


    ebender at u dot washington dot edu
    Last modified: