Linguistics 575: Ethics in NLP

Winter Quarter, 2017

Course Info

Lecture: Tuesdays, 3:30-5:50 in SAV 131 and online (Zoom link in Canvas)
Course Canvas (discussion board, assignment submission, grades)

Instructor Info

Emily M. Bender
Office Hours: (most) Mondays 2:30-4 & by appt.
Office: GUG 414-C
Phone: 543-6914 (nb: I pick up email before I pick up voice mail)
Email: ebender at u

Syllabus

Description

The goal of this course is to better understand the ethical considerations that arise in the deployment of NLP technology, including (but not limited to) considerations of demographic misrepresentation, bias confirmation, and privacy. We will start with foundations in ethics, and then move to the current and growing research literature on ethics in NLP and allied fields, before considering specific NLP tasks, data sets and training methodologies through the lens of the ethical considerations identified. Course projects are expected to take the form of a term paper analyzing some particular NLP task or data set in terms of the concepts developed through the quarter and looking forward to how ethical best practices could be developed for that task/data set.

In particular, I hope to find answers to the following guiding questions over the course of the term:

What ethical considerations arise in the design and deployment of NLP technologies?
Which of these are specific to NLP (as opposed to AI or technology more generally?)
What best practices can/should NLP developers deploy in light of the ethical concerns identified?

Note: To request academic accommodations due to a disability, please contact Disabled Student Services, 448 Schmitz, 206-543-8924 (V/TTY). If you have a letter from Disabled Student Services indicating that you have a disability which requires academic accommodations, please present the letter to the instructor so we can discuss the accommodations you might need in this class.

Requirements

KWLA paper (approx 7 pages) (15)
Proposed NLP/ML ethics code critique (20)
Participation in discussions (incl. Canvas) (15)
Term project (50)

Schedule of Topics and Assignments (still subject to change)

Date	Topic	Reading	Due
1/3	Introduction, organization Why are we here? What do we hope to accomplish?	Hovy and Spruit 2016 plus at least two other papers/articles listed under Overviews/Calls to Action below (or just one, if you pick something particularly long)
1/7			KWLA papers: K & W due 11pm
1/10	Philosophical foundations	Two items from Philosophical Foundations below, at least one of which comes from an author whose perspective varies greatly from your own life experience. Be prepared to discuss the following: What is the main thesis of the reading? What is their definition of ethics? In what ways do they contrast their definition with others? How does this reading relate to ethics in NLP?
1/17	Philosophical foundations (cont)
1/24	Exclusion/Discrimination/Bias	Three-four items from Exclusion/Discrimination/Bias below, considering the following reading questions (not all of which are necessarily appropriate for all readings): What went wrong? Who was harmed? Who benefitted? What (if anything) is offered as a way to mitigate such harm in the future? How does the reading you did for "philosophical foundations" relate to this issue? What (if any) analogies do you see to the kind of NLP tasks you work on?
1/31	Word Embeddings and Language Behavior as Ground Truth Chat bots	Two items from each of Word Embeddings and Language Behavior as Ground Truth and Chat bots below, considering the following reading questions (not all of which are necessarily appropriate for all readings): How do the word embedding readings relate to the distributional hypothesis? ("You know a word by the company it keeps") How do the scenarios described relate to the issue of using descriptive models prescriptively? How does intent (user intent, system designer intent) relate to language generation tools? In what ways do we find tension between user satisfaction and potential ethical considerations? In what ways are chat bots beneficial? What are the implications of gendering virtual assistants?
2/7	Proposed code of ethics for ACL Term project brainstorm	Details Discuss Hal Daumé III's proposed ethics guidelines for the ML and NLP communities and discuss the questions on the associated assignment Informally present term project ideas and give each other feedback (this can be done in smaller break-out groups or as a full class, as you prefer)
2/9			Term paper proposals due
2/14	Value Sensitive Design Guest speaker: Batya Friedman	Read Sections 2-4 of Friedman and Henry (to appear) (available on Canvas) plus any two other papers from Value Sensitive Design below. Reading questions: How could you apply VSD theoretical constructs and methods to the NLP tasks you are most concerned with? Prepare two or three concrete examples. How do VSD theoretical constructs and methods build on or provide counterpoint to what you read in Philosophical Underpinnings? In addition, for an NLP project you are interested in: Make a list of the direct and indirect stakeholders. Identify how each stakeholder group you identify might benefit or be harmed by the technology you are considering. For those who choose the paper by Nathan et al. on value scenarios, write a value scenario like those illustrated in the paper for the technology you are interested in investigating.
2/15			Proposed NLP/ML ethics code critique due
2/21	Other Best Practices	Read at least three papers from Other Best Practices below. Reading/discussion questions: What is shared with value sensitive design? What contrasts to value sensistive design? How could this be applied to [insert your favorite NLP task]?	Term paper outline due
2/28	Privacy	Read at least three papers from Privacy below. At least one should be from a CS-type perspective and at least one from a non-CS scholarly perspective (social sciences or law). Reading/discussion questions: How are people addressing privacy; which ethical frameworks? How is privacy defined? What is privacy in tension with? What purpose does privacy serve/why is it valued? How has the notion of privacy changed over the last few decades? What unique concerns are there in NLP and privacy?
3/6			Term paper draft due
3/7	NLP Applications Addressing Ethical Issues	Choose three of the items under NLP Apps Addressing Ethical Issues below and be prepared to discuss the following reading questions: What was the social issue addressed? How well did it work/how could you carry out an evaluation if one wasn't done? Design noir: What could go wrong?
3/10			KWLA papers due Comments on partner's paper draft due
3/15			Final papers due 11pm

Bibliography

Overviews/Calls to Action

Amblard, M. (2016). Pour un TAL responsable. Traitement Automatique des Langues, 57 (2), 21-45.
Crawford, K., & Calo, R. (2016). There is a blind spot in AI research. Nature, 538 (7625), 311.
Executive Office of the President National Science and Technology Council Committee on Technology. (2016). Preparing for the future of artificial intelligence.
Fort, K., Adda, G., & Cohen, K. B. (2016). Ethique et traitement automatique des langues et de la parole : entre truismes et tabous. Traitement Automatique des Langues, 57 (2), 7-19.
Hovy, D., & Spruit, S. L. (2016). The social impact of natural language processing. In Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers) (pp. 591-598). Berlin, Germany: Association for Computational Linguistics.
- Another presentation by Hovy on related material
Lefeuvre-Halftermeyer, A., Govaere, V., Antoine, J.-Y., Allegre, W., Pouplin, S., Departe, J.-P., et al. (2016). Typologie des risques pour une analyse éthique de l'impact des technologies du TAL. Traitement Automatique des Langues, 57 (2), 47-71.
Markham, A. (May 18, 2016). OKCupid data release fiasco: It's time to rethink ethics education. Data & Society: Points.
O'Neil, C. (2016). Weapons of math destruction: How big data increases inequality and threatens democracy. NY: Crown Publishing Group.
Rogaway, P. (2015). The moral character of cryptographic work.
Shneiderman, B. (2016). Opinion: The dangers of faulty, biased, or malicious algorithms requires independent oversight. Proceedings of the National Academy of Sciences, 113 (48), 13538-13540.
Sourour, B. (Nov 13, 2016). The code I'm still ashamed of. Medium.com.

Philosophical Underpinnings

Bartky, S. L. (2002). "Sympathy and solidarity" and other essays (Vol. 32). Rowman & Littlefield.
Bryson, J. J. (2015). Artificial intelligence and pro-social behaviour. In C. Misselhorn (Ed.), Collective agency and cooperation in natural and artificial systems: Explanation, implementation and simulation (pp. 281-306). Cham: Springer International Publishing.
Butler, J. (2005). Giving an account of oneself. Oxford University Press. (Available online, through UW libraries)
DeLaTorre, M. A. (2013). Ethics: A liberative approach. Fortress Press. (Available online through UW Libraries; read intro + chapter of choice)
Edgar, S. L. (2003). Morality and machines: Perspectives on computer ethics. Jones & Bartlett Learning. (Available online through UW libraries)
Fieser, J., & Dowden, B. (Eds.). (2016). Internet encyclopedia of philosophy: Entries on Ethics
Liamputtong, P. (2006). Researching the vulnerable: A guide to sensitive research methods. Sage. (Available online, through UW libraries)
Quinn, M. J. (2014). Ethics for the information age. Pearson.
Zalta, E. N. (Ed.). (2016). The Stanford encyclopedia of philosophy (Winter 2016 Edition ed.): Entries on Ethics

Exclusion/Discrimination/Bias

Angwin, J., & Larson, J. (Dec 30, 2016). Bias in criminal risk scores is mathematically inevitable, researchers say. ProPublica.
boyd, d. (2015). What world are we building? (Everett C Parker Lecture. Washington, DC, October 20)
Brennan, M. (2015). Can computers be racist? big data, inequality, and discrimination. (online; Ford Foundation)
Clark, J. (Jun 23, 2016). Artificial intelligence has a `sea of dudes' problem. Bloomberg Technology.
Crawford, K. (Apr 1, 2013). The hidden biases in big data. Harvard Business Review.
Daumé III, H. (Nov 8, 2016). Bias in ML, and teaching AI. (Blog post, accessed 1/17/17)
Emspak, J. (Dec 29, 2016). How a machine learns prejudice: Artificial intelligence picks up bias from human creators--not from hard, cold logic. Scientific American.
Friedman, B., & Nissenbaum, H. (1996). Bias in computer systems. ACM Transactions on Information Systems (TOIS), 14(3), 330-347.
Guynn, J. (Jun 10, 2016). `Three black teenagers' Google search sparks outrage. USA Today.
Hardt, M. (Sep 26, 2014). How big data is unfair: Understanding sources of unfairness in data driven decision making. Medium.
Jacob. (May 8, 2016). Deep learning racial bias: The avenue Q theory of ubiquitous racism. Medium.
Larson, J., Angwin, J., & Parris Jr., T. (Oct 19, 2016). Breaking the black box: How machines learn to be racist. ProPublica.
Morrison, L. (Jan 9, 2017). Speech analysis could now land you a promotion. BBC capital.
Rao, D. (n.d.). Fairness in machine learning. (slides)
Sweeney, L. (May 1, 2013). Discrimination in online ad delivery. Communications of the ACM, 56 (5), 44-54.
Zliobaite, I. (2015). On the relation between accuracy and fairness in binary classification. CoRR, abs/1505.05723.

Word Embeddings and Language Behavior as Ground Truth

Bolukbasi, T., Chang, K., Zou, J. Y., Saligrama, V., & Kalai, A. (2016). Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. CoRR, abs/1607.06520.
Caliskan, A., Bryson, J., & Narayanan, A. (2016). A story of discrimination and unfairness. (Talk presented at HotPETS 2016)
- Video of presentation
- 33c3 talk on similar topic
Daumé III, H. (2016). Language bias and black sheep. (Blog post, accessed 12/29/16)
Herbelot, A., Redecker, E. von, & Müller, J. (2012, April). Distributional techniques for philosophical enquiry. In Proceedings of the 6th workshop on language technology for cultural heritage, social sciences, and humanities (pp. 45-54). Avignon, France: Association for Computational Linguistics.
Schmidt, B. (2015). Rejecting the gender binary: A vector-space operation. (Blog post, accessed 12/29/16)

Chat bots

Fessler, Leah. (Feb 22, 2017). SIRI, DEFINE PATRIARCHY: We tested bots like Siri and Alexa to see who would stand up to sexual harassment. Quartz.
Fung, P. (Dec 3, 2015). Can robots slay sexism? World Economic Forum.
Mott, N. (Jun 8, 2016). Why you should think twice before spilling your guts to a chatbot. Passcode.
Paolino, J. (Jan 4, 2017). Google home vs Alexa: Two simple user experience design gestures that delighted a female user. Medium.
Seaman Cook, J. (Apr 8, 2016). From Siri to sexbots: Female AI reinforces a toxic desire for passive, agreeable and easily dominated women. Salon.
Twitter. (Apr 7, 2016). Automation rules and best practices. (Web page, accessed 12/29/16)
Yao, M. (n.d.). Can bots manipulate public opinion? (Web page, accessed 12/29/16)

Privacy

Abadi, M., Chu, A., Goodfellow, I., Brendan McMahan, H., Mironov, I., Talwar, K., et al. (2016). Deep Learning with Differential Privacy. ArXiv e-prints.
Amazon.com. 2017. Memorandum of Law in Support of Amazon's Motion to Quash Search Warrant
Brant, T. (Dec 27, 2016). Amazon Alexa data wanted in murder investigation. PC Mag.
Friedman, B., Kahn Jr, P. H., Hagman, J., Severson, R. L., & Gill, B. (2006). The watcher and the watched: Social judgments about privacy in a public place. Human-Computer Interaction, 21(2), 235-272.
Golbeck, J., & Mauriello, M. L. (2016). User perception of facebook app data access: A comparison of methods and privacy concerns. Future Internet, 8(2), 9.
Narayanan, A., & Shmatikov, V. (2010). Myths and fallacies of "personally identifiable information". Communications of the ACM, 53 (6), 24-26.
Nissenbaum, H. (2009). Privacy in context: Technology, policy, and the integrity of social life. Stanford: Stanford University Press.
Solove, D. J. (2007). 'I've got nothing to hide' and other misunderstandings of privacy. San Diego Law Review, 44 (4), 745-772.
Steel, E., & Angwin, J. (Aug 4, 2010). On the Web's cutting edge, anonymity in name only. The Wall Street Journal.
Tene, O., & Polonetsky, J. (2012). Big data for all: Privacy and user control in the age of analytics. Northwestern Journal of Technology and Intellectual Property, 11(45), 239-273.
Vitak, J., Shilton, K., & Ashktorab, Z. (2016). Beyond the Belmont principles: Ethical challenges, practices, and beliefs in the online data research community. In Proceedings of the 19th ACM conference on computer-supported cooperative work & social computing (pp. 941-953).

NLP Apps Addressing Ethical Issues

Fokkens, A. (2016). Reading between the lines. (Slides presented at Language Analysis Portal Launch event, University of Oslo, Sept 2016)
Gershgorn, D. (Feb 27, 2017). NOT THERE YET: Alphabet's hate-fighting AI doesn’t understand hate yet. Quartz.
Google.com. (2017). The women missing from the silver screen and the technology used to find them. Blog post, accessed March 1, 2017.
Greenberg, A. (2016). Inside Google'S Internet Justice League and Its AI-Powered War on Trolls. Wired.
Kellion, L. (Mar 1, 2017) Facebook artificial intelligence spots suicidal users. BBC News.
Munger, K. (2016). Tweetment effects on the tweeted: Experimentally reducing racist harassment. Political Behavior, 1-21.
Munger, K. (Nov 17, 2016). This researcher programmed bots to fight racism on twitter. It worked. Washington Post.
Murgia, M. (Feb 23, 2017). Google launches robo-tool to flag hate speech online. Financial Times.
The times is partnering with jigsaw to expand comment capabilities. (Sep 20, 2016). The New York Times.

Fake News Challenge
Jigsaw Challenges
Perspective (from Jigsaw)
- But see: Hosseini, H, S. Kannan, B. Zhang and R. Poovendran. 2017. Deceiving Google's Perspective API Built for Detecting Toxic Comments. ArXiv.
Textio See also:
- CEO Kieran Snyder's posts on medium.com
- Recording of Kieran Snyder's NLP Meetup talk from Aug 15, 2016

Crowdsourcing

Bederson, B. B., & Quinn, A. J. (2011). Web workers unite! Addressing challenges of online laborers. In CHI'11 extended abstracts on human factors in computing systems (pp. 97-106).
Callison-Burch, C. (2016). Crowd workers. (Slides from Crowdsoucing and Human Computation, accessed online 12/30/16)
Callison-Burch, C. (2016). Ethics of crowdsourcing. (Slides from Crowdsoucing and Human Computation, accessed online 12/30/16)
Fort, K., Adda, G., & Cohen, K. B. (2011). Amazon mechanical turk: Gold mine or coal mine? Computational Linguistics, 37 (2), 413-420.
Snyder, J. (2010). Exploitation and sweatshop labor: Perspectives and issues. Business Ethics Quarterly, 20 (2), 187-213.

Other

Cohen, K. B., Pestian, J., & Fort, K. (2015). Annotateurs volontaires investis et éthique de l'annotation de lettres de suicidés. In ETeRNAL (ethique et traitement automatique des langues).
Fort, K., & Couillault, A. (2016). Yes, we care! results of the ethics and natural language processing surveys. In Proceedings of the tenth interna- tional conference on language resources and evaluation (LREC 2016). Paris, France: European Language Resources Association (ELRA).
Gillespie, T. (2014). The relevance of algorithms. In T. Gillespie, P. J. Boczkowski, & K. A. Foot (Eds.), Media technologies: Essays on communication, materiality, and society (pp. 167-194). MIT Press.
Hardt, M., Price, E., & Srebro, N. (2016). Equality of opportunity in supervised learning. (Accessed online, 12/30/16)
- Related blog post
Kleinberg, J. M., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. CoRR, abs/1609.05807.
Metcalf, J., Keller, E. F., & boyd, d. (2016). Perspectives on big data, ethics, and society. (Accessed 12/30/16)
Meyer, M. N. (2015). Two cheers for corporate experimentation: The A/B illusion and the virtues of data-driven innovation. Colo. Tech. L.J., 13, 273.
Wallach, H. (Dec 19, 2014). Big data, machine learning, and the social sciences: Fairness, accountability, and transparency. Medium.
Wattenberg, M., Viégas, F., & Hardt, M. (Oct 7, 2016). Attacking discrimination with smarter machine learning.

Value Sensitive Design

Borning, A., & Muller, M. (2012). Next steps for value sensitive design. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1125-1134).
Friedman, B. (1996). Value-sensitive design. ACM Interactions, 3 (6), 17-23.
Friedman, B., & Hendry, D. (To appear). Value Sensitive Design: a twenty-year synthesis and retrospective. In Foundations and trends in human computer interaction. [Now available on "Files" tab of the course canvas]
Friedman, B., & Kahn Jr., P. H. (2008). Human values, ethics, and design. In J. A. Jacko & A. Sears (Eds.), The human-computer interaction handbook (Revised second ed., pp. 1241-1266). Mahwah, NJ.
Nathan, L. P., Klasnja, P. V., & Friedman, B. (2007). Value scenarios: a technique for envisioning systemic effects of new technologies. In CHI'07 extended abstracts on human factors in computing systems (pp. 2585-2590).

(Proposals for) codes of ethics

ACM Ethics Task Force. (2016). Code 2018 | ACM ethics. (Web page, accessed 1/5/17)
The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems. (2016). Ethically aligned design: A vision for prioritizing human wellbeing with artificial intelligence and autonomous systems (AI/AS) (Version 1 -- For Public Discussion).
Etlinger, S., & Groopman, J. (2015). The trust imperative: A framework for ethical data use.
Daumé III, H. (Dec 12, 2016). Should the NLP and ML Communities have a Code of Ethics? (Blog post, accessed 12/30/16)

Other Best Practices

Amodei, D., Olah, C., Steinhardt, J., Christiano, P., Schulman, J., & Mané, D. (2016). Concrete problems in AI safety. CoRR, abs/1606.06565.
Markham, A. (2012). Fabrication as ethical practice: Qualitative inquiry in ambiguous Internet contexts. Information, Communication & Society, 15(3), 334-353.
Ratto, M. (2011). Critical making: Conceptual and material studies in technology and social life. The Information Society, 27 (4), 252-260.
Russell, S., Dewey, D., & Tegmark, M. (2015). Research priorities for robust and benefcial artifcial intelligence. AI Magainze.
Shilton, K., & Anderson, S. (2016). Blended, not bossy: Ethics roles, responsibilities and expertise in design. Interacting with Computers.
Shilton, K., & Sayles, S. (2016). "We aren't all going to be on the same page about ethics": Ethical practices and challenges in research on digital and social media. In 2016 49th Hawaii international conference on system sciences (HICSS) (pp. 1909-1918).