Term Project Specs

Term projects for this class will take one of the following forms:

1. Take an existing off the shelf system and run it over some new data set with demographic information included. Do a demographically-informed error analysis. Present the error analysis and a discussion of its ethical implications, both in terms of what will happen if the system is used unmodified and how the system could/should be improved.
   • For example: running an off-the-shelf English ASR system over the data collected (and transcribed) by the Voices from the Rwanda Tribunal Project
2. Take an existing NLP task and analyze it in terms of Value Sensitive Design. Assume that systems approaching the task will be deployed by companies and/or government agaencies and/or available for use by anyone as open source projects. Who are the stakeholders that should be consulted? What questions should the stakeholders be asked? If possible, find representative stakeholders from multiple categories and interview them. On the basis of these interviews (or your own analysis, if interviews aren't possible), write recommendations for improvements in the training data, task design, or task deployment. If you are working on an MS thesis, and you know what NLP task it will involve, it is strongly recommended that you take this option with your thesis task.
3. Some other project that you propose.

Paper

• Abstract: State the task/software/question discussed and your main conclusions.
• Introduction: Briefly overview the questions you are approaching, summarize the main conclusions, and give an overview of the paper.
• Background/related work:
  • Present, with citations, the framework you are using for your analysis
  • Describe in siffucient detail the NLP task you are considering
  • Discuss any related work looking at ethical considerations pertaining to the same or similar tasks and how this work informs or differs from your approach.
• Methodology: Describe what you did (e.g. running of software, interviewing people); describe any data sets you used
• Results: Present what you found (error analysis, interview responses)
• Discussion: Apply your analysis framework to your results, describe your recommendations
• Ethical Considerations: What potential for harm does this project have, and how can such harm be mitigated?
• Conclusion: Summarize the main points and provide look ahead. What should happen next?
• Acknowledgments: Acknowledge any help you received (e.g. thoughtful feedback from classmates, help from developers to run their software, your interviewees' time)
• Bibliography: References for all papers cited in the text. Does not count towards/against the page requirements. It is expected that your bibliography will include assigned readings from the class as well as additional readings specific to your term project.

Milestones

To help you with this term paper, there will be several milestones along the way:

1. 10/25 Project proposal: If taking options (1) or (2), this can be a paragraph explaining the system and data set (1) or task (2) you are approaching. If taking option (3), I'm looking for a page explaining the project.
2. 11/13 Paper outline: Fill in the sections above (or modify, if somethign there doesn't fit) with bullet points specific to your paper.
3. 11/22 Paper draft
4. 12/2 Comments on peer's paper draft
5. 12/12 Final paper