Instructor: Judy Ramey
Email: jramey@u.washington.edu
Web site: http://faculty.washington.edu/jramey/TC517/syllabus.shtml
Office hours: by appointment
Class meetings: MW 1:30-3:20, Loew 115

Course description

This course will introduce you to usability testing and, in less detail, to usability research as a user-centered design strategy. The course takes a process approach; you will learn how to define your audiences and issues, create investigative procedures that answers your questions, administer the procedure, analyze the results, and report your findings effectively.

Course objectives:

At the end of this course, you should be able to:

Understand and explain to others what usability testing and usability research are and what they can contribute to a design effort
Analyze the usability issues that a product has and prioritize the ones that merit investigation through a usability test; analyze the various audiences for the product and prioritize the ones that are most critical at the current moment
Design a usability test that answers the questions you have for the audiences of interest
Administer the test, analyze the results, and report the findings effectively

Books and readings

You are expected to come to class prepared to discuss the readings for the day. The following readings are all required:

Barnum, Carol M. Usability Testing and Research (available in the UW bookstore)
Course packet Ave Copy Center, 4141 University Way NE
Other readings, assigned as necessary and posted on the course web site.

Course policies

I hope you enjoy the course, have some fun, and learn a lot. Please familiarize yourself with the course policies, listed below. Thank you for turning off phones and pagers before class begins.

Office hours, email and communication

I hold office hours by appointment. Please email me if you have questions or if you wish to set up an appointment. I will respond to email in a timely manner, usually within two days. I will not respond to questions about assignments 24 hours before they are due. I encourage you to use the discussion board on the course web site.

Attendance and participation

I expect you to attend class and be on time. If you need to miss a class, please talk to me. If your absence from class impacts a group project, you will be required to negotiate missed work with your group.

Assignments

If you wish to discuss an assignment, please make an appointment. Please bring a printed version of your assignment with you. Typically, I do not review assignments via email. You must be present in order to receive feedback.

To submit

All assignments, unless otherwise noted, are due at the beginning of class in hardcopy format. In some cases, you may be permitted to submit your assignment electronically; please talk to me if you want to do this. If the file is larger than 2 MB, please compress it with WinZip or upload it to a website and send me the URL.

Late assignments

If I do not receive your assignment at the beginning of class on the day that it is due, it is considered late. It is imperative for you to turn in assignments on time. For each day an assignment is late, you will lose .5 off the final grade. I will not accept assignments more than three days late.

Student rights and responsibilities

Please read Department of Technical Communication (TC) policies for students registered in TC courses regarding student rights, plagiarism, and the TC human subjects pool.

Student rights:
https://intranet.uwtc.washington.edu/academicresources/studentrights.php
Plagiarism:
https://intranet.uwtc.washington.edu/academicresources/plagiarismpolicy.php
TC Human Subjects Pool:
http://www.uwtc.washington.edu/research/hsr/

Students registered in TC courses are part of the TC Human Subjects Pool, which means that they may be asked to participate in research studies. Because participation in research studies is voluntary, students who do not wish to participate will be offered an alternative assignment.

Course assignments

The assignments in this class are a linked set, each of which builds on the last. You are expected to turn in assignments at the beginning of class on the day indicated.

Assignment	Description	Percentage of grade	Due date
1. Prelim Proposal	Preliminary proposal of the usability issues your group will investigate. One-page (maximum) This assignment is not graded, but the more carefully you do it, the better your chances of success in the rest of the assignments.		Wed Week 3
2. Problem Statement	Prepare a detailed (5-page maximum) problem statement and audience definition; briefly discuss three articles that bear on the kind of problem you are going to investigate or the kind of procedure that you are going to use	15%	Wed Week 5
3. Test Design Kit	Prepare a detailed test design kit that: 1. Describes the procedures that you will use to investigate the problems specified in your problem statement 2. Provides a participant profile and describes how you will verify that your recruits match it, and 3. Describes what data you will collect and how. Attach your complete set of test materials (script, participant profile, task list, etc.)	20%	Wed Week 6
4. Study Results	Do a two-participant pilot study: run two participants. You will present the results of your study: 1. In writing, in the form of a formal written report.	40%	Wed Week 9
	2. Orally as an informal presentation to the class. a. Discuss one major finding and a recommendation for a change to the product b. Discuss one aspect of your study you would change if you were to run it again.	15%	Mon Week 10
Class participation	Being actively involved in class discussions, participating in workshops, being actively involved in your group's project and posting to the class discussion board	10%	Ongoing

Course Schedule (subject to change)

Consult the course web site for the latest version of the course schedule.

Week/Date

Topics

Activities

Readings and Assignments due

Sept. 26

· Introduction to the course

- Introductions of course, syllabus, students

Week 1

Oct. 1 & 3

Defining usability

· What is usability?

· What is usability engineering?

· What is user-centered design?

· What is usability testing?

· Where did the field of usability come from?

- Discussion: human characteristics/factors

· Barnum: Ch: 1: What Is Usability and What is Usability Testing?

· Hafner, Katie. "Wanted by the Police: A Good Interface" New York Times, Nov 11, 2004.

· Gould, J.D. and Lewis, C., "Designing for usability: Key principles and what designers think," Communications of the ACM, 1985, 28, 300-311.

· Rosenbaum, S., "Usability Evaluations Versus Usability Testing: When and Why?"

Week 2

Oct. 8 & 10

Evaluating a product

· How do we define and state usability issues?

· How do we distill concerns about products?

· How do we determine what to test?

· What are the different types of usability studies?

Workshop:

- Product critique

- Form teams, determine what product you will study

· Barnum Ch 4: Iterative Testing for User-Centered Design.

· Neilsen, J., Usability Engineering, ch. 5, "Usability Heuristics"

· Jong, M. de, and Geest, T.v.d., "Characterizing Web Heuristics"

· Research-Based Web Design & Usability Guidelines, http://www.usability.gov/pdfs/guidelines.html

· One-pager on the Cognitive Walkthrough: http://www.pages.drexel.edu/~zwz22/CognWalk.htm

Week 3

Oct. 15 & 17

Understanding and prioritizing users

· What methods do we use to get feedback from users?

· How to choose which users will participate in the usability study?

· How many users do we need for a usability study?

· How do we learn about users' goals and tasks?

· What characteristics of users are salient to the design?

Workshop:

- Prioritizing users

- Developing user profiles

· Barnum: Ch 2: Other Methods for Getting Feedback and Ch 3: User and Task Analysis

· Dumas, Joe, "How many participants in a usability test is enough?," Common Ground, UPA

· Barnum, Carol, "What's in a Number?," Usability Interface, January 2003, pp. 7-9.

Due: Preliminary Proposal

Week 4

Oct. 22 & 24

Planning for a usability study

· What do we need to do before a usability study?

· How do we recruit representative users?

· How do we get organizational support for usability?

· What are some of the common objections or arguments against usability and how do we respond?

LUTE tour

Workshop:

- The planning grid

- Responding to usability critics

Work in your teams

· Barnum: 5. Planning For Usability Testing.

· UW Office of Research, "Using Human Subjects in Research: Why informed consent is crucial," Office of Research Newsletter, 1999, pp. 1-3.

· Dray and Siegel, "Penny-Wise, Pound-Wise: Making Smart Trade-offs in Planning Usability Studies"

· Bias and Mayhew, Cost Justifying Usability, ch. 2 and 22

Week 5

Oct. 29 & 31

Preparing for the usability study

· What test materials do we need for a usability study?

· What are scenarios and how are they used in a usability study?

· How do we create successful questionnaires?

Workshop

- Scenario writing

- Questionnaires

· Barnum: 6. Preparing For Usability Testing.

· Dumas, Joe, "Usability testing Methods: Subjective Measures Part I—Creating Effective Questions and Answers"

· Dumas, Joe, "Usability testing Methods: Subjective Measures Part II—Measuring Attitudes and Opinions"

Due: Problem Statement

Week 6

Nov. 5 & 7

Conducting a usability study

· What is "thinking aloud"?

· What is the key to successful facilitation?

· How do we collect data from a usability study?

Workshop:

- Practice facilitating

- Clock workshop

· Barnum: 7. Conducting The Usability Test.

· Boren and Ramey, "Thinking Aloud: Reconciling Theory and Practice" Professional Communication, IEEE Transactions on Volume: 43 Issue: 3, Sept. 2000

· Norgaard and Hornbaek, "What Do Usability Evaluators Do in Practice? An Explorative Study of Think-Aloud Testing," DIS 2006

· Weiss, "Interviewing," from Learning from Strangers

Due: Test Design Kit

Week 7

(Nov 12 is Veteran's Day)

Nov. 14

Analyzing data from a usability study

· Where do we start?

· What tools can we use?

· What are informal and formal approaches to data analysis?

Workshop:

- Data analysis: making sense of data

· Barnum: Ch 8. Analyzing and Reporting Results
and Ch 9. Web Usability.

· Wilson, Chauncy "Analyzing and Reporting Usability Data" STC Usability Sig Newsletter. http://www.stcsig.org/usability/newsletter/9710-analyzing-data.html

· Kantner et al, "Organizing Qualitative Data from Lab and Field: Challenges and Methods" paper from UPA 2005, available on http://www.teced.com/ourwork_published.html

Week 8

Nov. 19 & 21

Remote usability, usability in the field, and other contexts

· What are the challenges of conducting usability activities in the field?

· What are other tools and approaches?

Exercise:

Do a speed contextual inquiry

· Perkins, Ron. "Remote Usability Evaluation Over the Internet," in Design by People for People: Essays on Usability, Russell J. Branaghan, ed., UPA: 2002, pp. 153-162.

· Gough and Phillips, "Remote Online Usability Testing: Why How and When to Use it"

· Froehlich et al, "MyExperience: a system for in-situ tracing and capturing of user feedback on mobile phones," MobiSys '07

· Tamler, Howard, "High-tech versus hightouch: Some limits of automation in diagnostic usability testing," copyright 2001 by Howard Tamler PhD,
http://www.ht_consulting/papers/

· Beyer and Holtzblatt, ch. 3, "Principles of Contextual Inquiry" and ch. 4, "Contextual Inquiry in Practice"

· "We Are All Connected: The Path from Architecture to Information Architecture" by Fu-Tien Chiou http://www.boxesandarrows.com/

· Winschiers, "Context: Designing for Usability in Namibia"

· Salvador et al., "Design Ethnography," Design Management Journal 10(4): 35-41

Week 9

Nov. 26 & 28

Reporting and communicating results

· What are the options for communicating the results of a study?

· How do you choose which way to communicate results?

· When are highlight tapes useful?

· How do we convey findings and recommendations and how can you keep them alive and actionable?

· What reports make the most impact?

Workshop:

- Responding to questions of rigor and validity

· Ehrlich, et al "More for Less: A Novel Hybrid Method to Maximize the Impact of Research" ACM 2003.

· Hughes, Michael. "Rigor in Usability Testing." Technical Communication, Fourth Quarter (1999): 488-94.

· Yeats, Dave; Carter, Locke. "The Role of the Highlights Video in Usability Testing: Rhetorical and Generic Expectations" Technical Communication, May 2005: pp. 156-162(7).

· Krug, Steve "Usability Testing on 10 cents a day" Ch 9 from Don't Make Me Think. New Riders 2000.

Week 10

Dec. 3 & 5

Course wrap up

· What did you discover about usability? What questions do you have?

· What are the challenges of the field?

· What is the future of usability?

- Team presentations

- Course evaluations

· O'Hara et al, "Consuming Video on Mobile Devices," CHI 2007.

· Wilson, Chauncey E. "Usability and User Experience Design: The Next Decade." Intercom: 6-9. January 2005.

· Rosenbaum, "The Future of Usability Evaluation: Increasing Impact on Value"

Due: Study Results