The Information Gateway, the University of Washington Libraries’ website, is the critical point of entry for the Libraries’ online services, but the usability of the completed product has never been assessed. As we consider changes to the gateway it is important to have a basic understanding of what parts are working and which need the most improvement.

Programmatic usability testing has been a long-time goal of the Libraries, but one only recently realized. Changes in staffing have brought a usability coordinator as well as a graduate student dedicated to usability studies on board. Lab space is tight on campus, so equipment was purchased to aid in the conducting and monitoring of tests. The equipment is currently housed in a shared conference room, but will soon move to a dedicated space once library renovation is complete. Assessment of the site began last summer and will continue as the site evolves and more services are added. This paper discusses the local environment and the methods used to date in evaluating the UW Libraries website and services.

Local Environment
Library Systems, the unit responsible for the overall health and well-being of the Information Gateway, sponsors usability studies. Web publishing in our environment is very much a decentralized effort; some pages are maintained centrally, but units do have their own site and are free to provide whatever content they deem fit for their users. A Web Steering Committee (WSC) is charged with providing policy and general management and oversight for the University Libraries website. Currently, the WSC suggests areas of the website to test and is responsible for design prototyping and implementing any changes.

Triennial Survey
Results from a triennial survey (http://www.lib.washington.edu/surveys/) show that the Information Gateway has a very high satisfaction level with our users. It’s gratifying that our website is well thought of, but as someone who wants to improve our online services, this is not necessarily what I wanted to hear! While this positive news is good, we needed more detailed information about our site. To achieve that goal, we are using a variety of methods to assess our site including online surveys, a card sorting test, and we are currently in the process of organizing usability testing.

Online Survey – Summer, 2001
In an attempt to get a better sense of how people use and feel about our website, a survey was run in summer, 2001. Results from this effort were not very satisfactory – our timing was off and we received approximately 50 responses over a four-week period. Another problem was with the survey design – it was structured such that many of the responses were left blank. We revised the survey and ran it again when more users were on campus.

Online Survey Revisited – Autumn, 2001
Results from this second attempt were much improved. The survey was publicized in an online newsletter that was sent to all students, faculty, and staff and an image at the bottom of the library’s main page lead users to the survey. Of the 150+ responses we received, approximately 130 contained useful information. Our OPAC received the greatest number of comments but, like most OPACs, the layout and functionality is limited to that which the vendor provides. As with the triennial survey, the overwhelming sentiment was that while some areas of the site need work, for the most part it was not “broken”.

Card Sorting – Autumn, 2001
While the survey addressed more general questions, we wanted to know how users were fairing with our site’s organization. Do the labels we put on our resources make sense to our users? Are we grouping resources and services in a manner that users can intuitively understand? To explore these questions we used a method called card sorting. It’s a very low-tech method and involved the use of index cards, envelopes, and a pen. Users were given two identical sets of index cards that contained the names of library resources and services. They were asked to rank the first set according to how important the item is in their work. Participants grouped, labeled, and renamed the second set of cards into an organization that was meaningful to them. As of early January, 2002 results from these studies are still being analyzed.

Usability Testing – Winter, 2002
As you have probably surmised, we have the luxury of not having a deadline by which our testing must be complete. The WSC has asked us to look at a specific section of our site, the “By Subject” area. This section of the site is powered by an SQL database and dynamically creates lists of electronic resources according to specific subject areas. Not surprisingly, it has grown in size and complexity since the site was launched and now needs quite a bit of work. Two areas of exploration are how to help the beginning researcher find appropriate resources and how to make this set of pages more easily navigable. Formal usability testing will be used for this inquiry and participants will be encouraged to think aloud.

Other Data Sources
In addition to the evaluation methods listed above, web usage statistics and search engine logs are also analyzed. While these other sources can provide valuable information, the final word comes from the user – what she says and does during the course of other usability studies.

Conclusion
Creating and maintaining a usable website is very much an on-going process, one that is just beginning at the University of Washington Libraries. The presentation will cover the steps we’ve taken, lessons learned, and a look at what the future holds for our site. More importantly, time will be left after the presentation for participants to share their experiences with usability studies or the challenges of interface design.

URL for presentation materials: http://faculty.washington.edu/jlward1/talks/onlinenw/

URL for UW Libraries Usability site: http://www.lib.washington.edu/usability/

last modified: Monday, March 4, 2002