Adventures in Site-seeing: Implementing Web Usability Tests for Your Library Web Site
Rob Behary, Duquesne University

Rob gave an excellent presentation; unfortunately, Sydney Walden, his co-presenter was unable to attend due to personal reasons, but he gave her full credit for her help in preparing the session.

I was very interested to hear what he had to say, as we are in the process of redesigning our web page using LibGuides.

Assumptions they started with:
-No budget for testing
-No software, no staff budget
-Library web sites still matter
-Any improvements are worth the time

Duquesne is a conservative culture, want to manage university's 'image' closely. The library's goal was to make their web page more user focused. Their LibQUAL study showed that users found their webpage deficient.

Good article: Jeng, J. Usability Assessment of Academic Digital Libraries Libri 2005

Elements of their study
-learnability
-satisfaction
-efficiency
-effectiveness
-error correction
-ease of use
-visual appearance

In their next LibQUAL study they added some library specific questions, to get more information about their web site. They had some support in place, multimedia graduate program and strong faculty partners, had some software and hardware available for testing. Students approached them about testing their web site

Used Morae - TechSmith software (expensive $2,000 for license, but their IT dept. had already)- used for experimental testing. It has a high learning curve. Tracks mouse clicks and mouse movements, time between mouse clicks, and time on a task, also works with a video camera to track user responses.

Needed additional data - did brief survey to library staff; did a card sort/free association; 'what would you put at the top level?', etc.

Advice:
- Before starting come up with elements, what you're going to be testing
- Do an environmental scan, what resources are available to you - student or faculty partners?
- Don't get frustrated by user expectations

Minimal expertise needed:
- Ability to conduct and analyze primary research
- Ability to conduct interviews
- Ability to do a basic modal analysis

Ideal
- Understand how to design a study
- Some experience with inferential statistics

Consider your resources -- ideally need at least 2 people; both taking notes, helpful to get 2 perspectives (one to handle interruptions)

Minimal requirements for technology:
-Workstations
-Isolated room, preferably soundproofed (can also use video camera)
-Survey software

If you need to build your skills, ACRL offers an e-learning course: Intro to Website Usability Course (developed in 2007). Can also audit university courses on research design (Ed or social sciences).

If you need to obtain software, can use Morae for a 30 day trial. There are less expensive/free options. Possibly forego the software?

How We Did Our Testing
- Referred to article "Why you only need to test with 5 users" Jakob Nielsen
- Needed mix of faculty/staff/students, undergrad and grad
- It didn't really matter what tasks they gave their users, they all seemed to make the same comments about font, color, etc.

Morae analyzed total time, time to complete, number of times back button used, umber of mouse clicks, success rate, number of errors, negative feedback (facial expressions and negative comments)

Some students and faculty analyzed the library's site, and the librarians made a point of attending their sessions where they presented their research/

Extremely useful web site for creating detailed statistics on text comments: TextSTAT - they used it to analyze their LibQUAL comments; it even shows key word in context

Also surveyed users, including library staff: "Please answer the following from very important to not important, and then from very easy to not easy"
-Accessing the library webpage
-Asking the library a question online
-Finding a book
etc.

They used 30 users for their card sort: they had them choose categories, and organize other items under the categories they suggested.

Constraints
-University template was developed to recruit students; library web site is more an application than a public relations tool or fundraising tool (although it does have that function, too)
-Limitations on coding/scripting
-Duquesne uses Luminis Content Management System - allows them to share pages with each other, sustainability advantage

Has Usability Helped?
-University redesign, everyone is switching to the CMS, library was 'early adopter'
-New appreciation for library as such, are performing leadership role on campus

Where are we now?
-Some subject pages redesigned; Rome wasn't built in a day, however
-New design in progress