To: Students in Psychology 384, Research Laboratory in Human Factors

From: Susan Brennan and Tony Weaver

Date: Tuesday, Feb. 10, 2009

Re: Assignment 2--Heuristic evaluation (Due in class Th, Feb 17, 2009)

In this exercise, you and your classmates will practice the heuristic evaluation technique. Make sure you are familiar with the reading on heuristic evaluation (Nielsen’s article entitled, "How to conduct a heuristic evaluation," and be sure to follow the links). Review the 10 heuristics at the end of this handout.

The application you will evaluate is the Word dialog box for Track Changes. Start up Word and then try the following 3-part task:

1. Type in a paragraph of text, save it, and then turn on Track Changes. You can do this by selecting Tools on the top menu bar (or you may discover other ways to turn it on too). Then make a few tracked, highlighted edits to your document.

2. Change the color that your edits are highlighted in.

3. Explore the toolbar associated with Track Changes (located at the top of your document) and try to use it to “accept” the changes you have made (this makes the highlighting and the old version go away).

Regardless of whether you already know how to do all this, apply the heuristics from Nielsen's list and try to critique the system from the perspective of someone who does not know how to do this. We will compile the results of your evaluation with those of your classmates to see how many people find the same problems, as well as to see who finds (a) the most problems and (b) the hardest-to-find problems. Be sure to include all of the following parts in your memo:

(1) Start your memo by saying whether you have ever used Track Changes before (and if so, how often and successfully). Then consider your experience with computers, Windows, Word, and editing documents, and classify yourself (in Nielsen's terms) as "novice evaluator", "regular specialist”, or “double specialist".

(2) As you investigate and use the Track Changes tool, take careful notes of any features of the interface that seem to violate Nielsen’s 10 heuristics, as well as any dead ends and false starts you experience, as these provide good evidence of problems. Even if you find the task easy, attempt to apply the heuristics and find as many problems as you can.

(3) Then write up the list of all the problems you found. Number each problem and describe it in a short paragraph. Each paragraph about a problem should include the following information. It’s essential that you complete all of the following steps for each problem you found:

· Briefly describe the problem, saying why it is a problem and where it arose in your use of the Track Changes tool. We should be able to recreate the problem from your description.

· Say how it could be avoided or fixed.

· List in parentheses at the end of the paragraph which heuristic is being violated (see Nielsen’s list of 10 heuristics, next page).

· Finally, rate each problem for its severity, considering both how serious it is and how often it comes up. Use the following numeric ratings:

1 - cosmetic problem

2 - minor usability problem

3 - major usability problem; important to fix

4 - usability catastrophe; imperative to fix

Make sure your memo is polished and professional; proofread it for typos, misspellings, and awkward or unclear sentences. Ask us for help if you have any questions. Don’t consult with your classmates while you evaluate the interface; it’s important that each evaluator work separately for the heuristic evaluation to be valid.

Ten Usability Heuristics (Nielsen, www.useit.com/papers/heuristic/heuristic_list.html)

(See the lecture slides also.)

1. Visibility of system status

The system should always keep users informed about what is going on,

through appropriate feedback within reasonable time.

2. Match between system and the real world

The system should speak the users' language, with words, phrases and concepts familiar

to the user, rather than system-oriented terms.

Follow real-world conventions, making information appear in a natural and logical order.

3. User control and freedom

Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

4. Consistency and standards

Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

5. Error prevention

Even better than good error messages is a careful design which prevents a problem from occurring in the first place.

6. Recognition rather than recall

Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

7. Flexibility and efficiency of use

Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

8. Aesthetic and minimalist design

Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

9. Help users recognize, diagnose, and recover from errors

Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

10. Help and documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.