To: Psychology 384 students

From: Susan Brennan and Tony Weaver

Date: Thursday, March 5, 2009

Re: Assignment #5, Computer Agents (due Thursday, March 12, 2009)

The point of this assignment is to evaluate the use of an interface agent. You have a choice of

· (Option 1) evaluating a GPS car navigation system such as the Garmin, or

· (Option 2) evaluating the paper clip agent in Microsoft Word,

· (Option 3) comparing two customized Web search agents (for ideas, see the list below; consult one of us if you want to evaluate agents not listed here).

In a 2-3 page memo (1 1/2 spaced) addressed to us, evaluate how helpful the agent was and identify times when the agent misunderstood or misunderstood your intentions and goals. For part of the assignment, 1) describe what you did and how the agent responded, 2) try to figure out the rules the agent used to help you or to evaluate your intentions, and 3) draw appropriate conclusions.

Option 1: Car Navigation Systems

If you have access to a car navigation system like the Garmin, take at least two trips, where at least one is to a familiar destination, and the other, to an unfamiliar destination. Since you can’t take notes while you’re driving, get someone else to drive (or if you’re by yourself, make sure you can pull over to take notes when necessary). Include the following:

· What is represented on the graphic display? Does it work well to guide you?

· Describe the agent’s voice; comment on its way of interacting vocally.

· What does the agent do when you make a wrong turn? Is this effective, and why?

· Were you able to avoid taking a certain route? (that is, if you tried to take a certain route instead of what the agent recommended, did it recalculate with your route, or keep directing you to its route?

· Do you always feel confident that the agent is giving you the right instructions?

· How easy was it to indicate a destination if you weren’t sure of the address?

· What did the agent do differently than a human navigator?

· How did you find the experience, overall? What if anything did the agent do that was unexpected, frustrating, or peculiar?

Do a bit of reverse-engineering – describe how you think the GPS system works, based on what you noticed.

Option 2: Microsoft Word Agent

This is a text-editing agent that makes suggestions about the document you are working on. It is represented as various kinds of text highlights, as well as by an animated character (a paperclip on PCs, a little computer with feet on Macs.) If you evaluate this agent, try a set if different tasks with which the agent could assist. Some suggestions:

· Try something monotonous, such as making a list or cutting and pasting. Does the agent do anything? If something is highlighted, is it clear why?

· Add a comma in where it doesn’t belong and see what the agent does. Misspell a word on purpose.

· Make an outline and see if the agent jumps in to assist you in formatting.

· Use the Spelling and Grammar section to see if you have any ungrammatical sentences. Then try to write one and see what kind of feedback the agent gives you.

· Write a short paragraph on a given topic and then go to Auto Summarize to see if the agent can highlight the important points in your paragraph.

· Then write a paragraph and see for what grade level the agent deems it appropriate. Go to Spelling and Grammar and click on Options to have the agent evaluate Readability. Do you find its evaluation surprising in any way? Change your writing and see if you can get the agent to give you a different evaluation. Describe the rules you think the agent is using to judge grade level.

· See if you can get the agent to capitalize something for you automatically; then see if you can get it to not capitalize in that kind of situation.

· Then try to customize your agent by changing the Options under Tools in AutoCorrect and under Spelling and Grammar.

For each of these tasks, describe what you did and how the agent responded (e.g., when I typed "Mr. paperclip", Microsoft Word capitalized the "P" in paperclip (after Mr.) because it assumed that I was typing a proper name). When did the animated character pop up, and what was your reaction? For both the text-editing suggestions and the pop-up animated character, try to deduce the rules the agent used to produce its behavior.

Option 3: Customized Information Agents

The other option for this assignment is to review two customizable Web sites that can customize personal information for users. You might want to compare the agent-like features in two or more of the following Web sites (AMAZON.COM, BARNESANDNOBLE.COM, GARDEN.COM, IMAGINERADIO.COM, CDNOW.COM, EXCITE.COM, YAHOO.COM, REEL.COM, COLDWELLBANKER.COM). Search agents are supposed to infer personal preferences and judge whether information on the Web is similar enough to be desirable to you (e.g., the AMAZON.COM agent should remember your book preferences from previous inquiries and recommend additional books you would like). Try to deduce the rules behind the agent’s ability to deal with search commands and how it secures private information.

You should start the assignment by trying out some of the above Web sites. Try at least two, so you can judge how successful they are relative to each other.

· Start by entering any requested information. If your browser is set up to enable cookies (software programs that web sites can place on your system and track your browsing behavior), personalization may be easier, but you may be giving up some privacy.

· Determine how intelligent the agent seems. Does it provide what you are looking for?. Can you tell how its suggestions are related, or on what basis suggestions are made? For example, can the REEL.COM agent search be done based on film genre info alone or does it need more data?

· After establishing an identity (most likely with a login and a password), perform a few searches with the agent. Then log out and back in again. When you log back in, the agent should know your preferences. For example, if you used AMAZON.COM to search for books on the Battle of Bull Run, it might suggest other books on the Civil War. Determine if the agent is responding according to your preferences.

· Try to infer how the agent is able to customize the Web page it displays for you.

· Does the agent explain how or whether it keeps your records confidential? Can you imagine any negative consequences of using this particular agent?

· Repeat this process with at one other personal search program. Compare your experience with the two. Which agent was more successful, and why?

Conclusions (Don’t forget to include this section!)

Whether you choose Option 1 or 2, conclude your paper by assessing the overall usefulness of agents. Were there situations in which you found the agent annoying or useless, or when you wished you had direct access to the material (such as a browser or map)? Was the way the agent was represented on the screen appropriate - that is, was it informative or just intended to be entertaining? Did you find it annoying or engaging? Did it cause you to have accurate or unrealistic expectations about what the agent could do? Does your agent seem to have a personality? In your memo, did you find yourself writing about the agent as if it were a person? Give at least one way in which you think the agent you used could be improved. Discuss under what circumstances and why you think agents can be useful. Conclude by commenting on any relevant ethical considerations that come to mind about relating to agents or about privacy.