ࡱ> VXUa QAjbjbA]A] 2V+?+?B;@$$$8\ h<@1 f,!R#|T 6 $:Q$BR 01 $$@@$@@$Task Analysis for the use of ABC-Products and Information Access - S. Brennan (This is an excerpt from a task analysis done by a research group that was investigating the possibility of building a natural language interface to an database used by a marketing group in a large company. Names are changed or deleted to protect confidentiality) ABC-Products is a large database of product, marketing, and sales information. Information Access is an existing graphical interface to ABC-Products. The users of these are employees in a group in a large company. The three most important things in their business are: "Orders, orders, and orders." We compiled the following information by interviewing employees and by leaving them with a questionnaire to fill out whenever they used ABC-Products and Information Access (IA). I. Users a. Categories of users We've identified several populations of potential users. These are distinguishable on several dimensions, such as by how frequently they need to use ABC-Products, by what we can project as their tolerance for learning to use new technology, and by how they are evaluated in their jobs (that is, whether they are responsible upwards to top management, sideways to other less-specialized product marketing engineers, or outward to particular sales forces in the field). Here are the kinds of users we are aware of. 1. Financial analysts. These people are experts in the use of ABC-Products and Information Access. (1 or 2 per division; 10-12 company-wide) 2. Product marketing engineers and Market Analysts who specialize in using ABC-Products and its COBOL predecessor SOS to provide information to their groups (1 or 2 per division; 10-12 company-wide) 3. Product marketing engineers who manage products and sales areas (about 20 in the division we investigated, and hundreds company-wide) 4. Managers of product marketing engineers and financial analysts 5. Executives. There are about 150 high level marketing executives in the company. These are the decision makers who look at only the graphs of sales and marketing information that are generated automatically on a daily basis by the Information Access program. These people go to the specialists in their marketing groups when they need customized views on ABC-Products or materials to take into presentations. We should probably avoid trying to support these users, because they are likely to be more busy, demanding and inflexible than the others, and may not be best supported by an NL interface. For the time being, we will concentrate on categories 1-3 as our potential group of users. b. User profiles 1. R.A. is a financial analyst, and he is responsible for feeding information upward to the high level marketers/decision makers and assisting other users who need to use Information Access in more sophisticated ways. He is probably our most expert user of Information Access, and has trained his corresponding members of other divisions. He helps out D.W (see below) when she has a question. 2. D.W. is a product marketing engineer who specializes in getting information out of ABC-Products and the old COBOL system. She serves the other product marketing engineers as the local expert in this, in addition to preparing information for the division's quarterly reviews and other sales and marketing presentations. She regularly generates reports, both batch and ad hoc, to provide market information for her group of 20. D.W. must understand the orders and provide interpretations and marketing forecasts based on them. She generally accesses ABC-Products once or twice a day. 3. Q.S. is a product marketing engineer who has a background as a programmer and belongs to the ~20% of technically-oriented product marketing engineers who use ABC-Products via Information Access. Q.S. creates promotions and suggests marketing strategy for his group, using subsets of Information Access/ABC-Products. II. Uses. As a first pass, the tasks seem to fall into the following categories: - Monitor present state/ Information filtering and presentation - Troubleshooting/ Discover properties accounting for current state - Forecasting/ Look for indications of possible future state - Develop Strategies/ Look for ways to improve outcomes a. High level tasks: 1. Working on "big deals." This means looking at sales information with respect to a particular customer: what they've ordered, what else they should be convinced to order, keeping close track of a customer just before the close of a month, etc. 2. Performance measurement of sales/marketing employees, managers, groups, divisions. Managers want to know how everyone under them is doing, and employees likewise want to compare their own performance to others'. 3. Financial management at the product level. This use could be called "behavioral," meaning that while there is a big demand for visually displayed, shared data on how a product line is doing, this may not directly affect marketing decisions. The graphed data becomes a motivating force for the marketing group, or the justification for tracking performance more closely, or the evidence for an evaluation of success. 4. Identifying and monitoring irregularities. Patterns that display dollars over time can instantly show potential problem areas (for instance, sudden cancellations in orders in a particular area), which managers or product marketing engineers can then pinpoint in more detail and track. 5. Fixing snafus. Product marketing engineers sometimes mount "road shows" during which they go out to sales offices and try to motivate those who aren't doing too well and get those sales people who ARE doing well to share their secrets. 6. Forecasting. To set future targets, financial analysts look at the patterns of data form previous years with respect to a particular product line. 7. Developing marketing strategies. Sometimes a PME puts together a promotion which includes a package deal to sell something new (for instance, LAN), and in doing so they look at what customers have already ordered to figure out how to attach the new product to what might be ordered next. 8. Making resource allocation decisions. This is useful in figuring out whether a particular effort such as localizing software for a foreign country has paid off in orders. 9. Gain better understanding of customers, orders, 3rd party vs. proprietary, field engineer success or failure. b. Possible additional uses: 10. Re-engineering products by development engineers. YYY could be used to make projections concerning the potential demand for products that don't exist yet. This use makes the assumption that R\&D could be driven by what the market wants today, which may be a reasonable assumption on the divisional level. 11. Field use by individual sales people. Sales people might like to keep tabs on their own productivity. But right now ABC-Products does not go down to the level of detail that would include individual order numbers. III. Tools The ABC-Products database can be queried using SQL queries or the old COBOL system. To use either of these methods requires knowing a programming language (SQL or COBOL) and exactly how the database is organized. Information Access is a menu-based interface to ABC-Products that enables people without this kind of knowledge to retrieve information, graphcs, and charts from the database. Only the most frequently used kinds of queries, graphs, and charts are supported in this interface. IV. Task Breakdown We're in the process of taking these high level tasks, along with the specific examples of high level goals and individual queries that we elicited from the ABC-Products users we talked to, and breaking them down into lower level task components which could be the basis for a task language. Currently, the system can be used in two ways: a. Batch processing. 20 reports are generated automatically with up to date information each day. These reports detail the orders for each of the regions, then detail the individual products. Users toggle through the set of reports with the touch of the button. (There is no random access, but this would be desirable.) This is the only interface that the high-level executives ever see. Ad hoc queries are performed for them by others. The reports include graphs, bar charts, and pie charts. b. Ad Hoc queries. An Information Access interface is used to generate ad hoc ABC-Products reports. This consists of several menus, each with many choices, the selections of which combine to specify the desired request. This interface is apparently difficult for novice users so that the full functionality of Information Access is available only the few experts. The trick is knowing not only which fields to select, but also what is in the database to look for. Here are some sample queries collected by users, in response to our questionnaires. The general goal is specified first, followed by a more specific goal expressed as a query in ()s. a. Goal: to recognize unusual orders ("How can I go to 108% of quota?" - RA) "Show me any order over x." "Show me orders from over fy88." b. Goal: tracking a product in a region ("Where are the problems?" - QS) "Show all HPDesk orders in Taiwan in Feb." "Tell me all the cancellations in Latin America in Feb." "I want all activity for March, for ." "How many BSP's are on Spectrums?" c. Goal: negative filtering ("What would sales be without x?" - QS) "I want everything for BSP except for . d. Goal: determine sales opportunities ("How to set up this promotion?" DW) "Who bought 950's without BQE?" "Who sold them?" (Note: this kind of query would be really valuable if answer included order numbers, but this data is not available in ABC-Products) e. Goal: to track progress of a group ("How are we doing against quota?" - DW) "Give me fiscal year to date quota performance." or "Give me the marketing group's performance for BSP for 1988 by actual $'s and quota $'s. (Here, the desired answer would be a table with columns for Actual Order Dollars, Quota Dollars, and Time Frame, and the data would be grouped by division, product, sector, region, etc.) f. Goal: track progress of conjunction of products. (DW) "Give me ytd performance for Information Access and BSP." or "Give me ytd performance for end user applications." g. Goal: tracking customers (DW) "Who were the top 10 customers of for 1st quarter?" "Show their primary product groups and dollars as well." (In this example, 'primary' specifies that only the top product group is to be shown for each customer, yet the total dollars from all product groups would be shown. A customer may buy many products, but only the product group with the largest order is interesting. e.g. 10 System 950's are more interesting than 1 COBOL compiler.) h. Goal: regional analysis by product ("What products will help someone make his new stretch goal? - requires many steps - DW) "What are the $ for March for each region, grouped by product?" or "What are the top products for each region?" "Who is selling them?" i. Goal: troubleshoot/ identify cause when orders take a sudden down turn (RA) e.g. Orders in the Midwest Region are down, and you see that (product N) has taken a big drop there. You can then find that a particular big company canceled its $2M order and who the sales rep was so you can call the sales rep and find out what happened. "Show me the orders in the midwest for this month vs. quota." "Show me the Midwest orders by product for this month." "Of those orders, show me all BQE orders that were canceled, including dollars, sales office and field engineer." j. Goal: find all orders with old product # in order to offer these customers a new discount (unusual goal - QS. To accomplish this, he had to use SOS and cross-reference all the old orders, since order numbers are not in ABC-Products.) V. Evaluation The following information is not included in the ABC-Products database; however, many people said they wished they had this information. -order numbers -relationships of the different types of data -comment fields for unusual cases -forecasting information The old COBOL system is very difficult to use, and the SQL queries are not much easier. Information Access is difficult for occasional users to deal with. When occasional users have ad hoc queries, they cannot get the information they need immediately. We are in the process of estimating the amount of time spent on generating different kinds of queries, like the ones above. VI. Recommendations To improve the system, we may want to consider doing more than providing an NL interface; the system's functionality should be improved. People want to have individual order numbers and inventory information linked to the information in the ABC-Products database. An NL interface would very likely provide greatly improved access for occasional users; it might also provide more convenient access for constant users. Efficient use of ABC-Products/Information Access may also require integrating an NL interface with: 1) Search Keys The use of search keys can mean the difference between 1 minute and 20 minutes for a query. An estimate is given at the start of the search of how long it will take so that users may abort the query if it is excessive. 2) Save Tables The result of a query may be stored to isolate the subset of data to be manipulated, then used repeatedly to follow up with the actual formatting requests since the same data (or common subsets) is often presented multiple ways. Using the ABC-PRODUCTS database Name:______________________________________________________________________________ Task:_______________________________________________________________________________ What were you trying to do?______________________________________________________________ ___________________________________________________________________________________ How often were you able to get the information you needed? How often was your first attempt successful? On average, how often did you have to rephrase a particular request before you got the information you needed? How many times did you have to give up because you weren't able to get what you needed? Did you develop any special strategies that proved useful in dealing with the system? Were you satisfied with the way the information was presented? Please rank your preference (1=most favorable, 4=least). If you had a choice as to how to get this kind of information, would you prefer to: ______use this system, as is? ______use a system like this, with the following changes? (describe briefly) ____________________________________________________________________________________ __________ask another person? __________use a different kind of system? (describe briefly) ____________________________________________________________________________________ What comments do you have about using this system?  PAGE 1 @OX; [ CTp{?\q!}!k#~#+~,a-.|/0#51577AABACAJAKALAMAQAhXmHnHujhXUhXhX6CJOJQJhXCJOJQJhX5CJOJQJ#NOXY: ; D [ \ a b F G BCTU-BAPA-.op{HLMno_`>>?\] q!r!}!j#k#~##$$$&&((T)U))))))?*k******A+y+z++++~,,,-B-a-..W.....2/B/|/00J111111%3d3t33332434"5#51525555676867777889999:::;;;.</<<<<<<5=6=7=r=s=t====>>>l>m>n>n>>>>???????@@Y@Z@x@y@@@ A AAAABANAOAPAQA$a$000P:pX/ =!"#$`%D@D NormalCJOJQJmH sH tH uDA@D Default Paragraph FontRiR  Table Normal4 l4a (k(No List4 @4 Footer  !4@4 Header  !Q;V !z!z z z!z/j*5Q;@ NOXY:;D[\abFG B C T U -.op{HLMno_`>?\]qr}jk~ ""T#U######?$k$$$$$A%y%z%%%%~&&&'B'a'((W(((((2)B)|)**J+++++%-d-t----2.3."/#/1/2////07080111122333444555.6/666666576777r7s7t7777888l8m8n88889999999::Y:Z:x:y::: ; ;;A;B;N;R;0ˀ0000000000000000000000000000000000000000000000000000000000000000ˀ0ˀ00000000000000000000000000000000000000ˀ0ˀ000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000ɀ  QA"->*19n>QA#%&'()*PA$ !FMJ$P$$$a(d(((++A-H-B;M;O;R;FNOF t $ .>#,j9rv ] 7"A"K"""P#R#T#####'''B'D'F''((2)7)=)B)|)))#****.+0+8+J+d-i-o-t-------.2.//////00*070B4g4r444(5,55555.6/6365666666999999999:Z:d:g:x:y::::::B;M;O;R;::::::::::::::::::::::::::::::::::::::A;R;@q97Q;@ @UnknownGTimes New Roman5Symbol3 Arial9Palatino9New York;Helvetica"!!0h+xxd<+{]@Task Analysis for the use of ABC-Products and Information Access Susan Brennan Susan Brennan Oh+'08$ <H h t 'DTask Analysis for the use of ABC-Products and Information AccessSusan BrennanNormalSusan Brennan2Microsoft Word 11.3.8@@1@10GPICTzb HHb bHHbb         ^NsZZ^cJRZkZcVccVZZVZVkZZc^ZkZZRRVckZZZccVccsVwVo{Z^Z   ^wNscR^g9g9kZ^cRV^c:^ccVcc^g9g9ckZg9RckZg9^Vc^cwcF1cJR^g9ccVJRccc^cckZg9Vg9cwcg9Rcc^ccVcw ^^c1g9RcZ^g9Rcc^cccg9ccg9g9cV^kZRRcckZcc^Rc^kZcg9g9JRZc^g9NscVcZg9wc^cg9kZcg9ccg9cc^kZccRg9g9cRg9^cVRg9   ZVJo{Z^cZZ^^o{^Z^kZ^^g9ZJRo{Z^VRVo{ZkZR^wJRg9^Zg9RZo{VV^ZZwkZZZNsVkZo{ZZcNsg9^g9^ZZg9o{ZR^ ^o{^^wZVZkZZo{JRVZZVVsZR^cZw^wZ^g9Z,kZwNsZ^o{ZVVZcZNso{Z^VZ^^o{Z^^o{^ZcZVV^ZcZwZkZwo{ZZNVRg9^VVZo{kZJRkZkZwR^g9RkZwZo{^g9RZg9^o{ZsVg9Zo{Nso{Zo{g9ZZZo{Rg9wZkZZcZVVRkZwZZZVR^RZNsVJRZ )cRkZRcRVkZZ^ZVVZcZg9^ZcZ^co{Z^kZVVNskZg9^^^Ns^ZVo{ZRo{Z^kZVwZZcZo{kZwZNsg9ZZVVs CZR^cZ^ZkZZZo{VVkZRZZccZg9wkZZw  kZRcVkZc ) ^V^cVZZkZkZF1cVV   ZkZZcRZRo{Rg9ZcZo{^^Z^ ZZg9Z^ZkZZZkZV^g9ZZw^wZNs^o{Zo{ZR^^Zw^Zg9ZcZo{^NsRZg9RZg9c^cw^g9cwZVwo{JR^ZZkZckZwZcZ9kZZ^g9ZZVVsZR^cZwcVw^kZVVc^Z^o{ZRV^g9kZwRo{kZZR^ZNsZo{Z^o{ZR^kZZ^g9Z ZZVg9NswZ^cZZcwZVg9wZNso{ZZc^0kZZRkZwZo{Rg9wkZZkZZwVwZkZVo{kZwNs^o{Zo{ZRZZg9o{Z^^VZo{g9kZZkZZ^ 2VVZ^JRVRZkZco{F1ZRkZZZkZwRZg9^^ZcVVZ^R^ckZVVo{ZJR^ZZ^RZ wZo{Z^NsZo{kZZ^Z^^o{g9^Zg9 e+RcNsRkZwZRwwRZo{Z^o{ZkZVo{ZNs^g9Zo{g9VV^o{JRVZo{ZZw  *g9VR^ZcV^Z^cg9Zw^wZg9JRZZ^Z^o{ZZg9^RkZ^ZkZwZ^g9ZZZVo{Z^Vg9^ZkZZZo{VV^Z ccNsg9wwg9RZ^R 7ZZo{ZVg9R^ZcZVV^ZcRwo{Zw   ZVo{Z^VAo{ZkZRJRZ^o{ZZRg9^ZNskZVZRZ^cg9ZVwZg9^ZcVVZR^g9R^ZVVsZo{Z^cZ^Zo{kZg9VVRVZC^o{ZZRNso{VVRkZZ^o{Zco{ZRZo{VV^ZZkZZkZwRo{^R^^g9wg9Zo{Z^RZZRZwg9R^ZcZVV^ZcRkZZw   o^Vo{Z^V(o{ZkZRJRZ^o{ZZRg9VwZVVZ^^Z^o{Z^cZ^Zg9^Zg9^o{Z^g9 s2w^Z^RRRg9ZZo{g9RZVVRcZZV^kZZw^Zw^Zo{Zg9cZVV^ZcRwo{Zw  g^JRRZ^^Z^o{Z^Vo{ZkZR^ZZ^RZRg9^ZR^ZcV^Z^RkZg9   J^JRg9ZNskZZZg9ZwZo{Z^o{Z^JRkZg9^RwVwZcZVo{^o{F1kZ^Zg9Ns^kZZZ^Zg9ZcZVV^Zcw^wZg9JRo{Zg9ZZcZo{ZZKVVRg9VwZZZ^kZZZckZwZ^o{^Zg9Zg9^ZRZVVo{ZkZR^RZo{VVkZRZkZw^Ro{Z^ZZo{^R^^kZF1VNs^cZZ^ ^Z^g9ZckZwZkZZZo{VV^Z5ccNsg9^o{Z^o{JRkZ^Vg9Z^ZZ^Z^ZkZZkZVg9^ZcVo{g9Zo{ZkZwZo{VVo{ZkZR^^o{Z^^g9 VwZZkZVcZ c^g9RVVZcRVcZV/o{Z^Vg9ZNso{^kZZo{Vg9RkZ^RZkZZ^o{NsZZkZVRZcZkZZg9wZ^^o{Z^c  ^cRkZo{Z^Rg9^?Zo{kZkZVg9JRg9ZZwZc^g9ZkZwZNso{Zo{ZckZZZVRo{Z^RwZVVZR^^ZRZg9o{ZkZw^ZkZwZ_RwZZw^ZVVcZZkZZZZg9^Zo{Rc^ZZcRkZZo{^cZw   ZRkZwZkZNsRZRF1VVkZ*cZcZZkZo{JRRZc^kZJRZo{Rg9g9V^g9Z^o{^ZkZZkZV^R^^Z^g9ZZw   % RcVkZ^kZZkZVc  Cg9NswZwZ^R^ZcV^Z^cZwJRwRg9o{ZRZZg9o{ZZo{ZZR^RZNsVkZRZ^^Vw^o{RkZwZwVwZcZ &VVo{ZkZRg9wZZ^ZZVVZo{RZ^g9g9ZkZR^Zg9Zo{^g9Rg9VwZ kZZ^g9ZRZo{VV^ZccNsg9RVRo{ZNs^wZkZZ^R NVZccRZZ^o{Z^cZ^o{VRg9kZZg9JRo{Rg9Zo{ZkZZZo{VVJRZZcNsg9g9ZZw^g9kZo{^F1wZcZo{ZRZZR^VRVo{Zo{g9ZoRwZo{ZZo{Zg9NsZwZ^g9Z^F1wVkZwg9ZVwVwZZg9wZwR^^JRg9^ZZw   LZJRwZg9o{g9^^o{Z^ckZVVo{ZJR^Z^RZZo{VwZg9^ZcVVNso{Z^ZkZJR^RRVVkZRZZ^kZZZVVsZR^ckZRZkZwZZ JVVRVZg9cg9RVkZVwZg9ZRRkZwZZkZVo{^o{Z^ckZVVo{ZJR^ZZ^RZo{Rg9g9ZZc^Zg9JRo{JRkZwZwR^o{^ZZkZZJ^o{Z^^o{F1RZNsVkZo{ZZo{kZwZZo{^ZZg9^^VkZZZo{ZcRVwRZZkZVo{g9^ZRZVo{^o{RR^^o{Zg9ZJRkZRZg9RwZ o{Z^Z^ZEo{^Rg9o{ZJR^g9wZkZw^Vw^Z^wZcwR^o{Zco{ZVo{^o{BRVVkZRZZo{wZVo{Z^^ZZRwNsVkZwVVg9kZ C^ZRg9RZkZwZZo{Zo{RZ^Rco{ZRkZR^RkZ^^Zg9^ZVVo{ZkZR^Zo{ZF1kZg9^g9ZZZkZwZVkZVwZ^Zo{^c I^ccF1Zg9ZVVsZR^cZ cZZo{kZVVZ^^cw             skZ        ՜.+,0< hp  'Stony Brook Universityh< ATask Analysis for the use of ABC-Products and Information Access Title  !"#$%&'()*+-./0123456789:;<=>@ABCDEFGHIJKLNOPQRSTWRoot Entry FnY1Table,$WordDocument2VSummaryInformation(?hDocumentSummaryInformation8MCompObjXObjectPoolnn FMicrosoft Word DocumentNB6WWord.Document.8