; File: questionnaires.doc., Updated 10/11/91 ; ATIS subject debriefing questionnaires ; [ATT] Includes subject data form and post-experiment questionnaire. ----------------- Subject Data Form PLEASE FILL IN THE FOLLOWING INFORMATION: Last Name: First: Middle Init.: Age: Sex: Phone number: Where did you spend your childhood years (until age 13)? What is the native language of your parents (if English, please specify American or British)? Mother: Father: Do you use a computer at work or school? How often? Thank you for your participation!! Subject Number: Date of Recording: Time of Recording: ATIS Version #: Wizard Name: ----------------- Questionnaire Given to Subjects We are very interested in your reaction to the system and would appreciate your answering a few questions about your impressions. KEY: 1: Definitely no 2: Probably no 3: Maybe 4: Probably yes 5: Definitely yes Please circle your answer as indicated. Part 1: Solving the problem I was satisfied with the travel plans I made with the Air Travel Planning System. 1 2 3 4 5 The answers were provided quickly enough. 1 2 3 4 5 I could get the information I needed. 1 2 3 4 5 I enjoyed using this system because I disliked using this system because Part 2: Spoken language understanding The Air Travel Planning System understood me most of the time. 1 2 3 4 5 When the system didn't understand me, it was easy to figure out another way to get my idea across. 1 2 3 4 5 I focused most of my attention on solving the problems rather than trying to make the Air Travel Planning System understand me. 1 2 3 4 5 Was the dialogue between you and the computer natural, that is, like you were talking with another human being? 1 2 3 4 5 Part 3: About the Computer Voice The computer voice was always understandable. 1 2 3 4 5 How close did the voice sound to a human voice? 1 (machine-like) 2 3 4 5 (human-like) Did you grow tired of listening to the voice? Yes No Part 4: About the experiment I was aware that a human was performing the speech recognition in this system. 1 2 3 4 5 The travel planning problems suggested were realistic for the most part. 1 2 3 4 5 Part 5: General Comments ; ; [CMU] ##NOTE: Original is formatted in 12pt Helvetica, bolding, etc. ATIS Participant Questionaire Date: ____________________ Scenario Series: __________________ Name: ________________________________________________________ 1) Were you able to get the travel information you needed from the ATIS system? _____Yes ______No _______Sometimes/Maybe If not, please explain: ___________________________________________________________ ___________________________________________________________ 2) Were you satisfied with the way the information was presented? _____Yes ______No _______Sometimes/Maybe If not, please explain: ___________________________________________________________ ___________________________________________________________ 3) Were the answers provided quickly enough? _____Yes ______No _______Sometimes/Maybe 4) Did the system understand your requests the first time? _____Yes ______No _______Sometimes/Maybe 5) If the system did not understand you, could you easily find another way to get the information on a later try? _____Yes ______No _______Sometimes/Maybe If not, please explain: ______________________________________________________________ ______________________________________________________________ 6) Do you think a person unfamiliar with computers could use the system easily? _____Yes ______No _______Sometimes/Maybe If not, why not? ______________________________________________________________ ______________________________________________________________ 7) Were you concentrating more on: (select one or more) _______ Solving The Problem _______ Getting the System to Understand _______ Understanding the Scenarios _______ Understanding the Instructions 8) In what ways did speech make this system easy to use? ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 9) In what ways did speech make this system difficult to use? ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 10) Did you enjoy using this system? explain briefly. ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ ______________________________________________________________ 11) Did you at any time think there might be a human, and not a computer responding to your requests? ______________________________________________________________ ______________________________________________________________ If you have any questions or other comments we would like to know of them!! Please use the back of this page to write them down. Thank you for taking the time to participate in this study!!! ; ; [MIT] We are very interested in your reaction to the system and would appreciate your answering a few questions about your impressions. KEY 1 Definitely no 2 Probably no 3 Maybe 4 Probably yes 5 Definitely yes Part 1 -- Solving the problem: 1. I was satisfied with the travel plans I made with the Air Travel Planning System. 1 2 3 4 5 2. The answers were provided quickly enough. 1 2 3 4 5 3. I could usually get the information I needed. 1 2 3 4 5 4. I enjoyed using this system because... Part 2 -- Spoken language/system understanding: 1. The Air Travel Planning System understood me most of the time. 1 2 3 4 5 2. When the system didn't understand me, it was easy to figure out another way to get my idea across. 1 2 3 4 5 3. I understood the Air Travel Planning System most of the time. 1 2 3 4 5 4. I focused most of my attention on solving the problems rather than trying to make the Air Travel Planning System understand me. 1 2 3 4 5 5. I focused most of my attention on solving the problems rather than trying to understand the Air Travel Planning System. 1 2 3 4 5 Part 3 -- About the experiment: 1. I was aware that a human was performing the speech recognition in this system. 1 2 3 4 5 2. The travel planning problems suggested were realistic for the most part. 1 2 3 4 5 Part 4 -- General Comments: ; ; [SRI] AIR TRAVEL INFORMATION SYSTEM FOLLOW-UP QUESTIONNAIRE Thank you for participation in this study. We are very interested in your impressions of the system and would appreciate your answers to a few questions about your session. YES SOMETIMES/ NO NO MAYBE OPINION 1. Were you able to get the travel y s/m n no-opin information you needed? 2. Were you satisfied with the way y s/m n no-opin the information was presented? 3. Did the responses contain the y s/m n no-opin kinds of information you were seeking? 4. Were the answers provided quickly y s/m n no-opin enough? 5. Would you prefer this method to y s/m n no-opin looking up the information in a book? 6. Did the system understand your y s/m n no-opin requests the first time? 7. If the system did not understand y s/m n no-opin you, could you easily find another way to get the information on a later try? 8. I focused most of my attention on y s/m n no-opin solving the problems, rather than trying to make the system under- stand me. 9. Was the travel planning scenario y s/m n no-opin appropriate for a trial use of the system? 10. Do you think a person unfamiliar y s/m n no-opin with computers could use the system easily? 11. Do you think a human was y s/m n no-opin interpreting your questions? 12. What comments would you like to offer about the experience? 13. Could you please tell us where you were born and raised? 14. Is English your native language? 15. Is English your parents' native language? 16. If you have recently arranged travel plans, could you briefly describe what the constraints were so that we could use this as a future scenario? 17. How did you arrange your most recent travel plans? _____ asked a secretary to do it _____ called a travel agent _____ looked in the Office Airline Guide book _____ called an airline directly _____ other -- please explain: SPEAKER INFORMATION Speaker # _____ Please circle the appropriate choices. Gender: M F Please indicate where you lived between the ages of 4 and 14: New England Northern North Midland South Midland Southern New York City Western Army brat/moved around Outside USA (City:________________ Country:____________________) Birth Date:_________________________ Height:________________ Race/Ethnicity: White/European-American Black/African-American Native American Mexican-, Puerto-Rican-, Cuban- or other Hispanic-American Asian-American Other:____________________________________ Highest level of education completed: High school Associate degree Bachelor's degree (BS or BA) Master's degree (MS or MA) Doctorate degree (PhD, JD, or MD) Other:_____________________________________ Thank you very much! ; ;