Lots of computers are smart enough these days to hear what you say and do what you ask: Voice recognition software for computers is a rapidly expanding field in computer programming.
But an experimental program developed at the University of Rochester makes the leap from "voice recognition" to "intention recognition," taking computers farther than ever before in the direction of understanding normal conversation. Scientists will unveil the program at next week's meeting of the Association for Computational Linguistics in Santa Cruz, Calif.
In the Trains Project, a computerized personal assistant named "Phenelope DuJour" helps a human engineer route trains through cities in the Northeast. The engineer and Phenelope discuss anything that could affect the best way to route a train, from heavy traffic, to flooding, to a visit from a presidential delegation.
"This is much more than speech recognition and synthesis," says James Allen, the John H. Dessauer Professor of Computer Science, whose system uses a commercial speech synthesis system and a speech recognition system provided by Carnegie Mellon University for those functions. "Speech recognition, while important, is only the beginning. The critical missing component to a conversational computer has been understanding and reasoning." Presenting the Trains Project with Allen are graduate students Eric Ringger and Teresa Sikorski and programmer Brad Miller.
Up until now, voice-recognition systems have required humans to adapt to computers, rather than the other way around. "Many speech recognition systems work pretty well if you stay within the constraints of the system," says Allen, "but we want a computer to do more than just pose a set of questions. Let's say you want to book an airplane trip by computer. Callers should be able to say, 'I want to fly from Detroit to Washington on July 4, with my mother, and she wants an aisle seat, and we want to fly at night and get the cheapest fare.' This is more like we really communicate."
But the problem with most systems is that many ordinary sentences are confusing to a computer. "My favorite is: 'Do you have the time?'" says Allen. "You could be asking what time it is. It could be a yes-or-no question. Or it can be intended as a gentle reminder that it's getting late."
Allen's team has taught Phenelope to deal with such ambiguous phrases. If the human manager says, "Take me to Chicago," for instance, or "New York is slow -- let's try Philadelphia," Phenelope usually interprets the sentence correctly or asks the speaker to clarify, enabling her to carry on a dialog where other computer systems falter.
While no one is predicting the demise of pop-up menus, icons, the mouse, or other computer-centered forms of communication, the work does show that it's possible to create a conversational computer that can be used by novices.
The team is now working on the next version of Trains, where the assistant will work in a more complex world of trains, planes and trucks that will shuttle boxcars and cargo between cities. The project, to which research associate George Ferguson has also contributed, is funded by the National Science Foundation, the Navy, and the Advanced Research Projects Agency.
Allen has wanted to be able to just sit down and talk to a computer ever since he was a graduate student at the University of Toronto 20 years ago. "That's the dream: To communicate with a machine as effortlessly as you do with another person." Allen is the author of Natural Language Understanding, the leading textbook for graduate students in the field.
tr Sample verbal dialog exchanged between Phenelope and a new user
Despite some obvious problems recognizing the human speaker's words -- a common problem among speech recognition systems that try to understand many different words from many different people -- the text below shows how the computer is still able to carry on a dialog and assist the human manager routing trains throughout the Northeast.
Phenelope: OK. I think I'm ready to start.
Human (as heard by computer): OKAY LET'S SEE CONTAIN FROM DETROIT TO WASHINGTON. Human (words actually spoken): (Okay. Let's take a train from Detroit to Washington.)
Phenelope: What route would you like to get from Detroit to Washington?
Human: LET'S GO VIA TOLEDO AND PITTSBURGH. (Let's go via Toledo and Pittsburgh.)
Phenelope: (displays proposed route) The terminal at city Scranton is delaying traffic due to localized heavy winds. The terminal at city Baltimore is delaying traffic due to localized heavy winds. An additional five hours will be needed to travel through them due to decreased visibility.
Human: NOW LET'S TAKE THE TRAIN FROM DETROIT TO WASHINGTON D S_X ANY. (No. Let's take the train from Detroit to Washington via Cincinnati.)
Phenelope: (displays corrected route) OK.
Human: OKAY IT THAT'S OKAY NOW. (Okay that that's okay now.)
Phenelope: Pardon Me?
Human: UP OKAY NOW LET'S TAKE THE TRAIN FROM MONTREAL TO LEXINGTON. (Okay. Now let's take the train from Montreal to Lexington.)
Phenelope: I need help choosing a route from Montreal to Lexington.
Human: LET'S GO P_M TO TRY. (Let's go via Detroit.)
Phenelope: (displays a route to Detroit but not on to Lexington) Is this OK?
Human: YES NOW LET'S GO TO LEXINGTON (Yes now let's go to Lexington)
Phenelope: Yep.
Human: OKAY NOW ADD STATE THE LAST TRAIN AND GO FROM ALBANY TO O_O'S. (OK now let's take the last train and go from Albany to Milwaukee.)
Phenelope: No. What city are you trying to have the engine at Montreal arrive at, if I caught your drift? I don't understand where we are sending the engine at Albany to.
Human: ME A JET ADD ALBANY NEEDS TO GO TO MILWAUKEE. (The engine at Albany needs to go to Milwaukee.)
Phenelope: Tell me a route to use to get from Albany to Milwaukee, please.
Human: I'D GO VIA BUFFALO. (I'd go via Buffalo).
Phenelope: (displays a route through Toronto and Buffalo) Yes.
Human: GO B_X SYRACUSE AT BUFFALO. (Go via Syracuse and Buffalo.)
Phenelope: (displays the routes chosen thus far) Yeah.
Human: THAT'S COULD I CAN. (That's great. I'm done.)
Phenelope: Hey, its the programming.
Human: I_NEED DONE. (I'm done)