Tomorrow's robots could learn a thing or two from the way that humans eye their world, say scientists at the University of Rochester's Center for Visual Science (CVS).
By watching how students handle toy building blocks, scientists are learning some surprising things about the way we see. And by endowing a robot's "eyes" with human characteristics, they are building a new breed of robot that can roll with the punches of a changing environment.
"What's so magical about human behavior is that it's so flexible and adaptable, while robots remain limited," says Mary Hayhoe, associate professor of psychology and a member of CVS. "Even something as simple as dimming the lights can cause major problems for a robot, though humans do just fine. There is something very different about humans that has not been captured in robotics."
Recently Hayhoe and colleagues studied the short, jerky eye movements known as saccades we make several times each second. They found that we rely on saccades, not memory power, when performing all kinds of tasks.
"Eye movements provide a really efficient way to do things," says Hayhoe. "Instead of keeping track of every detail in the environment, you use the world as your memory. You don't get information until you need it."
Hayhoe's computer science colleagues have already found that skills like saccades, not megabytes of memory, are the key to smart, versatile robots. In their work with the Rochester Robot, Rochester scientists have mimicked the way humans execute simple tasks. Instead of boosting processing power to enable the robot to image every detail of its environment -- a widespread approach which has yielded limited success -- Rochester scientists take an approach known as "active vision," where the robot uses sensory input from its environment to make its decisions. Giving the robot some of the flexibility humans have, such as programming the robot to move its camera eyes just as humans do, simplifies its tasks and enhances its abilities. With no remote control and in real time, the robot has dodged tennis balls, played checkers, manipulated children's toys, searched for and identified objects such as cereal boxes and stuffed animals, and performed a variety of tasks.
It's skills like saccades which have long been overlooked in robotics -- and even human -- research. Saccades are the fastest movements humans can make. Our eyes, with the help of half a dozen muscles, are constantly darting from word to word on a page or from face to face in a crowd. We make roughly 100,000 to 200,000 saccades a day, yet we don't even notice because they're so automatic. Saccades are vital to our ability to see -- they give us the ability to move the tiny part of our retina which sees the world with high resolution from target to target so fast that we have the illusion of seeing everything clearly.
Jeff Pelz, a graduate student in CVS, found that when people are asked to copy a pattern of building blocks, they don't rely on an image stored in their brain but instead on hundreds of saccades through which they compare the original with the duplicate they're making.
"You don't just look once at a set of blocks, and then replicate it," says Hayhoe, who is Pelz's adviser. "Instead, your eyes are constantly going back and forth, picking up little bits of information, and checking that information again and again. It's really quite elegant."
In the current set of experiments students use Lego building blocks to re-create simple designs. A sensitive eye-tracking device monitors eye movements by bouncing a beam of infrared light off the subject's retina. A camera hooked up to the eye tracker records exactly what the subject is looking at, and sensors on a glove track hand position. Subjects commit very little to memory, instead checking the pattern again and again for block position and color. When subjects aren't allowed to move their eyes, the same task takes three times longer.
"We use the world as an external store of memory, much like a computer has a memory," says Dana Ballard, member of CVS and professor of computer science. "Eye movements give us access to that memory. There's no need to commit to memory every detail in your environment; information is handy at the tip of your gaze."
Adds Hayhoe: "Traditionally, artificial intelligence has assumed that huge amounts of memorized information are necessary to do even the simplest tasks. But you don't do that. You don't build up an elaborate model of the world. You get the information when you need it."
Lego blocks may seem low-tech, but they're a big improvement over previous work. "Much of what we know about eye movements has come from oversimplified, artificial environments," says Pelz, who is also on the faculty of the Center for Imaging Science at Rochester Institute of Technology. "In previous experiments the head was locked into place, and two or three small points of light took the place of a real scene. But this is not how we live our lives.
"Lego blocks may seem unnatural and trivial, but compared to previous experiments they're state of the art."
The group recently assembled a virtual reality system and will monitor eye movements in that realm, where subjects "perform" more realistic actions such as driving a car but where scientists have strict control over what subjects see.
This work was presented recently at meetings of the Proceedings of Cognitive Science and the Association for Research in Vision and Ophthalmology. The work is funded by the Air Force Office of Scientific Research, the National Science Foundation, and the National Institutes of Health. tr