Computational Cognition and Perception
Robert Jacobs
The Computational Cognition and Perception Lab uses experimental and computational methodologies to study human learning and decision making in cognitive, perceptual, and motor domains. How is new information—whether its a friend's new phone number or a newly recognized visual feature that distinguishes the appearances of identical twins—acquired and how does it become established in memory? How is perceptual learning similar or different from cognitive learning and motor learning? How do people reason about environments with complex temporal dynamics? How do they choose actions in these environments so as to achieve a goal? To what extent are people‚s behaviors consistent with the optimal behaviors of computational models based on Bayesian statistics? Our research lab addresses these and other questions.
Selected Research Projects
- Perceptual Learning Based on Sensory Redundancy: Perceptual environments are highly redundant. People obtain information from many sensory modalities, including vision, audition, and touch. Individual modalities also contain multiple information sources. Visual environments, for instance, give rise to many visual cues, including motion, texture, and shading. We‚ve been interested in how people take advantage of sensory redundancy for the purposes of perceptual learning. For example, a person might (unconsciously) notice that visual cues A and B provide consistent information about the shape of an object whereas cue C indicates a different shape. If so, then the person might conclude that cues A and B are reliable information sources, but that cue C is less reliable. The person can then adapt his or her sensory integration rule accordingly. A person who learns to adapt his or her perceptual system as described here would be behaving in a manner consistent with an optimal statistical model based on Bayesian statistics known as an Ideal Observer.
- Effective Perceptual Learning Training Procedures: It has often been noticed that motor training in adults often leads to large and robust learning effects whereas visual training often leads to relatively small learning effects. To date, it is not known if the limited effects of visual training in adults are due to inherent properties of our visual systems or, perhaps, due to our immature knowledge of how to optimally train people to improve their perceptual abilities. It may be the case that if we learn more about how to train people's visual systems, then visual therapy in the near future could be as effective as physical therapy is today. Our lab is studying new ways of training people to perform perceptual tasks. One hypothesis that we are examining is that people will learn best when they are in environments that are as natural as possible. For example, people should not only be able to see objects, but also to touch and move objects. Sophisticated virtual reality equipment allows us to create experimental environments that allow people to see, touch, and hear virtual objects.
- Dynamic Decision Making: In many situations, a person has to make a sequence of decisions before achieving a goal. Examples include driving a car, playing chess, or investing in the stock market. Decision making in these situations can be difficult because of complex temporal dependencies -- a decision at one moment in time may have implications for what decisions will need to be made in the future. We are interested in how people make decisions in these settings despite uncertainties about the true state of the environment and the effects of their own actions. Do people make decisions in a way that is mathematically optimal? If not, why not? Can we provide people with extra information so that they will make decisions in a more optimal manner?
- Computational Models Known as Ideal Observers and Ideal Actors: Ideal Observers and Ideal Actors are computational models based on Bayesian statistics that perform in an optimal manner. Ideal Observers make statistically optimal perceptual judgments, whereas Ideal Actors make optimal decisions and actions. Our lab is interested in using techniques from the machine learning and statistics literatures to develop new ways of defining Ideal Observers and Actors for interesting tasks. Once an Ideal Observer or Actor is defined, we can run experiments with people using the same tasks. This allows us to compare human and optimal performances. Do people perform a task optimally? If so, then we can conclude that people are using all the relevant information in an efficient manner. If not, then we can examine why they are sub-optimal and what can be done to improve their performances.