Research
Overview of the Lab
The Computational Cognition and Perception Lab uses experimental and computational methodologies to study human learning and decision making in cognitive, perceptual, and motor domains. How is new information—whether its a friend's new phone number or a newly recognized visual feature that distinguishes the appearances of identical twins—acquired and how does it become established in memory? How is perceptual learning similar or different from cognitive learning and motor learning? How do people reason about environments with complex temporal dynamics? How do they choose actions in these environments so as to achieve a goal? To what extent are people's behaviors consistent with the optimal behaviors of computational models based on Bayesian statistics? Our research lab addresses these and other questions.
We are currently seeking new graduate students and postdoctoral fellows to join our lab. If you're interested, please contact .
Selected Research Projects
Perceptual Learning Based on Sensory Redundancy
Perceptual environments are highly redundant. People obtain information from many sensory modalities, including vision, audition, and touch. Individual modalities also contain multiple information sources. Visual environments, for instance, give rise to many visual cues, including motion, texture, and shading. We've been interested in how people take advantage of sensory redundancy for the purposes of perceptual learning. For example, a person might (unconsciously) notice that visual cues A and B provide consistent information about the shape of an object whereas cue C indicates a different shape. If so, then the person might conclude that cues A and B are reliable information sources, but that cue C is less reliable. The person can then adapt his or her sensory integration rule so as to place greater weight on the information based on cues A and B, and less weight on the information based on cue C. A person who learns to adapt his or her perceptual system as described here would be behaving in a manner consistent with an optimal statistical model based on Bayesian statistics known as an Ideal Observer.
An example of the type of stimulus we might
use in our experiments is illustrated in the figure on the right. It depicts a corrugated surface
at an orientation of 45 degrees. Subjects might perceive this surface using both visual and haptic (touch) sensory modalities. Visually, the
surface is defined using shading and texture cues. The subject can touch the surface using a virtual reality device (see below).
Effective Perceptual Learning Training Procedures
It has often been noticed that motor training in adults often leads to large and robust learning effects whereas visual training often leads to relatively small learning effects. To date, it is not known if the limited effects of visual training in adults are due to inherent properties of our visual systems or, perhaps, due to our immature knowledge of how to optimally train people to improve their perceptual abilities. It may be the case that if we learn more about how to train people's visual systems, then visual therapy in the near future could be as effective as physical therapy is today. Our lab is studying new ways of training people to perform perceptual tasks. One hypothesis that we are examining is that people will learn best when they are in environments that are as natural as possible. For example, people should not only be able to see objects, but also to touch and move objects. Sophisticated virtual reality equipment allows us to create experimental environments that allow people to see, touch, and hear "virtual" objects.
The University of Rochester has fantastic (and highly unusual) virtual reality equipment. The figure on the top left shows a subject using the "visual-haptic" virtual reality environment. In this environment, a subject can both see and touch objects. The figure on the bottom left shows a subject using the "large field of view" virtual reality environment. Subjects in this environment are presented with visual stimuli spanning a full 180 degrees.
Dynamic Decision Making
In many situations, a person has to make a sequence of decisions before achieving a goal. Examples include driving a car, playing chess, or
investing in the stock market. Decision making in these situations can be difficult because of complex temporal dependencies—a decision at
one moment in time may have implications for what decisions will need to be made in the future. We are interested in how people make decisions in
these settings despite uncertainties about the true state of the environment and the effects of their own actions. Do people make decisions in a
way that is mathematically optimal? If not, why not? Can we provide people with extra information so that they will make decisions in a more
optimal manner?
An example of a type of task that we have used in our experiments is illustrated on the left. There exists an object which can move along the horizontal dimension. The subject applies forces to the object using the computer mouse to move the object left or right. The subject's goal is to apply a sequence of forces so that the object moves to the target region as quickly as possible and stays in this region for as long as possible. It's a difficult task because small forces might move the object a small amount toward the target whereas large forces might cause the object to overshoot the target.
Computational Models Known as Ideal Observers and Ideal Actors
Ideal Observers and Ideal Actors are computational models based on Bayesian statistics that perform in an optimal manner. Ideal Observers make
statistically optimal perceptual judgments, whereas Ideal Actors make optimal decisions and actions. Our lab is interested in using techniques from
the machine learning and statistics literatures to develop new ways of defining Ideal Observers and Actors for interesting tasks. Once an Ideal
Observer or Actor is defined, we can run experiments with people using the same tasks. This allows us to compare human and optimal performances. Do
people perform a task optimally? If so, then we can conclude that people are using all the relevant information in an efficient manner. If not, then
we can examine why they are sub-optimal and what can be done to improve their performances.
One way in which we create Ideal Observers or Ideal Actors is through the use of Bayesian Networks. An example of a Bayesian Network is illustrated on the right. Bayesian Networks are useful for defining the relationships among a set of random variables, such as the relationships among scene variables (describing what is in an environment) and sensory variables (describing what is in a retinal image, for example).