Email: ilker DOT yildirim AT boun DOT edu DOT tr
In this project, we propose a Bayesian nonparametric model of multisensory perception based upon the Indian buffet process. We include a set of latent variables that learn multisensory features from unisensory data in our model. Benefiting from nonparametrics, the model is highly flexible, and makes few statistical assumptions. In particular, the number of multisensory features is not fixed a priori. Instead, this number is estimated from the data.We are applying the model to a real-world visual-auditory data set obtained when people spoke English digits. Our current results are consistent with several hypotheses about multisensory perception from the cognitive neuroscience literature. For example, the model acquired multisensory representations that were relatively sensory invariant. Also, the model obtained the statistical advantages provided by sensory integration. And lastly, we found that the model was able to associate unisensory representations based on different modalities.