BCS 512: Lecture Schedule

Only students who are enrolled in the course may access the readings online. You must be logged into Blackboard to download the course readings.

A rough outline of the course is as follows:

Class 1: Introduction/Organization Introduction to computational modeling Introduction to Hebbian learning: Readings: "The Appeal of Parallel Distributed Processing" by McClelland, Rumelhart, and Hinton
Class 2: Linear algebra Supervised Hebbian learning: Readings: "An Introduction to Linear Algebra in Parallel Distributed Processing" by Jordan
Class 3: Expectation operator, mean, variance, and covariance Unsupervised Hebbian learning Principal components analysis Singular value decomposition: Notes: "Variance, Covariance, Correlation, and Correlation Coefficient";; "Principal Components Analysis";; "Principal Components Analysis and Unsupervised Hebbian Learning"; Distribute homework #1
Class 4:: Readings: "Localized Versus Distributed Representations" by Thorpe;
"Distributed Versus Local Representations" by van Gelder;
"Eigenfaces for Recognition" by Turk and Pentland;
"An Introduction to Latent Semantic Analysis" by Landauer, Foltz, and Laham
Class 5: Basic probability theory: Notes: "Lecture on probability"; Readings: "Probability Theory & Classical Statistics" by Lynch
Class 6: Bayesian networks: Notes: "Conditional Independence, Dependency-Separation, and Bayesian Networks"; Readings: "A Primer on Probabilistic Inference" by Griffiths and Yuille
Class 7: Linear regression Optimization and gradient ascent Maximum likelihood estimation Consistency, efficiency, minimum variance estimators Bias/Variance trade-off Bayesian estimation: Notes: "Gaussian, Bernoulli, and Multinomial Distributions";; "Maximum Likelihood Estimation";; "Optimization, Hill-climbing, and Maximum Likelihood Estimation";; "Bayesian Estimation"; Readings: "Tutorial on Maximum Likelihood Estimation" by Myung
Class 8: Statistical estimation Bayesian decision theory: Notes: "Bayesian Statistics: Normal-Normal Model";; "Bayesian Statistics: Beta-Binomial Model"; Readings: "Basics of Bayesian Statistics" by Lynch;
"When a good fit can be bad" by Pitt and Myung
Class 9: K-Means clustering, Gaussian mixture models, Kohonen networks, hierarchical clustering Multi-dimensional scaling, mixtures of experts: Notes: "K-Means Clustering";; "Mixture Models";; "Hierarchical Clustering";; "Mixtures-of-Experts"; Distribute homework # 2 | Gaussian Data | Vowel Data
Class 10:: Readings: "George Miller's Data and the Development of Methods for Representing Cognitive Structures" by Shepard;
"Properties of Synergies Arising From a Theory of Optimal Motor Behavior" by Chhabra and Jacobs;
"A Global Geometric Framework for Nonlinear Dimensionality Reduction" by Tenenbaum, de Silva, and Langford;
"Nonlinear Dimensionality Reduction by Locally Linear Embedding" by Roweis and Saul
Class 11: Temporal dynamics and Hidden Markov Models: Notes: "Hidden Markov Models"; Readings: "An Introduction to Hidden Markov Models" by Rabiner and Juang
Class 12: Temporal dynamics and Kalman filters: Notes: "Optimal Linear Cue Combination";; "Sensory Integration and Kalman Filtering";; "Expectation-Maximization (EM) Algorithm"; Optional readings: "A Gentle Tutorial of the EM Algorithm and ..." by Jeff Bilmes;
"The EM Algorithm for Mixtures of Factor Analyzers"; Distribute homework #3
Class 13: Ideal observers and ideal actors Sensory cue combination: Readings: "Motion Illusions as Optimal Percepts" by Weiss, Simoncelli, and Adelson;
"Humans Integrate Visual and Haptic Information in a Statistically Optimal Fashion" by Ernst and Banks
Class 14:: Readings: "Causal Inference in Multisensory Perception" by Kording, Beierholm, Ma, Quartz, Tenenbaum, and Shams;
"Optimal Predictions in Everyday Cognition" by Griffiths and Tenenbaum;
"A Rational Account of the Perceptual Magnet Effect" by Feldman and Griffiths
Class 15: Least Mean Squares (LMS) rule (a.k.a. Widrow-Hoff rule, delta rule) Adaptive learning rates Rescorla-Wagner rule Nonlinear networks and nonlinear activation functions Nearest-Neighbor classification/regression: Notes: "Linear, Threshold, Logistic, and Softmax Activation Functions"
Class 16: Radial basis function networks (and Grandmother cells) Backpropagation Algorithm Choice of activation function Recurrent networks: Notes: "Backpropagation Algorithm";; C program implementation of backpropagation algorithm; Readings: "Learning Internal Representations By Error Propagation" by Rumelhart, Hinton, and Williams; Optional reading: "Efficient Backprop" by LeCun, Bottou, Orr, and Müller; Distribute homework #4
Class 17:: Readings: "Simulating Brain Damage" by Hinton, Plaut, and Shallice,
"Neuropsychological inference with an interactive brain: A critique of the 'locality' assumption" by Farah;
"Distributed Representations, Simple Recurrent Networks, and Grammatical Structure" by Elman
Class 18-20: Student presentations of course projects

top