Exercise 5: Basis function network

Create a basis function network that computes a population code for the head-centered location of an object (a_x) from population codes for the eye-centered location of the object (r_x) and the position of the eyes (e_x) (stick to the horizontal dimension for all variables). In 1 D, these variables are related to one another through:

a_x= r_x+e_x

In the input layers (the eye-centered location and the eye position layers), use 64 units equally spaced between –90 and +90 degree with gaussian tuning curves. Choose the width of the tuning curves to ensure large overlap between neighboring units.

1- Plot the activity pattern in both layers for an object located at 10 degrees in eye-centered coordinates while the eyes fixate at –20 degrees.

In the hidden layer (i.e. the basis function layer), create 64x64 array of units in which each unit takes the product between one eye-centered unit and one eye position unit.

Plot the activity of a representative unit in each layer (including the basis function layer), as a function of horizontal eye-centered location and eye position.

2- The goal of such a model would be to obtain a population of neurons in the output layer with gaussian tuning curves to the head-centered location of the object. To make your life easier, we will settle for a single unit with a gaussian tuning curve centered at zero.

Use the delta rule to adjust the weights between the basis function units and the output unit until the output unit shows the desired gaussian tuning curve in head-centered coordinates (for the width of the tuning curve, use twice the value you used in the input layers. It happens to make training easier). Basically, you need to go through the following steps on every trial:

1. Pick a pair of eye position and eye-centered location (the best thing to do here is to choose in advance, say, 128 eye-centered locations and 128 eye positions and to sample each pair equally often)

2. Compute the activities in the input and basis function layers

3. Compute the activity of the output unit using the current weights: o = S_ij w_ij b_ij

4. Compute the desired activity for the ouput unit (o*) using the equation for a gaussian tuning curve in head-centered coordinates centered at 0, applied to the current eye-centered location and eye position.

5. Adjust the weight according to : w_ij(t+1)= w_ij(t)+dw_ij= w_ij(t)+ a (o*-o)b_ij

Tips: try various values for a but start small (0.0001 or smaller...), and initialize the weights to small random values.

Once you’re done with training, plot the weights pattern, the desired tuning curve for the output unit and its actual tuning curve.