Homework #6

  1. Devise a feedforward network with four input units and two output units which acts as a binary adder modulo 4. That is, if the sum of the inputs is 0, then the output is 00; if the sum is 1, the output is 01; if the sum is 2 it is 10; if the sum is 3, output is 11; and if the sum is 4, the output is again 00. (Hint: the XOR circuit is essentially a binary adder modulo 2)
  2. Study the properties of the two neuron winner-take-all model:

    x1'=-x1+f(i1-c*x2)

    x2'=-x2+f(i2-c*x1)

    f(u)=1/(1+exp(-u))

    More specifically, suppose that I1=I2 and vary c. Show that for c small and positive, then there is a fixed point solution x1=x2. Show that if c is large enough, then there are two stable fixed points, one in which x1 is the winner and one in which x2 is the winner. Show that as long as I1 and I2 are not too different, then either neuron can emerge as a winner
  3. Now, take the same model as above and this time add an adaptation variable:

    x1'=-x1+f(i1-c*x2-g*y1)

    x2'=-x2+f(i2-c*x1-g*y2)

    y1'=(-y1+x1)/tau

    y2'=(-y2+x2)/tau

    Start with, I1=3,I2=2.9 and c=8. Set the time constant for adaptation, tau=10. Look at what happens as you increase g. Show that for small g, the network is still a winner take all. However, if g gets big enough, show that it begins to oscillate between the two units. This is a simple model for the Necker cube illusion.
  4. Design a recurrent attractor neural network with 5 units which produces the following two (or more ) outputs: (11001) and (10110). Demonstrate pattern completion for initial conditions, (01001) and (00110). What happens if the input is (11010) ?
  5. Simulate a "ring" network of say 50 units using the standard model from the book but this time make the input rotate around the ring with frequency omega. Show that the network will "follow" the input as long as the frequency is not too high. What happens if the frequency is higher. Here is the model:

    u_j' = -u_j + F(h(j/50-omega*t)+sum(k=0,49)(M(k-j)*u_j)/50)

    M(k)=-7.3+11*cos(2*pi*k/50)

    h(u)=A c (1-eps + eps*cos(2*pi*u))

    F(u) is the positive part of u. Choose c=.8,a=40,eps=.2. Start with omega=.005 and start to increase it. Note that this could take a real long time if you dont rewrite the above sum. I include an XPP file for this 

    # sompolinsky model
#  I take advantage of the fact that these are
#  product kernels
#  
#  int[cos(x-y)*u(y)] = cos(x) int[cos(y)*u(y)] + sin(x)*int[sin(y)*u(y)]
# 
table cs % 51 0 50 cos(.02*2*pi*t)
table sn % 51 0 50 sin(.02*2*pi*t)
# input
h(u)=a*c*(1-eps-eps*cos(2*pi*u))
# integral of cos with v
cv=sum(0,49)of(cs(i')*shift(v0,i'))*.02
# integral of sin with v
sv=sum(0,49)of(sn(i')*shift(v0,i'))*.02
# integral over all v
vt=sum(0,49)of(shift(v0,i'))*.02
# now the ODE!
v[0..49]'=-v[j]+max(h(w*t-[j]*.02)-lam0*vt+lam1*(cs([j])*cv+sn([j])*sv),0)
par w=.005
par c=.8,a=40,lam0=7.3,lam1=11,eps=.2
# 
done