Pseudoinverse Network |

The pseudoinverse rule is a simple supervised learning algorithm in a Hopfield-like neural network. This associative memory system is based on learning rule inspired from Adaline:

dW_{ik}= n . [ X_{i}^{u}
- Sum_{j}(W_{ij} X_{j}^{u}) ] . X_{k}^{u} |

where W_{ik}
is the weight between neuron i and neuron k, n
is the learning rate, X_{i}^{u }is
the target pattern, Sum_{j}(W_{ij} X_{j}^{u})
is the output of a linear neuron and X_{k}^{u}
is the actual output. The iterative learning rule converges to a
connectivity matrix which is known as the pseudoinverse.

The applet was written by Olivier Michel (adapted from Matt Hill -- mlh1@cornell.edu).

Use the mouse to enter a pattern by clicking squares inside the rectangle "on" or "off". Then, have the network store your pattern by pressing "Memorize". After storing some patterns (typically two), try entering a new pattern which you will use as a test pattern. Do not impose this new pattern, but use it as an initial state of the network. Press "Test" repeatedly to watch the network settle into a previously imposed state.

- What is the theoretical maximum number of random classes the network is able to memorize?
- What is the experimental maximum number of random classes the network is able to memorize?
- Do the experimental results agree with the theory?
- Store a finite number of random patterns, e.g., 8. How many wrong pixels can the network tolerate in the initial state so that it still settles into correct pattern?
- Try to store characters as the relevant patterns. How good is the retrieval? What is the reason?