Abstract
We present an exact solution for the dynamics of online Hebbian learning in neural networks, with restricted and unrealizable training sets. In contrast to other studies on learning with restricted training sets, unrealizability is here caused by structural mismatch, rather than data noise: the teacher machine is a perceptron with a reversed-wedge-type transfer function, while the student machine is a perceptron with a sigmoidal transfer function. We calculate the glassy dynamics of the macroscopic performance measures, training error and generalization error, and the (non-Gaussian) student field distribution. Our results, which find excellent confirmation in numerical simulations, provide a new benchmark test for general formalisms with which to study unrealizable learning processes with restricted training sets.
| Original language | English |
|---|---|
| Pages (from-to) | L401 - L408 |
| Journal | JOURNAL OF PHYSICS A MATHEMATICAL AND GENERAL |
| Volume | 34 |
| Issue number | 30 |
| Publication status | Published - 3 Aug 2001 |