Learning Systems

A learning system can be defined as a system which can adapt its behaviour to become more effective at a particular task. It consists of an architecture with a set of variable parameters and an algorithm. This book covers major aspects of learning systems: system architecture, choice of performance index and methods measuring error. Major learning algorithms are explained, including proofs of convergence, and artificial neural networks, which are an important class of learning systems, are discussed. Where appropriate, examples are given to demonstrate the practical use of techniques developed in the text.

Table of Contents:
1 Introduction to Learning Systems.- 1.1 Systems, Memory.- 1.2 Performance Index.- 1.3 Learning Algorithms.- 1.4 Some Examples of Learning Systems.- References.- 2 Deterministic Algorithms.- 2.1 Simple Projection Algorithms in Spaces With Different Norms (Structure, Convergence, Properties).- 2.2 Modified Projection Algorithms With a High Rate of Convergence.- References.- 3 Deterministic and Stochastic Algorithms of Optimisation.- 3.1 Deterministic Methods for Unconstrained Minimisation.- 3.2 Stochastic Approximation and Recurrent Estimation.- References.- 4 Stochastic Algorithms: The Least Squares Method in the Non-Recurrent and Recurrent Forms and the Gauss-Markov Theorem.- 4.1 The Least Squares Method in Recursive and Non-Recursive Forms (The White Noise Case).- 4.2 The Gauss-Markov Theorem.- 4.3 Example System.- References.- 5 Stochastic Algorithms.- 5.1 Algorithms With Forgetting Factor.- 5.2 The Least Squares Method by Correlated Noise in the Non-Recursive and Recursive Forms: Connection With Decorrelation Procedures.- 5.3 Introduction to the Kaiman filter [1], [3].- References.- 6 Multilayer Neural Networks.- 6.1 Multilayer Neural Network as a Non-Linear Transformer. The Kolmogorov and Cybenko Theorems.- 6.2 Learning Algorithms for Single Elements of Multilayer Neural Networks.- References.- 7 Learning Algorithms for Neural Networks.- 7.1 The Back-Propagation Algorithm for MNN Learning.- 7.2 Autonomous Algorithms for Adjusting MNNs.- References.- 8 Identification and Control of Dynamic Systems Using Multilayer Neural Networks.- 8.1 Identification of Dynamic Systems Using Multilayer Neural Networks.- 8.2 Control of Dynamic Systems Using Multilayer Neural Networks.- 8.3 Control of Non-Linear Dynamic Systems Using Multilayer Neural Networks.- References.- 9 The Cerebellar Model Articulation Controller (CMAC).- 9.1 Introduction to CMAC.- 9.2 Data Storage and Learning Process in the CMAC.- 9.3 Albus’ Learning Algorithm.- 9.4 Modified Albus’ Algorithm.- 9.5 CMAC for Identification and Adaptive Control.- References.