VLSI Artificial Neural Networks Engineering

This text offers an engineering approach to the design of VLSI (very-large-scale integration) Artificial Neural Networks (ANNs). The design of analog, digital and mixed analog/digital VLSI ANNs are represented. A design methodology and a CAD environment are presented to highlight the tradeoff design factors. System applications of ANNs to automatic speech recognition and pattern recognition are included. Chapter 1 serves as an introduction. Chapters 2, 3, 4 and 5 deal with VLSI circuit design techniques (analog, digital and sampled data) and automated VLSI design environment for ANNs. Chapter 2 reports on a sampled data approach to the implementation of ANNs with application to character recognition. It also contains an overview of the different approaches of VLSI implementation of ANNs; explaining the advantage and disadvantage of each approach. In Chapter 3, the topic of design exploration of mixed analog/digital ANNs at the high level of the design hierarchy is addressed. The need for creating such a design automation environment, with its supporting CAD tools, is a necessary condition for the widespread use of application-specific chips of ANN implementation. In Chapter 4 the same topic of design exploration is discussed, but at the low level of the hierarchy and targeting analog implementation. Chapter 5 reports on all-digital implementation of ANNs using the Neocognitron as the ANN model. Chapters 6, 7, 8 and 9 deal with the application of ANNs to a number of fields. Chapter 6 addresses the topic of automatic speech recognition using neural predictive hidden Markov models. Chapter 7 deals with the topic of classification using minimum complexity ANNs. Chapter 8 addresses the topic of pattern recognition using a fuzzy clustering ANNs. Chapter 9 deals with speech recognition using pipelined ANNs. "VLSI Artificial Neural Networks Engineering" will be useful to researchers and graduated engineers working in the area of VLSI circuit and system design and to the students of upper-undergraduate and graduate level courses on analog circuits, digital circuits, ANNs and VLSI system applications.

Table of Contents:
1 An Overview.- 1.1 Introduction.- 1.2 Biological Neural Networks.- 1.3 Artificial Neural Networks (ANNs).- 1.4 Artificial Neural Network Algorithms.- 1.5 Supervised Neural Networks.- 1.6 Unsupervised Neural Networks.- 1.7 Neural Network Architectures and Implementations.- 1.8 Book Overview.- 2 A Sampled-Data CMOS VLSI Implementation of a Multi-Character ANN Recognition System.- 2.1 Introduction.- 2.2 ANN Implementation Techniques.- 2.3 Developed CMOS Circuits for ANNs.- 2.4 The Prototype MLP ANN Model Architecture.- 2.5 MLP ANN Model Simulations.- 2.6 ANN Circuit Simulations.- 2.7 The Developed VLSI Architectures.- 2.8 The Developed Two-Character ANN Recognizer.- 2.9 The Proposed Multi-Character ANN Recognition System.- 2.10 Conclusions.- 3 A Design Automation Environment for Mixed Analog/Digital ANNs.- 3.1 Introduction.- 3.2 Mixed Analog/Digital ANN Hardware.- 3.3 Overview of the Design Automation Environment.- 3.4 Data Flow Graph.- 3.5 The Analyzer.- 3.6 The Design Library.- 3.7 The Synthesizer.- 3.8 Design Examples.- 3.9 Conclusions.- 4 A Compact VLSI Implementation of Neural Networks.- 4.1 Introduction.- 4.2 The Building Blocks.- 4.3 The Circuit Implementation Example.- 4.4 Expanding the Network with Multiple Chips.- 4.5 Conclusions.- 5 An All-Digital VLSI ANN.- 5.1 Introduction.- 5.2 Neocognitron Neural Network Model.- 5.3 Digi-Neocognitron (DNC): A Digital Neural Network Model for VLSI.- 5.4 Character Recognition Example.- 5.5 Advantages for VLSI Implementation.- 5.6 Conclusions.- 6 A Neural Predictive Hidden Markov Model Architecture for Speech and Speaker Recognition.- 6.1 Introduction.- 6.2 Automatic Speech Recognition Methodologies.- 6.3 An ANN Architecture for Predictive HMMs.- 6.4 Discriminative Training of the Neural Predictive HMM.- 6.5 Speaker Recognition Using the Neural Predictive HMM.- 6.6 Conclusions.- 7 Minimum Complexity Neural Networks for Classification.- 7.1 Introduction.- 7.2 Adaptive Probabilistic Neural Networks: APNN and ANNC.- 7.3 Bayesian PDF Model Selection.- 7.4 Maximum Likelihood and Maximum Mutual Information Train-ing.- 7.5 Experimental Results.- 7.6 The Adaptive Feature Extraction Nearest Neighbor Classifier “AFNN”.- 7.7 Conclusions.- 8 A Parallel ANN Architecture for Fuzzy Clustering.- 8.1 Introduction.- 8.2 Fuzzy Clustering and Neural Networks.- 8.3 Fuzzy Competitive Learning Algorithm.- 8.4 Mapping Algorithm Onto Architecture.- 8.5 Fuzzy Clustering Neural Network (FCNN) Architecture: Processing Cells.- 8.6 Comparison With The Fuzzy C-Mean (FCM) Algorithm.- 8.7 Conclusions.- 9 A Pipelined Ann Architecture for Speech Recognition.- 9.1 Introduction.- 9.2 Definition and Notation.- 9.3 PNN Architecture: Processing Stages.- 9.4 Case Studies.- 9.5 Performance Analysis.- 9.6 Conclusions.