Event-Based Neuromorphic Systems

Neuromorphic electronic engineering takes its inspiration from the functioning of nervous systems to build more power efficient electronic sensors and processors. Event-based neuromorphic systems are inspired by the brain's efficient data-driven communication design, which is key to its quick responses and remarkable capabilities.  This cross-disciplinary text establishes how circuit building blocks are combined in architectures to construct complete systems. These include vision and auditory sensors as well as neuronal processing and learning circuits that implement models of nervous systems. Techniques for building multi-chip scalable systems are considered throughout the book, including methods for dealing with transistor mismatch, extensive discussions of communication and interfacing, and making systems that operate in the real world. The book also provides historical context that helps relate the architectures and circuits to each other and that guides readers to the extensive literature. Chapters are written by founding experts and have been extensively edited for overall coherence. This pioneering text is an indispensable resource for practicing neuromorphic electronic engineers, advanced electrical engineering and computer science students and researchers interested in neuromorphic systems. Key features: Summarises the latest design approaches, applications, and future challenges in the field of neuromorphic engineering. Presents examples of practical applications of neuromorphic design principles. Covers address-event communication, retinas, cochleas, locomotion, learning theory, neurons, synapses, floating gate circuits, hardware and software infrastructure, algorithms, and future challenges.

Table of Contents:
List of Contributors xv Foreword xvii Acknowledgments xix List of Abbreviations and Acronyms xxi 1 Introduction 1 1.1 Origins and Historical Context 3 1.2 Building Useful Neuromorphic Systems 5 References 5 Part I UNDERSTANDING NEUROMORPHIC SYSTEMS 7 2 Communication 9 2.1 Introduction 9 2.2 Address-Event Representation 12 2.2.1 AER Encoders 13 2.2.2 Arbitration Mechanisms 13 2.2.3 Encoding Mechanisms 17 2.2.4 Multiple AER Endpoints 19 2.2.5 Address Mapping 19 2.2.6 Routing 19 2.3 Considerations for AER Link Design 20 2.3.1 Trade-off: Dynamic or Static Allocation 21 2.3.2 Trade-off: Arbitered Access or Collisions? 23 2.3.3 Trade-off: Queueing versus Dropping Spikes 24 2.3.4 Predicting Throughput Requirements 25 2.3.5 Design Trade-offs 27 2.4 The Evolution of AER Links 28 2.4.1 Single Sender, Single Receiver 28 2.4.2 Multiple Senders, Multiple Receivers 30 2.4.3 Parallel Signal Protocol 31 2.4.4 Word-Serial Addressing 32 2.4.5 Serial Differential Signaling 33 2.5 Discussion 34 References 35 3 Silicon Retinas 37 3.1 Introduction 37 3.2 Biological Retinas 38 3.3 Silicon Retinas with Serial Analog Output 39 3.4 Asynchronous Event-Based Pixel Output Versus Synchronous Frames 40 3.5 AER Retinas 40 3.5.1 Dynamic Vision Sensor 41 3.5.2 Asynchronous Time-Based Image Sensor 46 3.5.3 Asynchronous Parvo–Magno Retina Model 46 3.5.4 Event-Based Intensity-Coding Imagers (Octopus and TTFS) 48 3.5.5 Spatial Contrast and Orientation Vision Sensor (VISe) 50 3.6 Silicon Retina Pixels 54 3.6.1 DVS Pixel 54 3.6.2 ATIS Pixel 56 3.6.3 VISe Pixel 58 3.6.4 Octopus Pixel 59 3.7 New Specifications for Silicon Retinas 60 3.7.1 DVS Response Uniformity 60 3.7.2 DVS Background Activity 62 3.7.3 DVS Dynamic Range 62 3.7.4 DVS Latency and Jitter 63 3.8 Discussion 64 References 67 4 Silicon Cochleas 71 4.1 Introduction 72 4.2 Cochlea Architectures 75 4.2.1 Cascaded 1D 76 4.2.2 Basic 1D Silicon Cochlea 77 4.2.3 2D Architecture 78 4.2.4 The Resistive (Conductive) Network 79 4.2.5 The BM Resonators 80 4.2.6 The 2D Silicon Cochlea Model 80 4.2.7 Adding the Active Nonlinear Behavior of the OHCs 82 4.3 Spike-Based Cochleas 83 4.3.1 Q-control of AEREAR2 Filters 85 4.3.2 Applications: Spike-Based Auditory Processing 86 4.4 Tree Diagram 87 4.5 Discussion 87 References 89 5 Locomotion Motor Control 91 5.1 Introduction 92 5.1.1 Determining Functional Biological Elements 92 5.1.2 Rhythmic Motor Patterns 93 5.2 Modeling Neural Circuits in Locomotor Control 95 5.2.1 Describing Locomotor Behavior 96 5.2.2 Fictive Analysis 97 5.2.3 Connection Models 99 5.2.4 Basic CPG Construction 100 5.2.5 Neuromorphic Architectures 102 5.3 Neuromorphic CPGs at Work 108 5.3.1 A Neuroprosthesis: Control of Locomotion in Vivo 109 5.3.2 Walking Robots 111 5.3.3 Modeling Intersegmental Coordination 112 5.4 Discussion 113 References 115 6 Learning in Neuromorphic Systems 119 6.1 Introduction: Synaptic Connections, Memory, and Learning 120 6.2 Retaining Memories in Neuromorphic Hardware 121 6.2.1 The Problem of Memory Maintenance: Intuition 121 6.2.2 The Problem of Memory Maintenance: Quantitative Analysis 122 6.2.3 Solving the Problem of Memory Maintenance 124 6.3 Storing Memories in Neuromorphic Hardware 128 6.3.1 Synaptic Models for Learning 128 6.3.2 Implementing a Synaptic Model in Neuromorphic Hardware 132 6.4 Toward Associative Memories in Neuromorphic Hardware 136 6.4.1 Memory Retrieval in Attractor Neural Networks 137 6.4.2 Issues 142 6.5 Attractor States in a Neuromorphic Chip 143 6.5.1 Memory Retrieval 143 6.5.2 Learning Visual Stimuli in Real Time 145 6.6 Discussion 148 References 149 Part II BUILDING NEUROMORPHIC SYSTEMS 153 7 Silicon Neurons 155 7.1 Introduction 156 7.2 Silicon Neuron Circuit Blocks 158 7.2.1 Conductance Dynamics 158 7.2.2 Spike-Event Generation 159 7.2.3 Spiking Thresholds and Refractory Periods 161 7.2.4 Spike-Frequency Adaptation and Adaptive Thresholds 162 7.2.5 Axons and Dendritic Trees 164 7.2.6 Additional Useful Building Blocks 165 7.3 Silicon Neuron Implementations 166 7.3.1 Subthreshold Biophysically Realistic Models 166 7.3.2 Compact I&F Circuits for Event-Based Systems 169 7.3.3 Generalized I&F Neuron Circuits 170 7.3.4 Above Threshold, Accelerated-Time, and Switched-Capacitor Designs 174 7.4 Discussion 176 References 180 8 Silicon Synapses 185 8.1 Introduction 186 8.2 Silicon Synapse Implementations 188 8.2.1 Non Conductance-Based Circuits 188 8.2.2 Conductance-Based Circuits 198 8.2.3 NMDA Synapse 200 8.3 Dynamic Plastic Synapses 201 8.3.1 Short-Term Plasticity 201 8.3.2 Long-Term Plasticity 203 8.4 Discussion 213 References 215 9 Silicon Cochlea Building Blocks 219 9.1 Introduction 219 9.2 Voltage-Domain Second-Order Filter 220 9.2.1 Transconductance Amplifier 220 9.2.2 Second-Order Low-Pass Filter 222 9.2.3 Stability of the Filter 223 9.2.4 Stabilised Second-Order Low-Pass Filter 225 9.2.5 Differentiation 225 9.3 Current-Domain Second-Order Filter 227 9.3.1 The Translinear Loop 227 9.3.2 Second-Order Tau Cell Log-Domain Filter 229 9.4 Exponential Bias Generation 230 9.5 The Inner Hair Cell Model 233 9.6 Discussion 234 References 234 10 Programmable and Configurable Analog Neuromorphic ICs 237 10.1 Introduction 238 10.2 Floating-Gate Circuit Basics 238 10.3 Floating-Gate Circuits Enabling Capacitive Circuits 238 10.4 Modifying Floating-Gate Charge 242 10.4.1 Electron Tunneling 242 10.4.2 pFET Hot-Electron Injection 242 10.5 Accurate Programming of Programmable Analog Devices 244 10.6 Scaling of Programmable Analog Approaches 246 10.7 Low-Power Analog Signal Processing 247 10.8 Low-Power Comparisons to Digital Approaches: Analog Computing in Memory 249 10.9 Analog Programming at Digital Complexity: Large-Scale Field Programmable Analog Arrays 251 10.10 Applications of Complex Analog Signal Processing 253 10.10.1 Analog Transform Imagers 253 10.10.2 Adaptive Filters and Classifiers 253 10.11 Discussion 256 References 257 11 Bias Generator Circuits 261 11.1 Introduction 261 11.2 Bias Generator Circuits 263 11.2.1 Bootstrapped Current Mirror Master Bias Current Reference 263 11.2.2 Master Bias Power Supply Rejection Ratio (PSRR) 265 11.2.3 Stability of the Master Bias 265 11.2.4 Master Bias Startup and Power Control 266 11.2.5 Current Splitters: Obtaining a Digitally Controlled Fraction of the Master Current 267 11.2.6 Achieving Fine Monotonic Resolution of Bias Currents 271 11.2.7 Using Coarse–Fine Range Selection 273 11.2.8 Shifted-Source Biasing for Small Currents 274 11.2.9 Buffering and Bypass Decoupling of Individual Biases 275 11.2.10 A General Purpose Bias Buffer Circuit 278 11.2.11 Protecting Bias Splitter Currents from Parasitic Photocurrents 279 11.3 Overall Bias Generator Architecture Including External Controller 279 11.4 Typical Characteristics 280 11.5 Design Kits 281 11.6 Discussion 282 References 282 12 On-Chip AER Communication Circuits 285 12.1 Introduction 286 12.1.1 Communication Cycle 286 12.1.2 Speedup in Communication 287 12.2 AER Transmitter Blocks 289 12.2.1 AER Circuits within a Pixel 289 12.2.2 Arbiter 290 12.2.3 Other AER Blocks 295 12.2.4 Combined Operation 297 12.3 AER Receiver Blocks 298 12.3.1 Chip-Level Handshaking Block 298 12.3.2 Decoder 299 12.3.3 Handshaking Circuits in Receiver Pixel 300 12.3.4 Pulse Extender Circuits 301 12.3.5 Receiver Array Peripheral Handshaking Circuits 301 12.4 Discussion 302 References 303 13 Hardware Infrastructure 305 13.1 Introduction 306 13.1.1 Monitoring AER Events 307 13.1.2 Sequencing AER Events 311 13.1.3 Mapping AER Events 313 13.2 Hardware Infrastructure Boards for Small Systems 316 13.2.1 Silicon Cortex 316 13.2.2 Centralized Communication 317 13.2.3 Composable Architecture Solution 318 13.2.4 Daisy-Chain Architecture 324 13.2.5 Interfacing Boards using Serial AER 324 13.2.6 Reconfigurable Mesh-Grid Architecture 328 13.3 Medium-Scale Multichip Systems 329 13.3.1 Octopus Retina + IFAT 329 13.3.2 Multichip Orientation System 332 13.3.3 CAVIAR 335 13.4 FPGAs 340 13.5 Discussion 342 References 345 14 Software Infrastructure 349 14.1 Introduction 349 14.1.1 Importance of Cross-Community Commonality 350 14.2 Chip and System Description Software 350 14.2.1 Extensible Markup Language 351 14.2.2 NeuroML 351 14.3 Configuration Software 352 14.4 Address Event Stream Handling Software 352 14.4.1 Field-Programmable Gate Arrays 353 14.4.2 Structure of AE Stream Handling Software 353 14.4.3 Bandwidth and Latency 353 14.4.4 Optimization 354 14.4.5 Application Programming Interface 355 14.4.6 Network Transport of AE Streams 355 14.5 Mapping Software 356 14.6 Software Examples 357 14.6.1 ChipDatabase – A System for Tuning Neuromorphic aVLSI Chips 357 14.6.2 Spike Toolbox 359 14.6.3 jAER 359 14.6.4 Python and PyNN 360 14.7 Discussion 363 References 363 15 Algorithmic Processing of Event Streams 365 15.1 Introduction 365 15.2 Requirements for Software Infrastructure 367 15.2.1 Processing Latency 369 15.3 Embedded Implementations 369 15.4 Examples of Algorithms 370 15.4.1 Noise Reduction Filters 370 15.4.2 Time-Stamp Maps and Subsampling by Bit-Shifting Addresses 372 15.4.3 Event Labelers as Low-Level Feature Detectors 372 15.4.4 Visual Trackers 374 15.4.5 Event-Based Audio Processing 378 15.5 Discussion 379 References 379 16 Towards Large-Scale Neuromorphic Systems 381 16.1 Introduction 381 16.2 Large-Scale System Examples 382 16.2.1 Spiking Neural Network Architecture 382 16.2.2 Hierarchical AER 384 16.2.3 Neurogrid 386 16.2.4 High Input Count Analog Neural Network System 388 16.3 Discussion 390 References 391 17 The Brain as Potential Technology 393 17.1 Introduction 393 17.2 The Nature of Neuronal Computation: Principles of Brain Technology 395 17.3 Approaches to Understanding Brains 396 17.4 Some Principles of Brain Construction and Function 398 17.5 An Example Model of Neural Circuit Processing 400 17.6 Toward Neuromorphic Cognition 402 References 404 Index 407