AI Applications to Communications and Information Technologies: The Role of Ultra Deep Neural Networks

AI Applications to Communications and Information Technologies Apply the technology of the future to networking and communications. Artificial intelligence, which enables computers or computer-controlled systems to perform tasks which ordinarily require human-like intelligence and decision-making, has revolutionized computing and digital industries like few other developments in recent history. Tools like artificial neural networks, large language models, and deep learning have quickly become integral aspects of modern life. With research and development into AI technologies proceeding at lightning speeds, the potential applications of these new technologies are all but limitless. AI Applications to Communications and Information Technologies offers a cutting-edge introduction to AI applications in one particular set of disciplines. Beginning with an overview of foundational concepts in AI, it then moves through numerous possible extensions of this technology into networking and telecommunications. The result is an essential introduction for researchers and for technology undergrad/grad student alike. AI Applications to Communications and Information Technologies readers will also find: In-depth analysis of both current and evolving applications Detailed discussion of topics including generative AI, chatbots, automatic speech recognition, image classification and recognition, IoT, smart buildings, network management, network security, and more An authorial team with immense experience in both research and industry AI Applications to Communications and Information Technologies is ideal for researchers, industry observers, investors, and advanced students of network communications and related fields.

Table of Contents:
About the Authors xi Preface xiii 1 Overview 1 1.1 Introduction and Basic Concepts 1 1.1.1 Machine Learning 5 1.1.2 Deep Learning 6 1.1.3 Activation Functions 13 1.1.4 Multi-layer Perceptrons 17 1.1.5 Recurrent Neural Networks 21 1.1.6 Convolutional Neural Networks 21 1.1.7 Comparison 26 1.2 Learning Methods 26 1.3 Areas of Applicability 39 1.4 Scope of this Text 41 A. Basic Glossary of Key AI Terms and Concepts 44 References 57 2 Current and Evolving Applications to Natural Language Processing 65 2.1 Scope 65 2.2 Introduction 66 2.3 Overview of Natural Language Processing and Speech Processing 72 2.3.1 Feed-forward NNs 74 2.3.2 Recurrent Neural Networks 74 2.3.3 Long Short-Term Memory 75 2.3.4 Attention 77 2.3.5 Transformer 78 2.4 Natural Language Processing/Natural Language Understanding Basics 81 2.4.1 Pre-training 82 2.4.2 Natural Language Processing/Natural Language Generation Architectures 85 2.4.3 Encoder-Decoder Methods 88 2.4.4 Application of Transformer 89 2.4.5 Other Approaches 90 2.5 Natural Language Generation Basics 91 2.6 Chatbots 95 2.7 Generative AI 101 A. Basic Glossary of Key AI Terms and Concepts Related to Natural Language Processing 103 References 109 3 Current and Evolving Applications to Speech Processing 117 3.1 Scope 117 3.2 Overview 119 3.2.1 Traditional Approaches 119 3.2.2 DNN-based Feature Extraction 123 3.3 Noise Cancellation 126 3.3.1 Approaches 128 3.3.1.1 Delay-and-Sum Beamforming (DSB) 129 3.3.1.2 Minimum Variance Distortionless Response (MVDR) Beamformer 130 3.3.1.3 Non- adaptive Beamformer 131 3.3.1.4 Multichannel Linear Prediction (MCLP) 132 3.3.1.5 ML-based Approaches 132 3.3.1.6 Neural Network Beamforming 135 3.3.2 Specific Example of a System Supporting Noise Cancellation 138 3.4 Training 141 3.5 Applications to Voice Interfaces Used to Control Home Devices and Digital Assistant Applications 142 3.6 Attention-based Models 146 3.7 Sentiment Extraction 148 3.8 End-to-End Learning 148 3.9 Speech Synthesis 150 3.10 Zero-shot TTS 152 3.11 VALL- E: Unseen Speaker as an Acoustic Prompt 152 A. Basic Glossary of Key AI Terms and Concepts 156 References 166 4 Current and Evolving Applications to Video and Imaging 173 4.1 Overview and Background 173 4.2 Convolution Process 176 4.3 CNNs 181 4.3.1 Nomenclature 181 4.3.2 Basic Formulation of the CNN Layers and Operation 181 4.3.2.1 Layers 181 4.3.2.2 Operations 188 4.3.3 Fully Convolutional Networks (FCN) 190 4.3.4 Convolutional Autoencoders 190 4.3.5 R-CNNs, Fast R-CNN, Faster R-CNN 193 4.4 Imaging Applications 195 4.4.1 Basic Image Management 195 4.4.2 Image Segmentation and Image Classification 199 4.4.3 Illustrative Examples of a Classification DNN/CNN 202 4.4.4 Well-Known Image Classification Networks 204 4.5 Specific Application Examples 213 4.5.1 Semantic Segmentation and Semantic Edge Detection 213 4.5.2 CNN Filtering Process for Video Coding 215 4.5.3 Virtual Clothing 216 4.5.4 Example of Unmanned Underwater Vehicles/Unmanned Aerial Vehicles 218 4.5.5 Object Detection Applications 218 4.5.6 Classifying Video Data 222 4.5.7 Example of Training 224 4.5.8 Example: Image Reconstruction is Used to Remove Artifacts 225 4.5.9 Example: Video Transcoding/Resolution-enhancement 228 4.5.10 Facial Expression Recognition 228 4.5.11 Transformer Architecture for Image Processing 230 4.5.12 Example: A GAN Approach/Synthetic Photo 230 4.5.13 Situational Awareness 231 4.6 Other Models: Diffusion and Consistency Models 236 A. Basic Glossary of Key AI Terms and Concepts 238 B. Examples of Convolutions 246 References 250 5 Current and Evolving Applications to IoT and Applications to Smart Buildings and Energy Management 257 5.1 Introduction 257 5.1.1 IoT Applications 257 5.1.2 Smart Cities 258 5.2 Smart Building ML Applications 275 5.2.1 Basic Building Elements 275 5.2.2 Particle Swarm Optimization 276 5.2.3 Specific ML Example – Qin Model 279 5.2.3.1 EnergyPlus™ 281 5.2.3.2 Modeling and Simulation 282 5.2.3.3 Energy Audit Stage 286 5.2.3.4 Optimization Stage 287 5.2.3.5 Model Construction 289 5.2.3.6 EnergyPlus Models 289 5.2.3.7 Real- Time Control Parameters 290 5.2.3.8 Neural Networks in the Qin Model (DNN, RNN, CNN) 290 5.2.3.9 Finding Inefficiency Measures 294 5.2.3.10 Particle Swarm Optimizer 294 5.2.3.11 Integration of Particle Swarm Optimization with Neural Networks 296 5.2.3.12 Deep Reinforcement Learning 298 5.2.3.13 Deployments 298 5.3 Example of a Commercial Product – BrainBox AI 301 5.3.1 Overview 301 5.3.2 LSTM Application – Technical Background 302 5.3.3 BrainBox AI Commercial Energy Optimization System 305 A. Basic Glossary of Key IoT (Smart Building) Terms and Concepts 314 References 339 6 Current and Evolving Applications to Network Cybersecurity 347 6.1 Overview 347 6.2 General Security Requirements 349 6.3 Corporate Resources/Intranet Security Requirements 353 6.3.1 Network and End System Security Testing 358 6.3.2 Application Security Testing 360 6.3.3 Compliance Testing 362 6.4 IoT Security (IoTSec) 363 6.5 Blockchains 365 6.6 Zero Trust Environments 369 6.7 Areas of ML Applicability 370 6.7.1 Example of Cyberintrusion Detector 373 6.7.2 Example of Hidden Markov Model (HMM) for Intrusion Detection 374 6.7.3 Anomaly Detection Example 378 6.7.4 Phishing Detection Emails Using Feature Extraction 383 6.7.5 Example of Classifier Engine to Identify Phishing Websites 386 6.7.6 Example of System for Data Protection 388 6.7.7 Example of an Integrated Cybersecurity Threat Management 390 6.7.8 Example of a Vulnerability Lifecycle Management System 392 A. Basic Glossary of Key Security Terms and Concepts 396 References 400 7 Current and Evolving Applications to Network Management 407 7.1 Overview 407 7.2 Examples of Neural Network- Assisted Network Management 408 7.2.1 Example of NN-Based Network Management System (Case of FM) 413 7.2.2 Example of a Model for Predictions Related to the Operation of a Telecommunication Network (Case of FM) 416 7.2.3 Prioritizing Network Monitoring Alerts (Case of FM and PM) 419 7.2.4 System for Recognizing and Addressing Network Alarms (Case of FM) 424 7.2.5 Load Control of an Enterprise Network (Case of PM) 428 7.2.6 Data Reduction to Accelerate Machine Learning for Networking (Case of FM and PM) 431 7.2.7 Compressing Network Data (Case of PM) 435 7.2.8 ML Predictor for a Remote Network Management Platform (Case of FM, PM, CM, AM) 437 7.2.9 Cable Television (CATV) Performance Management System (Case of PM) 441 A. Short Glossary of Network Management Concepts 446 References 447 Super Glossary 449 Index 467