An expert compilation of on-device training techniques, regulatory frameworks, and ethical considerations of TinyML design and development
In Tiny Machine Learning: Design Principles and Applications, a team of distinguished researchers delivers a comprehensive discussion of the critical concepts, design principles, applications, and relevant issues in Tiny Machine Learning (TinyML). Expert contributors introduce a new low power resource, offering vast applications in IoT devices with system-algorithm co-design.
Tiny Machine Learning explores TinyML paradigms and enablers, TinyML for anomaly detection, and the learning panorama under TinyML. Readers will find explanations of TinyML devices and tools, power consumption and memory in IoT microcontrollers, and lightweight frameworks for TinyML. The book also describes TinyML techniques for real-time and environmental applications.
Additional topics covered in the book include:
- A thorough introduction to security and privacy techniques for TinyML devices, including the implementation of novel security schemes
- Incisive explorations of power consumption and memory in IoT MCUs, including ultralow-power smart IoT devices with embedded TinyML
- Practical discussions of TinyML research targeting microcontrollers for data extraction and synthesis
Perfect for industry and academic researchers, scientists, and engineers, Tiny Machine Learning will also benefit lecturers and graduate students interested in machine learning.
Table of Contents:
About the Editors xxiii
List of Contributors xxvii
Preface xxxv
1 Introduction to TinyML 1
Francisca Onyiyechi Nwokoma, Chidi Ukamaka Betrand, Juliet Nnenna Odii, Euphemia Chioma Nwokorie, and Ikechukwu Ignatius Ayogu
1.1 Introduction 1
1.2 Evolution of TinyML 6
1.3 Key Milestones and Current Trends 8
1.4 TinyML System Development 11
1.5 Challenges and Bottlenecks 17
1.6 Cost–Benefit Analysis 20
1.7 Key Findings 26
1.8 Limitations of TinyML 27
1.9 Conclusion 29
References 30
2 Learning Panorama Under TinyML 35
Ikechukwu Ignatius Ayogu, Euphemia Chioma Nwokorie, Juliet Nnenna Odii, Francisca Onyiyechi Nwokoma, and Chidi Ukamaka Betrand
2.1 Introduction 35
2.2 Challenges and Opportunities for Improved TinyML Model Design 37
2.3 Frontiers in Model Optimization for TinyML 47
2.4 Learning Frameworks and Tools for TinyML Development 59
2.5 Frontiers for Algorithmic Innovations for TinyML 63
2.6 TinyML Development Process 68
2.7 Key Findings 69
2.8 Conclusion 70
References 71
3 TinyML for Anomaly Detection 85
Richard Govada Joshua, Peter Anuoluwapo Gbadega, Agbotiname Lucky Imoize, and Samuel Oluwatobi Tofade
3.1 Introduction 85
3.2 Context and Literature Review 93
3.3 Lessons Learned 149
3.4 Future Scope 152
3.5 Conclusion 156
References 157
4 TinyML Power Consumption and Memory in IoT MCUs 163
Peter Anuoluwapo Gbadega, Agbotiname Lucky Imoize, Richard Govada Joshua, and Samuel Oluwatobi Tofade
4.1 Introduction 163
4.2 Context and Literature Review 171
4.3 Methodology 174
4.4 Results and Discussion: Bibliometric Analysis 184
4.5 Conclusion and Future Directions 198
References 199
5 Efficient Data Cleaning and Anomaly Detection in IoT Devices Using TinyCleanEDF 205
Ilker Kara
5.1 Introduction 205
5.2 IoT and TinyCleanEDF 206
5.3 The Importance of Data Cleaning in IoT Systems 207
5.4 Anomaly Detection and Its Importance in IoT Systems 208
5.5 Data Preprocessing and Cleaning Workflow 209
5.6 Implementation Details 213
5.7 Case Studies and Applications 216
5.8 Future Directions 219
5.9 Conclusion and Future Scope 221
References 222
6 TinyML Devices and Tools 225
Abeeb Akorede Bello, Agbotiname Lucky Imoize, and Abiodun Temitope Odewale
6.1 Introduction 225
6.2 Related Work 227
6.3 TinyML Devices 234
6.4 TinyML Tools 237
6.5 Deployment Procedure of TinyML 241
6.6 Lesson Learned and Prospects 250
6.7 Conclusion 253
References 254
7 Privacy-Preserving Techniques in TinyML for IoT 259
Oleksandr Kuznetsov, Emanuele Frontoni, Kateryna Kuznetsova, Marco Arnesano, and Pavlo Usik
7.1 Introduction 259
7.2 Related Works 261
7.3 Homomorphic Encryption in TinyML 262
7.4 Differential Privacy for TinyML 271
7.5 Secure Multi-Party Computation in TinyML 277
7.6 Case Studies and Applications of Privacy-Preserving TinyML 281
7.7 Discussion and Future Directions 293
7.8 Conclusion 296
Acknowledgment 296
References 296
8 Enhancing Cybersecurity in TinyML with Lightweight Cryptographic Algorithms 303
Oleksandr Kuznetsov, Roman Minailenko, Aigul Shaikhanova, Yelyzaveta Kuznetsova, and Agbotiname Lucky Imoize
8.1 Introduction 303
8.2 Literature Review 305
8.3 Lightweight Cryptographic Algorithms for TinyML 306
8.4 Comparative Analysis of Lightweight Block Ciphers 315
8.5 Comparative Analysis of Lightweight Hash Functions 318
8.6 Comparative Analysis of Lightweight Stream Ciphers 322
8.7 Implementation Strategies for Lightweight Cryptography in TinyML 325
8.8 Conclusion 327
Acknowledgment 328
References 329
9 Tiny Machine Learning for Enhanced Edge Intelligence 335
Emmanuel Alozie, Agbotiname Lucky Imoize, Hawau I. Olagunju, Nasir Faruk, Salisu Garba, and Ayobami P. Olatunji
9.1 Introduction 335
9.2 Overview of Tiny Machine Learning (TinyML) 337
9.3 TinyML as a Service (TMLaaS) Architecture 349
9.4 Results and Discussion 354
9.5 Open Challenges and Further Research Directions 357
9.6 Conclusion 358
References 359
10 Advanced Security Schemes for TinyML Devices 367
Wasswa Shafik and Mumin Adam
10.1 Introduction 367
10.2 Fundamentals of TinyML 369
10.3 Privacy Concerns in TinyML 374
10.4 Security and Privacy Solutions 376
10.5 The Implementation of Novel Security Schemes for TinyML Applications 384
10.6 Privacy-Enhancing Techniques for TinyML 388
10.7 Future Directions of Security and Privacy in TinyML Devices 391
10.8 Lessons Learned 393
10.9 Limitations of this Study 394
10.10 Conclusion 395
References 396
11 Robust Ground Truth Data Mining for Enhanced Privacy and Accuracy in Noisy TinyML Environments 403
Yuichi Sei and Agbotiname Lucky Imoize
11.1 Introduction 403
11.2 Related Research Work 406
11.3 Models 408
11.4 Gdp 412
11.5 Evaluation 416
11.6 Discussion 420
11.7 Conclusions 423
References 424
12 Security and Privacy of TinyML Devices 431
Eftychia Mistillioglou, Evangelia Konstantopoulou, Nicolas Sklavos, and Andronikos Kyriakou
12.1 Introduction 431
12.2 Related Work 433
12.3 Secure and Privacy-Aware Training of TinyML Models 437
12.4 Implementation of Novel Security Schemes for TinyML Applications 455
12.5 Lessons, Challenges, and Future Directions 463
12.6 Conclusions and Outlook 464
References 465
13 Semantic Management of TinyML for Industrial Application 469
Kinzah Noor, Hasnain Ahmad, and Agbotiname Lucky Imoize
13.1 Introduction 469
13.2 Introduction to TinyML 472
13.3 Recent Advances in TinyML 477
13.4 Methodology 488
13.5 Results and Discussion 495
13.6 Conclusions and Future Scope 498
References 498
14 Fight Poison with Poison: Tiny Machine Learning Resilience Against Poisoning Attacks 503
Tomoki Chiba, Yasuyuki Tahara, Akihiko Ohsuga, Agbotiname Lucky Imoize, and Yuichi Sei
14.1 Introduction 503
14.2 Problem Definition 505
14.3 Related Work 507
14.4 Proposed Method 512
14.5 Evaluation Experiment 523
14.6 Discussion 541
14.7 Conclusion 543
References 544
15 TinyML for Real-Time Medical Image Classification and Diagnosis 549
Jelil O. Agbo-Ajala, Lateef A. Akinyemi, Olufisayo S. Ekundayo, and Ernest Mnkandla
15.1 Introduction 549
15.2 Literature Review 551
15.3 Methodology 566
15.4 Results and Discussion 567
15.5 Conclusion 578
References 578
16 Biometric Authentication in TinyML: Opportunities and Challenges 587
Oleksandr Kuznetsov, Emanuele Frontoni, Marco Arnesano, Oleksii Smirnov, and Boris Khruskov
16.1 Introduction 587
16.2 Related Work 589
16.3 Overview of Biometric Authentication Techniques 591
16.4 Comparative Analysis of Biometric Authentication Methods 609
16.5 Adapting Biometric Techniques for TinyML Systems 615
16.6 Discussion and Future Directions 625
16.7 Conclusion 628
Acknowledgment 628
References 629
17 Secure Deployment of TinyML Applications: Strategies and Practices 635
Oleksandr Kuznetsov, Sergii Kavun, and Gulvira Bekeshova
17.1 Introduction 635
17.2 Related Work 636
17.3 Security Architectures for TinyML Deployments 638
17.4 Secure Bootstrapping and Key Management 643
17.5 Secure Communication Protocols for TinyML 646
17.6 Data Privacy in TinyML Applications 650
17.7 Future Directions and Emerging Technologies 652
17.8 Conclusion 655
Acknowledgment 657
References 657
18 TinyML for Environmental Applications 665
Duy Nam Khanh Vu and Anh Khoa Dang
18.1 Introduction 665
18.2 Related Work 667
18.3 Methodology 669
18.4 Case Study: New Insights on Air Writing from Pławiak and Alblehai 689
18.5 Results and Discussion 692
18.6 Conclusion and Future Scope 697
References 697
19 Benchmarking TinyML Encrypted Federated Learning with Secret Sharing in Medical Computer Vision 701
Ruduan B. F. Plug, Putu H. P. Jati, Samson Y. Amare, and Mirjam van Reisen
19.1 Introduction 701
19.2 Related Work 702
19.3 Methodology 703
19.4 Results 711
19.5 Conclusion 715
References 716
Index 721
About the Author :
Agbotiname Lucky Imoize is a Lecturer in the Department of Electrical and Electronics Engineering at the University of Lagos, Nigeria. He is a Fulbright Fellow, the Vice Chair of the IEEE Communication Society Nigeria chapter, and a Senior Member of IEEE.
Dinh-Thuan Do, PhD, is an Assistant Professor with the School of Engineering at the University of Mount Union, USA. He is an editor of IEEE Transactions on Vehicular Technology and Computer Communications. He is a Senior Member of IEEE.
Houbing Herbert Song, PhD, IEEE Fellow, is a Professor in the Department of Information Systems, and the Department of Computer Science and Electrical Engineering and Director of the Security and Optimization for Networked Globe Laboratory (SONG Lab) at the University of Maryland, Baltimore County. He is also Co-Editor-in-Chief of IEEE Transactions on Industrial Informatics.
