This book explores the integration and interplay of model-based optimization and model-free deep reinforcement learning (DRL). It addresses the growing complexity of future wireless networks. This book begins with a concise overview of foundational DRL algorithms and then delves into advanced frameworks, including optimization-driven DRL, hierarchical DRL, multi-agent DRL, Bayesian-enhanced DRL, and Lyapunov-guided DRL. Each framework is illustrated through case studies in emerging scenarios such as intelligent reflecting surface (IRS)-assisted wireless communications, UAV-assisted wireless networks, backscatter-assisted relay communications, and mobile edge computing.
Each chapter of this book demonstrates how partial system knowledge, inherent structural properties, and problem decomposition can dramatically accelerate learning convergence. It also improves sample efficiency, and enhance robustness in decentralized, dynamic, and large-scale wireless networks.
Tailored for researchers and graduate students focused on wireless communications and AI-driven networking, it bridges theoretical innovation with practical implementation challenges. It provides a roadmap for designing intelligent, autonomous, and resource-efficient next-generation wireless systems. Engineers and professional specializing in AI and machine learning for wireless systems will also find this book useful as a reference.
Table of Contents:
Preface.- Chapter 1 Introduction.- Chapter 2 Optimization-driven DRL in Wireless Networks.- Chapter 3 Hierarchical DRL for Heterogeneous Wireless Networks.- Chapter 4 Hierarchical DRL for IRS-assisted AoI Minimization.- Chapter 5 Hierarchical MADRL for Mobile Edge Computing.- Chapter 6 Hierarchical MADRL for UAV-assisted Wireless Networks.- Chapter 7 Lyapunov-guided DRL for Stochastic AoI Minimization.- Chapter 8 Summary.
About the Author :
Shimin Gong received the B.Eng. and M.Eeng. degrees in electronics and information engineering from the Huazhong University of Science and Technology, Wuhan, China, in 2008 and 2012, respectively, and the Ph.D. degree in computer engineering from Nanyang Technological University, Singapore, in 2014. He is currently a Professor with the School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China. His research interests include wireless powered communications, backscatter communications, and machine learning in wireless communications. He was a co-recipient of the IEEE WCNC 2019 Best Paper Award on MAC and Cross-layer Design and the 2023 IEEE Communications Society Best Survey Paper Award. He is an Associate Editor of IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY.
Dusit Niyato (Fellow, IEEE) received the B.Eng. from the King Mongkuts Institute of Technology Ladkrabang, Bangkok, Thailand, in 1999 and the Ph.D. degree in electrical and computer engineering from the University of Manitoba, Winnipeg, MB, Canada, in 2008. He is currently a Professor with the School of Computer Science and Engineering, Nanyang Technological University, Singapore. His research interests include the area of energy harvesting for wireless communication, Internet of Things, and sensor networks.
Bo Gu received the Ph.D. degree from Waseda University, Tokyo, Japan, in 2013. He is currently a Professor with the School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China. He was a Research Engineer with Sony Digital Network Applications, Yokohama, Japan, from 2007 to 2011, an Assistant Professor with Waseda University from 2011 to 2016, and an Associate Professor with Kogakuin University, Tokyo, from 2016 to 2018. His research interests include the Internet of Things, edge computing, network economics, and machine learning. He was a recipient of the IEEE ComSoc Communications Systems Integration and Modeling (CSIM) Technical Committee Best Journal Article Award in 2019, the Asia-Pacific Network Operations and Management Symposium (APNOMS) Best Paper Award in 2016, and the IEICE Young Researcher’s Award in 2011. He is a member of IEICE.
Kaibin Huang (Fellow, IEEE) received the B.Eng. and M.Eng. degrees from the National University of Singapore, and the Ph.D. degree from The University of Texas at Austin, all in electrical engineering. He is a Philip K, H, Wong Wilson K, L, Wong Professor in electrical engineering and the Department Head with the Department of Electrical and Electronic Engineering, The University of Hong Kong (HKU), Hong Kong. His work was recognized with seven Best Paper awards from the IEEE Communications Society. He is a member of the Engineering Panel of Hong Kong Research Grants Council (RGC) and a RGC Research Fellow (2021 Class).
