About the Book
How is it possible to allow multiple data owners to collaboratively train and use a shared prediction model while keeping all the local training data private?
Traditional machine learning approaches need to combine all data at one location, typically a data center, which may very well violate the laws on user privacy and data confidentiality. Today, many parts of the world demand that technology companies treat user data carefully according to user-privacy laws. The European Union's General Data Protection Regulation (GDPR) is a prime example. In this book, we describe how federated machine learning addresses this problem with novel solutions combining distributed machine learning, cryptography and security, and incentive mechanism design based on economic principles and game theory. We explain different types of privacy-preserving machine learning solutions and their technological backgrounds, and highlight some representative practical use cases. We show how federated learning can become the foundation of next-generation machine learning that caters to technological and societal needs for responsible AI development and application.
Table of Contents:
Preface.- Acknowledgments.- Introduction.- Background.- Distributed Machine Learning.- Horizontal Federated Learning.- Vertical Federated Learning.- Federated Transfer Learning.- Incentive Mechanism Design for Federated Learning.- Federated Learning for Vision, Language, and Recommendation.- Federated Reinforcement Learning.- Selected Applications.- Summary and Outlook.- Bibliography.- Authors' Biographies.
About the Author :
Qiang Yang is the head of the AI department at WeBank (Chief AI Officer) and Chair Professor at the Computer Science and Engineering (CSE) Department of the Hong Kong University of Science and Technology (HKUST), where he was a former head of CSE Department and founding director of the Big Data Institute (2015-2018). His research interests include AI, machine learning, and data mining, especially in transfer learning, automated planning, federated learning, and case-based reasoning. He is a fellow of several international societies, including ACM, AAAI, IEEE, IAPR, and AAAS. He received his Ph.D. in Computer Science in 1989 and his M.Sc. in Astrophysics in 1985, both from the University of Maryland, College Park. He obtained his B.Sc. in Astrophysics from Peking University in 1982. He had been a faculty member at the University of Waterloo (1989-1995) and Simon Fraser University (1995-2001). He was the founding Editor-in-Chief of the ACM Transactions on Intelligent Systems and Technology (ACM TIST)and IEEE Transactions on Big Data (IEEE TBD). He served as the President of International Joint Conference on AI (IJCAI, 2017-2019) and an executive council member of Association for the Advancement of AI (AAAI, 2016-2020). Qiang Yang is a recipient of several awards, including the 2004/2005 ACM KDDCUP Championship, the ACM SIGKDD Distinguished Service Award (2017), and AAAI Innovative AI Applications Award (2016). He was the founding director of Huawei's Noah's Ark Lab (2012-2014) and a co-founder of 4Paradigm Corp, an AI platform company. He is an author of several books including Intelligent Planning (Springer), Crafting Your Research Future (Morgan & Claypool), and Constraint-based Design Recovery for Software Engineering (Springer).Yang Liu is a Senior Researcher in the AI Department of WeBank, China. Her research interests include machine learning, federated learning, transfer learning, multi-agent systems, statistical mechanics, and applications of these technologies in the financial industry. She received her Ph.D. from Princeton University in 2012 and her Bachelor's degree from Tsinghua University in 2007. She holds multiple patents. Her research has been published in leading scientific journals such as ACM TIST and Nature.Yong Cheng is currently a Senior Researcher in the AI Department of WeBank, Shenzhen, China. Previously, he had worked in Huawei Technologies Co., Ltd. (Shenzhen) as a Senior Engineer, and in Bell Labs Germany as a Senior Researcher. Yong had also worked as a Researcher in the Huawei-HKUST Innovation Laboratory, Hong Kong. His research interests and expertise mainly include Deep Learning, Federated Learning, Computer Vision and OCR, Mathematical Optimization and Algorithms, Distributed Computing, as well as Mixed-Integer Programming. He has published more than 20 journal and conference papers and filed more than 40 patents. Yong received the B.Eng. (1st class honors), MPhil, and Ph.D. (1st class honors) degrees from Zhejiang University (ZJU), Hangzhou, PR China, the Hong Kong University of Science and Technology (HKUST), Hong Kong, and Technische Universitat Darmstadt (TU Darmstadt), Darmstadt, Germany, in 2006, 2010, and 2013, respectively. He received the best Ph.D. thesis award of TU Darmstadt in 2014, and the best B.Eng. thesis award of ZJU in 2006. Yong gave a tutorial on ""Mixed-Integer Conic Programming"" at ICASSP'15, and he was the PC Member of FML'19 (in conjunction with IJCAI'19).Yan Kang is a Senior Researcher in the AI department of Webank in Shenzhen, China. His work is focusing on the research and implementation of privacy-preserving machine learning and federated transfer learning techniques. He received M.S. and Ph.D. degrees in Computer Science from the University of Maryland, Baltimore County, USA. His Ph.D. work was awarded a doctoral fellowship and centered around machine learning and semantic web for heterogeneous data integration. During his graduate work,he participated in multiple projects collaborating with the National Institute of Standards and Technology (NIST) and the National Science Foundation (NSF) for designing and developing ontology integration systems. He also has adequate experiences in commercial software projects. Before joining WeBank, he had been working for Stardog Union Inc. and Cerner Corporation for more than four years on system design and implementation.Tianjian Chen is the Deputy General Manager of the AI Department of WeBank, China. He is now responsible for building the Banking Intelligence Ecosystem based on Federated Learning Technology. Before joining WeBank, he was the Chief Architect of Baidu Finance, Principal Architect of Baidu. Tianjian has over 12 years of experience in large-scale distributed system design and enabling technology innovations in various application fields, such as web search engine, peer-to-peer storage, genomics, recommender system, digital banking, and machine learning.Han Yu is a Nanyang Assistant Professor (NAP) in the School of Computer Science and Engineering (SCSE), Nanyang Technological University (NTU), Singapore. Between 2015 and 2018, he held the prestigious Lee Kuan Yew Post-Doctoral Fellowship (LKY PDF). Before joining NTU, he worked as an Embedded Software Engineer at Hewlett-Packard (HP) PteLtd, Singapore. He obtained his Ph.D. in Computer Science from NTU in 2014. His research focuses on online convex optimization, ethical AI, federated learning, and their applications in complex collaborative systems such as crowdsourcing. He has published over 120 research papers leading international conferences and journals and won multiple research awards.
