The book offers an introduction to large language models (LLMs) that bridge foundational natural language processing (NLP) concepts with the advanced techniques underlying large language models. It offers a structured exploration of NLP evolution, from rule- based approaches to transformer architectures. Covering key principles such as tokenization, attention mechanisms, and model architectures (BERT, GPT, T5), the book explains pretraining objectives like masked and causal language modeling. It also addresses optimization techniques such as LoRA, pruning, and quantization for efficient LLM deployment. Multimodal models, including GPT- 4 and PaLM- E, are explored alongside retrieval- augmented generation and AI- powered agents.
Features:
• Discusses foundational NLP concepts, theoretical depth, advanced techniques, and real- world applications.
• Covers perplexity, BLEU, ROUGE, and datasets like SuperGLUE and SQuAD for assessing LLM performance, discusses LoRA, pruning, and quantization to optimize LLM deployment in resource- constrained settings.
• Explores GPT- 4, PaLM- E, and retrieval- augmented generation, expanding beyond traditional NLP models.
• Provides Python implementations for fine- tuning, classification, summarization, and conversational AI tasks.
• Highlights use cases in text generation, code generation, sentiment analysis, and multimodal AI.
This book is an invaluable textbook for students, researchers, and industry professionals seeking a deep technical understanding of LLMs and their applications.
Table of Contents:
Chapter 1. Introduction to Natural Language Processing (NLP). 1.1 Brief History of NLP. 1.2 Key NLP Tasks. 1.3 Basics of Language Models. 1.4 Challenges in NLP. Chapter 2. Linguistic Foundations and Feature Representation of NLP. 2.1 Levels of NLP in computational linguistics. 2.2 Morphology. 2.3 Syntax and Semantics. 2.4 Lexical and Compositional Semantics. 2.5 Vector Space Model (VSM). 2.6 Word Embeddings. Chapter 3. Sequence Modelling with Neural Networks. 3.1 Recurrent Neural Networks (RNNs). 3.2 Convolutional Neural Networks (CNNs) for NLP. 3.3 Sequence-to-Sequence (Seq2Seq) Models. Chapter 4. Attention Mechanism and Pre-trained Language Models. 4.1 Introduction to the Transformer Architecture. 4.2 Attention Mechanism. 4.3 Pretrained Language Models. 4.4 Pretraining Objectives: MLM and Causal LM. 4.5 Fine-Tuning vs. Prompting. 4.6 Transfer Learning for NLP. Chapter 5. Fundamentals of Large Language Models. 5.1 Tokenization. 5.2 Encoding Positions. 5.3 Activation Functions. 5.4 Layer Normalization. 5.5 Distributed LLM Training. 5.6 Libraries. Chapter 6. Variants of LLM Architecture. 6.1 Encoder-only architecture. 6.2 Decoder-only architecture. 6.3 Encoder-Decoder architecture. 6.4 Other variants. Chapter 7. Pre-Trained LLMs. 7.1 Single Modal pre-trained LLMs.7.2 Multi Modal pre-trained LLMs. Chapter 8. Fine Tuning of LLMs. 8.1 Introduction to Fine-tuning. 8.2 Process of LLM fine-tuning. 8.3 Types of Fine-tuning methods. 8.4 Instruction Tuning. 8.5 Alignment Techniques. 8.6 Advanced Fine-tuning Techniques. 8.7 Challenges in Fine-tuning. Chapter 9. Efficient Large Language Models (LLMs). 9.1 Introduction to Efficient LLMs. 9.2 Parameter-Efficient Fine-Tuning Techniques. 9.3 Quantization Techniques. 9.4 Pruning Strategies. 9.5 Efficient Attention Mechanisms. 9.6 Distributed and Parallel Computing for LLMs. 9.7 Energy Efficiency in LLM Inference. 9.8 Optimizing LLMs for Edge Devices. 9.9 Practical Applications of Efficient LLMs. Chapter 10. Increasing Context Window. 10.1 Position Interpolation. 10.2 Efficient Attention Mechanism. 10.3 Extrapolation without Training. Chapter 11. Augmented LLMs. 11.1 Retrieval Augmented LLMs-Introduction. 11.2 Classification of Retrieval Augmented LLMs. 11.3 Retrieval Augmented Generation (RAG). Chapter 12. LLMs-Powered Agents. 12.1 LLMs Steering Autonomous Agents. 12.2 LLMs in Physical Environment. Chapter 13. Evaluation of LLMs. 13.1 Natural Language Understanding. 13.2 Natural Language Generation. 13.3 Metrics for Language Models. 13.4 Benchmarking LLMs. Chapter 14. Applications of Large Language Models. 14.1 Generative Applications. 14.2 Task-Specific Applications. 14.3 Multimodal Language Models. Chapter 15. Ethical Considerations in Large Language Models. 15.1 Introduction. 15.2 Bias and Fairness in LLMs. 15.3 Explainability and Transparency. 15.4 Privacy and Data Protection. 15.5 Misinformation and Content Moderation. 15.6 Intellectual Property and Plagiarism. 15.7 Environmental Impact of LLMs. 15.8 Regulation and Governance of LLMs. 15.9 Ethical Use Cases and Best Practices
About the Author :
Joe Dhanith P. R. is an Assistant Professor at the Vellore Institute of Technology (VIT), Chennai, with a strong academic and research background in the field of computer science. He has made notable contributions to high- impact journals, particularly in advancing the domains of natural language processing (NLP), web mining, and information retrieval. His research focuses on developing intelligent systems that enhance text understanding, knowledge extraction, and data- driven decision- making. Dr. Joe has been actively involved in mentoring students, guiding research projects, and delivering courses related to programming, machine learning, and deep learning. His work reflects a commitment to bridging theoretical foundations with practical applications, contributing to both academic research and real- world problem solving.
Geetha S. is a Professor and Associate Dean (Research) in School of Computer Science and Engineering, VIT University, Chennai Campus, India. She has received the B.E. and M.E. degrees in Computer Science and Engineering from Madurai Kamaraj University, India and Anna University of Chennai, India, and Ph.D. degree from Anna University, respectively. She has 23 years of rich teaching and research experience. She has published more than 100 papers in reputed International conferences and refereed journals. Her h- Index is 21 and i- 10 index is 46. Her research interests include image processing, deep learning, feature selection, steganography, steganalysis, multimedia security, intrusion detection systems, malware detection, machine learning paradigms, and information forensics. She has given many expert lectures and keynote addresses in international and national conferences. She has organized many workshops, conferences, and FDPs. She is a recipient of University Rank and Academic Topper Award in B.E. and M.E, respectively. She is also the proud recipient of ASDF Best Academic Researcher Award 2013, ASDF Best Professor Award 2014, Research Award 2016– 2020, High Performer Award 2016, from VIT University and DST – ISCA Best Poster Award 2018, Innovation Award 2022 – Cybersecurity Roadshow conducted in IISc.
Sheik Abdullah A. is working as an Associate Professor in the School of Computer Science Engineering, Vellore Institute of Technology, Chennai. He completed his Ph.D. from Anna University. He is a visiting faculty at The Institute of Mathematical Sciences IMSC Chennai and contributed and worked in computational biology, mathematical decision support models in clinical informatics, data analytics, and statistics. He is also a visiting researcher at Chennai Mathematical Institute (CMI) and is engaged in works corresponding to Timed automata and its applications. More recently, he has contributed his novelty in assessing risk factors that contribute to type II diabetes with swarm intelligence and machine learning approaches. His works correspond to real- time analysis of medical data and medical experts with the development of clinical decision support models for hospitals in rural areas. He also contributed his research intelligence in NLP, big data, knowledge- based systems, E- governance, learning analytics, and probabilistic planning algorithms. He has published over 45+ archival research papers to his credit, 15 book chapters, and a book. Being a Gold medalist (PG), he has been awarded the honorable chief minister award for the best project in E- governance.
