Hands-On Generative AI with Transformers and Diffusion Models

Learn how to use generative media techniques with AI to create novel images or music in this practical, hands-on guide. Data scientists and software engineers will understand how state-of-the-art generative models work, how to fine-tune and adapt them to your needs, and how to combine existing building blocks to create new models and creative applications in different domains. This book introduces theoretical concepts in an intuitive way, with extensive code samples and illustrations that you can run on services such as Google Colaboratory, Kaggle, or Hugging Face Spaces with minimal setup. You'll learn how to use open source libraries such as Transformers and Diffusers, conduct code exploration, and study several existing projects to help guide your work. Learn the fundamentals of classic and modern generative AI techniques Build and customize models that can generate text, images, and sound Explore trade-offs between training from scratch and using large, pretrained models Create models that can modify images by transferring the style of other images Tweak and bend transformers and diffusion models for creative purposes Train a model that can write text based on your style Deploy models as interactive demos or services

About the Author :
Omar Sanseviero is the Lead of Developer Advocacy Engineering at Hugging Face, where he builds collaborations with different libraries in the ML Ecosystem. Omar has extensive engineering experience working in Google in Google Assistant and TensorFlow Graphics. Omar's work at Hugging Face is at the intersection of community, engineering, and product, allowing him to have a horizontal understanding of the ML ecosystem and trends. Pedro Cuenca is a Machine Learning Engineer at Hugging Face working on diffusion software, models, and applications. He has 20+ years of software development experience in fields like Internet applications (in Spain, he helped create the first interactive educational portal, the first book store, and the first free ISP) and, more recently, iOS. As a co-founder and CTO of LateNiteSoft, he worked on the technology behind Camera+, a successful iPhone photography app. He created deep-learning models for tasks such as photography enhancement and super-resolution. He was also involved in the development and operations behind dalle-mini. He brings a practical vision of integrating AI research into real-world services and the challenges and optimizations involved. Apolinario Passos is a Machine Learning Art Engineer at Hugging Face working across different teams on multiple machine learning for art and creativity use-cases. Apolinario has 10+ years of professional and artistic experience, alternating between holding art exhibitions, coding, and product management, having been a Head of Product in World Data Lab. Apolinario aims to ensure that the ML ecosystem supports and makes sense for artistic use cases.Jonathan Whitaker is a data scientist and deep learning researcher focused on generative modelling. He created and taught the 'AIAIART' course and is working on a new version called 'The Generative Landscape' which covers many of the topics this book hopes to address. He also wrote the Hugging Face diffusion models class and is working with Jeremy Howard on the ongoing FastAI course -'Stable Diffusion from the Foundations '. Jonathan also works as a consultant, currently part-time as a Builder-In-Residence with PlaygroundAI.