Securing Microsoft Azure OpenAI: (Tech Today)

Securely harness the full potential of OpenAI’s artificial intelligence tools in Azure Securing Microsoft Azure OpenAI is an accessible guide to leveraging the comprehensive AI capabilities of Microsoft Azure while ensuring the utmost data security. This book introduces you to the collaborative powerhouse of Microsoft Azure and OpenAI, providing easy access to cutting-edge language models like GPT-4o, GPT-3.5-Turbo, and DALL-E. Designed for seamless integration, the Azure OpenAI Service revolutionizes applications from dynamic content generation to sophisticated natural language translation, all hosted securely within Microsoft Azure’s environment. Securing Microsoft Azure OpenAI demonstrates responsible AI deployment, with a focus on identifying potential harm and implementing effective mitigation strategies. The book provides guidance on navigating risks and establishing best practices for securely and responsibly building applications using Azure OpenAI. By the end of this book, you’ll be equipped with the best practices for securely and responsibly harnessing the power of Azure OpenAI, making intelligent decisions that respect user privacy and maintain data integrity.

Table of Contents:
Introduction xxiii Chapter 1 Overview of Generative Artificial Intelligence Security 1 Common Use Cases for Generative AI in the Enterprise 1 Generative Artificial Intelligence 1 Generative AI Use Cases 2 LLM Terminology 3 Sample Three-Tier Application 4 Presentation Tier 5 Application Tier 5 Data Tier 5 Generative AI Application Risks 5 Hallucinations 6 Malicious Usage 6 Shadow AI 7 Unfavorable Business Decisions 8 Established Risks 8 Shared AI Responsibility Model 8 Shared Responsibility Model for the Cloud 9 Shared Responsibility Model for AI 10 AI Usage 10 AI Application 10 AI Platform 11 Applying the Shared Responsibility Model 11 Regulation and Control Frameworks 12 Regulation in the United States 12 Regulation in the European Union 12 NIST AI Risk Management Framework 14 Govern 15 Map 15 Measure 16 Manage 16 Key Takeaways 16 References 17 Chapter 2 Security Controls for Azure OpenAI Service 19 On the Importance of Selecting Appropriate Security Controls 19 Risk Appetite 20 Comparing OpenAI Hosting Models 21 OpenAI ChatGPT 21 Privacy and Compliance 21 Identity and Access Management 21 Data Protection and Encryption 22 Audit Logging 22 Network Isolation 22 Data Residency 22 Azure OpenAI 22 Privacy and Compliance 23 Identity and Access Management 23 Data Protection and Encryption 23 Audit Logging 23 Network Isolation 23 Data Residency 23 Recommendation for Enterprise Usage 24 Evaluating Security Controls with MCSB 24 Control Domains 26 Network Security 27 Identity Management 28 Privileged Access 28 Data Protection 29 Asset Management 29 Logging and Threat Detection 29 Incident Response 30 Posture and Vulnerability Management 30 Endpoint Security 31 Backup and Recovery 31 DevOps Security 32 Governance and Strategy 32 Security Baselines 33 Applying Microsoft Cloud Security Baseline to Azure OpenAI 33 Security Profile 34 How to Approach the Security Baseline 34 Data Protection 35 Identity Management 36 Logging and Threat Detection 37 Network Security 38 Asset Management 38 Backup and Recovery 39 Endpoint Security 40 Posture and Vulnerability Management 40 Privileged Access 41 Selected Controls 42 Mapping the Selected Controls to CIS and NIST 44 Using Azure Policy to Secure Azure OpenAI at Scale 46 Azure Policy 46 Continuous Compliance Monitoring 47 Azure Policies for Azure OpenAI 48 Key Takeaways 49 References 49 Chapter 3 Implementing Azure OpenAI Security Controls 51 OWASP Top 10 for LLM Applications 51 Prompt Injection 52 Insecure Output Handling 52 Training Data Poisoning 53 Model Denial of Service 53 Supply Chain Vulnerabilities 53 Sensitive Information Disclosure 54 Insecure Plugin Design 54 Excessive Agency 54 Overreliance 55 Model Theft 55 Access Control 56 Implementing Access Control for Azure OpenAI 56 Cognitive Services OpenAI User 57 Cognitive Services OpenAI Contributor 58 Azure AI Administrator 59 Azure AI Developer 61 Azure AI Enterprise Network Connection Approver 62 Azure AI Inference Deployment Operator 64 Preventing Local Authentication 65 Disable Local Authentication Using Bicep 66 Disable Local Authentication Using Terraform 66 Disable Local Authentication Using ARM Templates 67 Prevent Local Authentication Using PowerShell 67 Enforcing with Azure Policy 67 Audit Logging 68 Control Plane Audit Logging 68 Data Plane Audit Logging 71 Enable Data Plane Audit Logging Using Azure Portal 72 Enable Data Plane Audit Logging Using Bicep 73 Enable Data Plane Audit Logging Using Terraform 73 Enable Data Plane Audit Logging Using ARM Templates 74 Enable Data Plane Audit Logging Using PowerShell 76 Enable Data Plane Audit Logging Using Azure cli 76 Enforcing with Azure Policy 77 Enable Logging by Category Group for Cognitive Services 77 Network Isolation 82 Default Network Controls 83 Control Inbound Network Traffic 83 Control Inbound Network Traffic Using the Azure Portal 84 Control Inbound Network Traffic Using Bicep 84 Control Inbound Network Traffic with Private Endpoints Using Infrastructure as Code 85 Control Inbound Network Traffic Using Terraform 87 Control Inbound Network Traffic with Private Endpoints Using Terraform 87 Control Inbound Network Traffic Using ARM Templates 89 Control Inbound Network Traffic with Private Endpoints Using ARM Templates 90 Control Inbound Network Traffic Using PowerShell 93 Control Inbound Network Traffic with Private Endpoints Using PowerShell 94 Control Inbound Network Traffic Using Azure cli 95 Control Inbound Network Traffic with Private Endpoints Using Azure cli 95 Control Outbound Network Traffic 97 Enable Data Loss Prevention Using REST 97 Enable Data Loss Prevention Using Bicep 98 Enable Data Loss Prevention Using Terraform 98 Enable Data Loss Prevention Using ARM Templates 99 Enforcing with Azure Policy 101 Azure AI Services Resources Should Restrict Network Access 101 Azure AI Services Resources Should Use Azure Private Link 103 Encryption at Rest 105 Implementing Azure OpenAI with CMK 106 Implement CMK Using Azure Portal 106 Implement CMK Using Bicep 107 Implement CMK Using Terraform 109 Implement CMK Using ARM Templates 111 Implement CMK Using PowerShell 114 Implement CMK Using the Azure cli 115 Enforcing with Azure Policy 116 Azure AI Services Resources Should Encrypt Data at Rest with a CMK 117 Content Filtering Controls 119 System Safety Prompts 119 Azure AI Content Safety 120 Content Filtering 120 Prompt Shields 121 Protected Material Detection 121 Groundedness Detection 121 Creating a Content Filter 121 Implementing Content Filtering Programmatically 122 Content Safety Input Restrictions 123 Key Takeaways 123 References 124 Chapter 4 Securing the Entire Application 125 The Three-Tier LLM Application in Azure 125 Presentation Tier 126 Application Tier 126 Data Tier 126 On Threat Modeling 126 Threat Model of the Three-Tier Application 127 Revised Application Architecture 129 Retrieval-Augmented Generation 129 RAG in Azure 130 Azure AI Search 130 Azure Cosmos DB 131 Application Architecture with RAG 131 Azure Front Door 132 Security Profile 132 Security Baseline 132 Implementing Security Controls 133 Access Control 133 Audit Logging 133 Network Isolation 141 Encryption at Rest 152 Enforcing Controls with Policies 152 Azure App Service 153 Security Profile 153 Security Baseline 153 Implementing Security Controls 155 Access Control 156 Audit Logging 163 Network Isolation 169 Encryption at Rest 176 Enforcing Controls with Policies 176 API Management 177 Security Profile 177 Security Baseline 178 Implementing Security Controls 178 Access Control 179 Audit Logging 180 Network Isolation 186 Encryption at Rest 201 Enforcing Controls with Policies 202 Storage Account 202 Security Profile 202 Security Baseline 203 Implementing Security Controls 204 Access Control 204 Audit Logging 209 Network Isolation 216 Encryption at Rest 225 Backup and Recovery 232 Discover, Classify, and Protect Sensitive Data 238 Enforcing Controls with Policies 238 Cosmos DB 238 Security Profile 239 Security Baseline 239 Implementing Security Controls 241 Access Control 241 Audit Logging 244 Network Isolation 249 Encryption at Rest 256 Backup and Recovery 262 Enforcing Controls with Policies 266 Azure AI Search 266 Security Profile 266 Security Baseline 267 Implementing Security Controls 268 Access Control 268 Audit Logging 272 Network Isolation 278 Encryption at Rest 287 Enforcing Controls with Policies 294 Key Takeaways 294 References 294 Chapter 5 Moving to Production 297 LLM Application Security Lifecycle 297 Model Supply Chain 298 Security Testing 299 Model Safety Evaluation 299 How to Use Model Safety Evaluation 300 Adversarial Testing 300 How to Use the Adversarial Simulator Service 301 Red Teaming 304 Crescendo Multiturn Attack 304 Red Teaming with PyRIT 304 Content Credentials 305 AI Security Posture Management 307 Discover and Manage Shadow AI 307 Discover SaaS Applications 307 Discover Generative AI Applications 309 Manage Generative AI Applications 312 Alert on Anomalous Activity and Applications 313 Defender for Cloud AI Workloads 314 Discovery 314 Posture Management 314 Security Alerting 314 Security Posture Management 315 Investigating Security Alerts 316 Alert Details 317 Supporting Evidence 318 Take Action 319 Managing Incidents 323 Instrumenting Security Alert Ingestion 324 Azure OpenAI Alerts 326 Detected Credential Theft Attempts on an Azure OpenAI Model Deployment 327 A Jailbreak Attempt on an Azure OpenAI Model Deployment Was Blocked by Azure AI Content Safety Prompt Shields 327 A Jailbreak Attempt on an Azure OpenAI Model Deployment Was Detected by Azure AI Content Safety Prompt Shields 327 Sensitive Data Exposure Detected in Azure OpenAI Model Deployment 327 Corrupted AI Application, Model, or Data Directed a Phishing Attempt at a User 328 Phishing URL Shared in an AI Application 328 Phishing Attempt Detected in an AI Application 328 Defender for Cloud Alerts for Other Services 328 App Service Alerts 329 API Management Alerts 330 Storage Account Alerts 331 Cosmos DB Alerts 332 LLM Application in Your Cloud Security Architecture 332 Cloud Security Control Domains 333 Asset Management 333 Incident Response 334 Privileged Access 336 Posture and Vulnerability Management 337 Landing Zones 339 About Landing Zones 339 Microsoft Enterprise-Scale Landing Zones 339 Microsoft Landing Zone Accelerator for OpenAI 342 LLM Application in the Landing Zone 342 The Sample Application in the Landing Zone 342 Access Control 343 Security Monitoring 343 Incident Response 344 Network 344 Key Takeaways 345 References 345 Index 347

About the Author :
KARL OTS is Global Head of Cloud Security at EPAM Systems, an engineering and consulting firm. He leads a team of experts in delivering security and compliance solutions for cloud and AI deployments for Fortune 500 enterprises in a variety of industries. He has over 15 years’ experience in tech and is a trusted advisor and thought leader. Karl is also a Microsoft Regional Director and Security MVP.