Network Security with NetFlow and IPFIX: Big Data Analytics for Information Security

A comprehensive guide for deploying, configuring, and troubleshooting NetFlow and learning big data analytics technologies for cyber security Today’s world of network security is full of cyber security vulnerabilities, incidents, breaches, and many headaches. Visibility into the network is an indispensable tool for network and security professionals and Cisco NetFlow creates an environment where network administrators and security professionals have the tools to understand who, what, when, where, and how network traffic is flowing. Network Security with NetFlow and IPFIX is a key resource for introducing yourself to and understanding the power behind the Cisco NetFlow solution. Omar Santos, a Cisco Product Security Incident Response Team (PSIRT) technical leader and author of numerous books including the CCNA Security 210-260 Official Cert Guide, details the importance of NetFlow and demonstrates how it can be used by large enterprises and small-to-medium-sized businesses to meet critical network challenges. This book also examines NetFlow’s potential as a powerful network security tool. Network Security with NetFlow and IPFIX explores everything you need to know to fully understand and implement the Cisco Cyber Threat Defense Solution. It also provides detailed configuration and troubleshooting guidance, sample configurations with depth analysis of design scenarios in every chapter, and detailed case studies with real-life scenarios. You can follow Omar on Twitter: @santosomar NetFlow and IPFIX basics Cisco NetFlow versions and features Cisco Flexible NetFlow NetFlow Commercial and Open Source Software Packages Big Data Analytics tools and technologies such as Hadoop, Flume, Kafka, Storm, Hive, HBase, Elasticsearch, Logstash, Kibana (ELK) Additional Telemetry Sources for Big Data Analytics for Cyber Security Understanding big data scalability Big data analytics in the Internet of everything Cisco Cyber Threat Defense and NetFlow Troubleshooting NetFlow Real-world case studies

Table of Contents:
Introduction xvi Chapter 1 Introduction to NetFlow and IPFIX 1 Introduction to NetFlow 1 The Attack Continuum 2 The Network as a Sensor and as an Enforcer 3 What Is a Flow? 4 NetFlow Versus IP Accounting and Billing 6 NetFlow for Network Security 7 Anomaly Detection and DDoS Attacks 8 Data Leak Detection and Prevention 9 Incident Response and Network Security Forensics 9 Traffic Engineering and Network Planning 14 IP Flow Information Export 15 IPFIX Architecture 16 IPFIX Mediators 17 IPFIX Templates 17 Option Templates 19 Introduction to the Stream Control Transmission Protocol (SCTP) 19 Supported Platforms 20 Introduction to Cisco Cyber Threat Defense 21 Cisco Application Visibility and Control and NetFlow 22 Application Recognition 22 Metrics Collection and Exporting 23 Management and Reporting Systems 23 Control 23 Deployment Scenarios 24 Deployment Scenario: User Access Layer 24 Deployment Scenario: Wireless LAN 25 Deployment Scenario: Internet Edge 26 Deployment Scenario: Data Center 28 Public, Private, and Hybrid Cloud Environments 32 Deployment Scenario: NetFlow in Site-to-Site and Remote VPNs 33 NetFlow Remote-Access VPNs 33 NetFlow Site-to-Site VPNs 34 NetFlow Collection Considerations and Best Practices 35 Determining the Flows per Second and Scalability 36 Summary 37 Chapter 2 Cisco NetFlow Versions and Features 39 NetFlow Versions and Respective Features 39 NetFlow v1 Flow Header Format and Flow Record Format 40 NetFlow v5 Flow Header Format and Flow Record Format 41 NetFlow v7 Flow Header Format and Flow Record Format 42 NetFlow Version 9 43 NetFlow and IPFIX Comparison 57 Summary 57 Chapter 3 Cisco Flexible NetFlow 59 Introduction to Cisco’s Flexible NetFlow 59 Simultaneous Application Tracking 60 Flexible NetFlow Records 61 Flexible NetFlow Key Fields 61 Flexible NetFlow Non-Key Fields 63 NetFlow Predefined Records 65 User-Defined Records 65 Flow Monitors 65 Flow Exporters 65 Flow Samplers 66 Flexible NetFlow Configuration 66 Configure a Flow Record 67 Configuring a Flow Monitor for IPv4 or IPv6 69 Configuring a Flow Exporter for the Flow Monitor 71 Applying a Flow Monitor to an Interface 73 Flexible NetFlow IPFIX Export Format 74 Summary 74 Chapter 4 NetFlow Commercial and Open Source Monitoring and Analysis Software Packages 75 Commercial NetFlow Monitoring and Analysis Software Packages 75 Lancope’s StealthWatch Solution 76 Plixer’s Scrutinizer 79 Open Source NetFlow Monitoring and Analysis Software Packages 80 NFdump 81 NfSen 86 SiLK 86 SiLK Configuration Files 87 Filtering, Displaying, and Sorting NetFlow Records with SiLK 87 SiLK’s Python Extension 88 Counting, Grouping, and Mating NetFlow Records with Silk 88 SiLK IPset, Bag, and Prefix Map Manipulation Tools 88 IP and Port Labeling Files 89 SiLK Runtime Plug-Ins 89 SiLK Utilities for Packet Capture and IPFIX Processing 90 Utilities to Detect Network Scans 90 SiLK Flow File Utilities 90 Additional SiLK Utilities 91 Elasticsearch, Logstash, and Kibana Stack 92 Elasticsearch 92 Logstash 92 Kibana 93 Elasticsearch Marvel and Shield 94 ELK Deployment Topology 94 Installing ELK 95 Installing Elasticsearch 96 Install Kibana 105 Installing Nginx 106 Install Logstash 107 Summary 109 Chapter 5 Big Data Analytics and NetFlow 111 Introduction to Big Data Analytics for Cyber Security 111 What Is Big Data? 111 Unstructured Versus Structured Data 112 Extracting Value from Big Data 113 NetFlow and Other Telemetry Sources for Big Data Analytics for Cyber Security 114 OpenSOC 115 Hadoop 116 HDFS 117 Flume 119 Kafka 120 Storm 121 Hive 122 Elasticsearch 123 HBase 124 Third-Party Analytic Tools 125 Other Big Data Projects in the Industry 126 Understanding Big Data Scalability: Big Data Analytics in the Internet of Everything 127 Summary 128 Chapter 6 Cisco Cyber Threat Defense and NetFlow 129 Overview of the Cisco Cyber Threat Defense Solution 129 The Attack Continuum 130 Cisco CTD Solution Components 131 NetFlow Platform Support 133 Traditional NetFlow Support in Cisco IOS Software 133 NetFlow Support in Cisco IOS-XR Software 135 Flexible NetFlow Support 135 NetFlow Support in Cisco ASA 140 Deploying the Lancope StealthWatch System 140 Deploying StealthWatch FlowCollectors 142 StealthWatch FlowReplicators 146 StealthWatch Management Console 146 Deploying NetFlow Secure Event Logging in the Cisco ASA 148 Deploying NSEL in Cisco ASA Configured for Clustering 151 Unit Roles and Functions in Clustering 152 Clustering NSEL Operations 152 Configuring NSEL in the Cisco ASA 153 Configuring NSEL in the Cisco ASA Using ASDM 153 Configuring NSEL in the Cisco ASA Using the CLI 155 NSEL and Syslog 156 Defining the NSEL Export Policy 157 Monitoring NSEL 159 Configuring NetFlow in the Cisco Nexus 1000V 160 Defining a Flow Record 161 Defining the Flow Exporter 162 Defining a Flow Monitor 163 Applying the Flow Monitor to an Interface 164 Configuring NetFlow in the Cisco Nexus 7000 Series 164 Configuring the Cisco NetFlow Generation Appliance 166 Initializing the Cisco NGA 166 Configuring NetFlow in the Cisco NGA via the GUI 168 Configuring NetFlow in the Cisco NGA via the CLI 169 Additional Cisco CTD Solution Components 171 Cisco ASA 5500-X Series Next-Generation Firewalls and the Cisco ASA with FirePOWER Services 171 Next-Generation Intrusion Prevention Systems 172 FireSIGHT Management Center 173 AMP for Endpoints 173 AMP for Networks 176 AMP Threat Grid 176 Email Security 177 Email Security Appliance 177 Cloud Email Security 179 Cisco Hybrid Email Security 179 Web Security 180 Web Security Appliance 180 Cisco Content Security Management Appliance 184 Cisco Cloud Web Security 185 Cisco Identity Services Engine 186 Summary 187 Chapter 7 Troubleshooting NetFlow 189 Troubleshooting Utilities and Debug Commands 189 Troubleshooting NetFlow in Cisco IOS and Cisco IOS XE Devices 194 Cisco IOS Router Flexible NetFlow Configuration 195 Troubleshooting Communication Problems with the NetFlow Collector 201 Additional Useful Troubleshooting Debug and Show Commands 204 Verifying a Flow Monitor Configuration 204 Displaying Flow Exporter Templates and Export IDs 207 Debugging Flow Records 212 Preventing Export Storms with Flexible NetFlow 213 Troubleshooting NetFlow in Cisco NX-OS Software 214 Troubleshooting NetFlow in Cisco IOS-XR Software 217 Flow Exporter Statistics and Diagnostics 219 Flow Monitor Statistics and Diagnostics 222 Displaying NetFlow Producer Statistics in Cisco IOS-XR 226 Additional Useful Cisco IOS-XR Show Commands 228 Troubleshooting NetFlow in the Cisco ASA 228 Troubleshooting NetFlow in the Cisco NetFlow Generation Appliance 235 Gathering Information About Configured NGA Managed Devices 235 Gathering Information About the Flow Collector 236 Gathering Information About the Flow Exporter 237 Gathering Information About Flow Records 237 Gathering Information About the Flow Monitor 238 Show Tech-Support 239 Additional Useful NGA show Commands 245 Summary 246 Chapter 8 Case Studies 247 Using NetFlow for Anomaly Detection and Identifying DoS Attacks 247 Direct DDoS Attacks 248 Reflected DDoS Attacks 248 Amplification Attacks 249 Identifying DDoS Attacks Using NetFlow 250 Using NetFlow in Enterprise Networks to Detect DDoS Attacks 250 Using NetFlow in Service Provider Networks to Detect DDoS Attacks 253 Using NetFlow for Incident Response and Forensics 254 Credit Card Theft 254 Theft of Intellectual Property 259 Using NetFlow for Monitoring Guest Users and Contractors 262 Using NetFlow for Capacity Planning 267 Using NetFlow to Monitor Cloud Usage 269 Summary 271 TOC, 9781587144387, 8/25/2015

About the Author :
Omar Santos is a Principal Engineer in the Cisco Product Security Incident Response Team (PSIRT) part of Cisco’s Security Research and Operations. He mentors and leads engineers and incident managers during the investigation and resolution of security vulnerabilities in all Cisco products. Omar has been working with information technology and cyber security since the mid-1990s. Omar has designed, implemented, and supported numerous secure networks for Fortune 100 and 500 companies and for the U.S. government. Prior to his current role, he was a Technical Leader within the World Wide Security Practice and the Cisco Technical Assistance Center (TAC), where he taught, led, and mentored many engineers within both organizations.   Omar is an active member of the security community, where he leads several industrywide initiatives and standard bodies. His active role helps businesses, academic institutions, state and local law enforcement agencies, and other participants that are dedicated to increasing the security of the critical infrastructure.   Omar is the author of several books and numerous whitepapers, articles, and security configuration guidelines and best practices. He has also delivered numerous technical presentations at many conferences and to Cisco customers and partners, in addition to many C-level executive presentations to many organizations. Omar is the author of the following Cisco Press books:   CCNA Security 210-260 Official Cert Guide, ISBN-13: 9781587205668 Deploying Next-Generation Firewalls Live Lessons, ISBN-13: 9781587205705 Cisco’s Advanced Malware Protection (AMP), ISBN-13: 9781587144462 Cisco ASA Next-Generation Firewall, IPS, and VPN Services (3rd Edition), ISBN-10: 1587143070 Cisco ASA: All-in-One Firewall, IPS, Anti-X, and VPN Adaptive Security Appliance (2nd Edition), ISBN-10: 1587058197 Cisco ASA: All-in-One Firewall, IPS, and VPN Adaptive Security Appliance, ISBN-10: 1587052091 Cisco Network Admission Control, Volume: Deployment and Management, ISBN-10: 1587052253 End-to-End Network Security: Defense-in-Depth, ISBN-10: 1587053322