VoIP Performance Management and Optimization

VoIP Performance Management and Optimization A KPI-based approach to managing and optimizing VoIP networks   IP Communications   Adeel Ahmed, CCIE® No. 4574 Habib Madani Talal Siddiqui, CCIE No. 4280   VoIP Performance Management and Optimization is the first comprehensive, expert guide to managing, monitoring, troubleshooting, and optimizing large VoIP networks. Three leading Cisco VoIP experts bring together state-of-the-art techniques for ensuring that customer service level agreements (SLA) are consistently met or exceeded.   The authors begin by reviewing how VoIP is deployed in enterprise and service provider networks and the performance tradeoffs and challenges associated with each leading VoIP deployment model. Next, they present a comprehensive approach to diagnosing problems in VoIP networks using key performance indicators (KPI) and proactively addressing issues before they impact service.   In this book, you will find a proven tools-based strategy for gauging VoIP network health and maximizing performance and voice quality. You also will learn how to perform trend analysis and use the results for capacity planning and traffic engineering—thereby optimizing your networks for both the short- and long-term.   The authors all work in the Cisco Advanced Services Group.   Deploy, manage, monitor, and scale multivendor VoIP networks more effectively Integrate performance data from multiple VoIP network segments and service flows to effectively manage SLAs Use performance counters, call detail records, and call agent trace logs to gauge network health in real time Utilize dashboards to analyze and correlate VoIP metrics, analyze trends, and plan capacity Implement a layered approach to quickly isolate and troubleshoot both localized and systemic problems in VoIP networks Optimize performance in networks where the service provider owns the “last mile” connection Improve performance when VoIP is deployed over publicly shared infrastructure Manage performance in enterprise networks using both centralized and distributed call processing Plan media deployment for the best possible network performance Monitor trends, establish baselines, optimize existing resources, and identify emerging problems Understand and address common voice quality issues   This IP communications book is part of the Cisco Press® Networking Technology Series. IP communications titles from Cisco Press help networking professionals understand voice and IP telephony technologies, plan and design converged networks, and implement network solutions for increased productivity.   Category: Networking: Unified Communications Covers: Voice over IP Network Management  

Table of Contents:
Foreword Introduction Part I VoIP Networks Today Chapter 1 Voice over IP (VoIP) and Network Management 1 VoIP Technology 2 VoIP Overview 3     Media Transport Protocol for VoIP—RTP 5     VoIP Signaling Protocols 8 Common Network Problems in VoIP Networks 9     Delay/Latency 9         Propagation Delay 10         Processing Delay 10         Serialization/Queuing Delay 11     Jitter 11     Packet Loss 12     Voice Activity Detection (VAD) 13     Other Issues 13 Common Voice Quality Problems in IP Networks 14 Strategic Importance of VoIP and Management 18 Network Management Methodologies 20     Telecommunications Management Network 20     FCAPS Model 21         Fault Management 21         Configuration Management 21         Accounting Management 22         Performance Management 22         Security Management 22     Information Technology Infrastructure Library (ITIL) 23         Service Strategy 23         Service Design 24         Service Transition 25         Service Operation 26         Continual Service Improvement 27     Enhanced Telecom Operations Map (eTOM) 27 Comprehensive Network Management Methodology 28     Focusing on Performance Metrics 30 Summary 32 Reference 32 Chapter 2 A Metrics-Based Approach for Managing the VoIP Network 33 VoIP Networks Require a Layered Management Approach 34     Tracking Systemic Performance Issues 37     Localized Performance Issues 39     Subjective Performance Issues 39     Downtime and Impact 40 Proactive Monitoring Concept 41 KPIs 43     VoIP-Signaling KPIs 44     VoIP Media KPIs 45 VoIP Network Segments and VoIP Service Flows 46     Voicemail Segment 46     Announcement Segment 47     Voice Termination Point Segment 47         Voice ONNET Call Leg Segment 47         Voice OFFNET or PSTN-Bound Segment 47     PSTN Bearer Traffic Segment 48 Service-Level Agreement (SLA) Management 48     SBC Trunk Uptime 50     PSTN/IMT Trunk Uptime 50     Signaling SS7 Link Uptime 50     Vendor Accountability 51     Tools Utilized 51 Summary 52 Reference 52 Part II VoIP Deployment Models Chapter 3 VoIP Deployment Models in Service Provider Networks 53 Service Provider Voice Implementation Models 54 Residential Applications: Voice over Broadband 55 Small/Medium Business Applications (Voice over T1/E1/PRI/CAS) 58 IP Trunks 59 Session Border Controller (SBC) Models 62 Key Components Used in SBC Models 63     PSTN Offload 64     Network Hiding 65 Voice Security in Service Provider Networks 65     Securing VoIP Network Elements 65     Securing Call Signaling and the Media 66 Common Issues and Problems When Deploying IP-Based Telephony Services 66     Convergence-Related Issues 66     Issues in Media Affecting Quality 67     Issues in Signaling Affecting the Services and Features 67     IP Routing–Related Issues 67     High Availability and Convergence for Business Continuity 68 Summary 68 References 68 Chapter 4 Internet Telephony 69 Internet Telephony Deployment Model 70     Internet Telephony Network Elements 72     Internet Telephony Applications 73         PC-Based Software Voice Applications 73         ATA-Based Voice Applications 74     Traffic Profiling 74     Potential Bottlenecks 75 Wholesale VoIP Solution 75     Key Network Elements 77         Media Gateway Controller (MGC) 77         IP Transfer Point (ITP) 78         Route Server 78         Gatekeepers 79         Application Servers 79         Element Management Systems (EMS) 79     Wholesale Voice Applications 80         Prepaid and Postpaid Calling Solutions 80         Network Transit and Trunking Applications 82         Managed Services for Enterprises 83         Applications and Benefits for Service Providers 83 Common Issues and Problems with Internet Telephony 83     Last-Mile Connection Bandwidth 84     End Device/Application-Related Issues 85     No Customer Service-Level Agreements (SLA) 86     Issues with Emergency Calls (E911) 86     Security Issues 87 Summary 88 References 88 Chapter 5 VoIP Deployment Models in Enterprise Networks 89 Unified Communication Solution Components in Enterprise Networks 90     Unified Communications Manager/CallManager 90     Voice Gateways 91     Gatekeepers 92     Session Border Controller 93     Messaging Application 94     Rich Media Applications 95         Cisco Unified MeetingPlace and WebEx 95         Cisco Unified Presence 95         Cisco Emergency Responder 96         Cisco Unified Contact Center 97         Cisco Unified Application Environment 97 Common Enterprise Deployment Models 97     Centralized Call Processing 98     Distributed Call Processing 100     Hybrid Models 102 Common Issues and Problems 104     Convergence-Related Issues 104     Issues Affecting Media Quality 105     Voice-Signaling Protocol Impairments 106     Voice Security in Enterprise Converged Networks 106 Summary 107 References 107 Part III Performance and Optimization of VoIP Networks Chapter 6 Managing VoIP Networks 109 Requirements for Enabling Voice in IP Networks 109     Network Readiness Assessment 110         Network Design 110         Network Infrastructure Services 112         Network Links 113         Hardware and Software Considerations 114         Power and Environment 115     Auditing for VoIP Network Readiness 116         Analyzing Configurations, Versions, and Topology 117         Synthetic Traffic Tests 118     Managing Network Capacity Requirements 118         Voice Traffic Engineering Theory 119         Example of Estimating Capacity Requirements 119         Monitoring Network Resources 122     An Audit for Gauging the Current VoIP Network Utilization 122         Device Utilization 123         Link Utilization 124     Measurements for Network Transmission Loss Plan 124 Effectively Monitoring the Network 127     Discovery—Complete Picture 128         Seed Devices for Network Discovery 129         Cisco Discovery Protocol (CDP) Discovery 129         Routing Table Discovery 130         ARP Discovery 130         Routing Protocol—OSPF Discovery 130         Ping Sweep Discovery 130         Seed Files 131     Voice Quality Metrics 131         MOS or K-factor 132         PSQM 132         PESQ 133     Approaches to Measure Jitter, Latency, and Packet Loss in the IP Network 133         Using Call Detail Records for Voice Quality Metrics 133         Using IP-SLA and RTTMON for Voice Quality Metrics 134         Using Cisco NetFlow for Measuring Voice Quality Metrics 135         Round-Trip Delay Measurement 136         Voice Jitter/Frame Slip Measurements 137         Measurement of Effective Bandwidth 137         Voice Band Gain Measurement 137         Silence Noise level Measurement 138         Voice Clipping 138         Echo Measurements 138     Voice-Signaling Protocol Impairments in IP Networks 139 How to Effectively Poll the Network 140     Polling Strategy 141     Key Alarms and Events Monitoring 143     SNMP Configuration and Setting 143         Basic Configuration 144         SNMP Trap Settings 144         Traps Use Case BTS 10200 Cisco Softswitch 144     Standard Polling Intervals and Traps 145         Scenario 1: Phones Unregistering from Unified CM and Reregistering to SRST Router Because of WAN Link Outage 145         Scenario 2: Phones Unregistering from the Unified CM and Reregistering to the SRST Router Because of WAN Congestion 146         Using eXtensible Markup Language (XML) for Polling and Extraction of Key Information 147         XML Overview 148         XML APIs 149 Using the Syslog/Trace Logs for Deep Analysis 150 Alarm and Event Audit and Correlation 151 Effectively Monitoring the PSTN Bearer Traffic 153 QoS in VoIP Networks 155     Defining a QoS Methodology 155         Differentiated Services (Diff Serv) for Applying QoS 155         Using Bandwidth/Resource Reservation and Call Admission Control (CAC) for Providing QoS 157     Managing QoS 157         PacketCable Use Case 159 Trouble Ticketing (TT) Systems 162     Identifying and Streamlining the Categories of Trouble Tickets 162     Correlating the TT to the Service Uptime 162 Summary 163 References 164 Chapter 7 Performance Analysis and Fault Isolation 167 Proactive Monitoring Through Performance Counters 168     Classification of Performance Counters 168         Network Device KPIs 168         Functional- or Services-Based Grouping of KPIs 169         Fault Isolation–Based Grouping of KPIs 173         Protocol-Based Grouping of KPIs 174         SLA Tracking Through KPIs 175         Equipment-Based Grouping of KPIs 177     Collection 177     Alarm Processing 178     Correlation 179         Simple Correlation 180         Advanced Correlation 180         Complex Correlations 181     Recommendations for VoIP-Centric Network Management Framework 182 Performance Analysis from a Transit Network Perspective 183     Signaling Protocol Transport Optimization 184         Enterprise Networks 184     Cisco IOS QoS Recommended SNMP Polling Guidelines 187         Case Study of Link Congestions 187         SP Networks 194     Performance Data in an Enterprise VoIP Environment 197         CPU Status 198         Physical Memory 198         Hard Disk Status 199         High Utilization of Disk Space 199         Virtual Memory 199         Number of Active Phones 200         Gateway Registration (MGCP) 200         Gatekeeper Registration (H.323 RAS) 200         Calls in Progress 201         Calls Active 201         Calls Attempted 202         Calls Completed 202         PRI Channels Active 203         Conferencing/Transcoding DSP’s Depletion 203         Available Bandwidth of a Location (CAC) 204     Recommendations for Categorizing Performance Measurements 204 Enterprise Case Study—Analyzing Network Performance 206     CPU Rate and Critical Processes 206     Rate of Active Calls 207         Tracking Trunk Utilization for PSTN Access 208         Trend Analysis Best Practices 211 Performance Analysis from Call Agent Perspective 211     Performance Analysis for VoIP Call Traffic 211     Performance Analysis for a PSTN Network (PSTN Trunk and SS7 Signaling) 215     Performance Analysis for an SIP Network 217     Performance Tracking for a Session Border Controller (SBC) 218     Performance Information Through the Call Detail Records (CDR) 219     Performance Enhancement Schemes and Their Effect on VoIP Network Monitoring 220         Effect of DNS Caching 220         Server Load Balancing 220         Firewall 220         Optimizing the SBC 221 Performance Analysis from a DOCSIS Network 221     VoIP Endpoints 222     DOCSIS/DQoS 224     CPU Impact/Link Utilization 226 Trace Log Monitoring on Softswitch and Network Devices 229     Analyzing and Correlating Syslog Messages 230     Log Files Management 231         Security 231         Storage Location (Local Versus Remote) and Archiving Logs 233 Tools and Scripts 234     Tools for Monitoring an Enterprise VoIP Network 234         Cisco Unified Operations Manager (CUOM) 234         Cisco Unified Service Manager 236         Cisco Unified Service Statistics Manager 237     Tools for Monitoring Service Provider VoIP Networks 239         IXIA’s IxRave Solution 239         IxRave Case Study—Voice Assurance for Cable Networks 240     Tools for Monitoring DOCSIS Networks—VoIP Dashboard 242     Tools for Monitoring VoIP Network Health Through Protocols 244     Tools for Analyzing Call Detail Records 246         SP CDR Report Scenario 246         Customizing CDR Reporting for Effective Monitoring 247     Dashboard Views for the VoIP Network 247 Software Maintenance 248     Software Release Management 249     Software Lifecycle Management 249     Software Resiliency 251     Periodic Auditing of a VoIP Network 251 Summary 254 References 254 Chapter 8 Trend Analysis and Optimization 257 Trend Analysis Through Key Metrics 258     Dashboard as a Profiling Tool 259         Network Utilization and Efficiency 260         Safeguarding Network Resources from Potential Security Threats 261     Dashboard for Trunk Resources Utilization 265         Feedback for Change Control 266     Profiling in an SP VoIP Network 271     Profiling in an Enterprise VoIP Network 277         Balancing the Device Load on CUCM Cluster Nodes 278         Maximizing Trunk Capacity and Avoiding Call Blocking 280 Call Detail Record–Based Trend Analysis 283     Benchmarking 283         Verifying VoIP Network Resources Capacity 284     SLA Conformance 286         Monitoring for Service Availability 286         Normal Versus Abnormal Termination Profiling: Categorizing and Correlating the Call Termination Code 288         Monitoring for Service Quality 289     Verifying Toll Savings (On-net Versus Off-net Profiling) 289     Detecting Toll Frauds 291 Resource Optimization and Capacity Planning 291 Network Resource Utilization and Optimization 291 Capacity Planning and Upgrade Strategies 296 Managing Subscriber Growth Impact by Using Trend Analysis 298         UC Manager Cluster Capacity 298         Network Bandwidth and Transcoding DSPs 299         Considerations for Adding Trunk Capacity 302 Summary 302 References 302 Part IV Appendixes Appendix A Scripts and Tools for Monitoring and Troubleshooting VoIP Networks 305 Appendix B Detailed Call Flows 331 Appendix C VoIP Dashboard 367 Appendix D Debugs, Traces, and Logs 373       9781587055287    TOC    7/1/2010  

About the Author :
Adeel Ahmed, CCIE No. 4574, is a senior manager in the Cisco Advanced Services group. He has been with Cisco for more than 11 years. His areas of expertise include Access/Dial, Broadband Cable, SP Voice, and IPv6. He has worked with major cable MSOs in North America, EMEA, and ASIAPAC in designing and troubleshooting cable networks. He has written several white papers and design guides used by customers, sales teams, and Cisco engineers in deploying multiservices over cable networks. He has represented Cisco at industry technical forums such as IETF, APRICOT, SCTE, CableLabs, NCTA, NAv6TF, Networkers, and Global IPv6 Summit. He is the coauthor of Deploying IPv6 in Broadband Access Networks. Adeel holds a bachelor’s and a master’s degree in electrical engineering.   Habib Madani works as a network consulting engineer in the Cisco Advanced Services group. He has been with Cisco for the past five years; before that, he worked for more than ten years as a lead engineer at Yahoo!, CenterPoint Broadband Technology, and Motorola. His background is SP VoIP, Data Center, Network Management, and OSS solutions. He has worked with major enterprise customers and MSOs in the United States and abroad across these segments. He has represented Cisco at industry technical forums such APRICOT and Networkers. Habib holds a bachelor’s degree in electrical engineering from the University of Arizona and a master’s in electrical engineering and computer science from the University of Illinois.   Talal Siddiqui,CCIE No. 4280, is a senior manager in the Cisco Advanced Services group. He has been with Cisco for 13 years. He has been involved with beachhead customers who were deploying these solutions for the past 12 years. As a consulting engineer, he has worked with MSOs, CLECs, and ILECs who have deployed Cisco SS7 Interconnect Solutions through planning, designing, deploying, network expansion, and operational troubleshooting phases. As a technical leader in Unified Communications Practice, he worked with Cisco partners in planning, designing, and deploying IP telephony solutions for large enterprise customers. Currently he is working on developing service offerings for Communication/Collaboration Enabled Business Process (CEBP) solutions and service delivery tools. Talal holds BSEE and MBA degrees and is a CCIE.