Web-Based Management of IP Networks and Systems

Building on the author’s extensive experience in industry and government agencies, this book proposes a general framework for integrating the management of networked systems in the IP world. The Web-based Integrated Management Architecture (WIMA) leverages XML’s self-description capability to integrate SNMP data and CIM objects in a seamless manner. The first part of the book begins with a detailed analysis of SNMP; it highlights its strengths and weaknesses and justifies the need for object-oriented information models such as CIM. Next, the author summarizes the different paradigms considered in management research and industry in the past decade, compares the suitability of different middleware solutions, and concludes that with current technologies, Web-based management makes a lot of technical sense and is not just trendy. The second part goes into software engineering detail and discusses tradeoffs for organizing manager-to-agent communication. In WIMA, agents publish the monitoring data and notifications they can send, and management applications (managers) subscribe to them in a semi- or fully automated way. The same publish-subscribe paradigm is used for manager-to-manager communication, when managers are organized hierarchically to manage a large domain or different domains. We explain why this solution decreases network overhead and improves scalability. Security issues are also investigated. The third part of the book presents the high-level design of JAMAP, a research prototype that implements the main components of the WIMA architecture. Its code is freely available and released under the GPL license. This prototype is currently being ported to Web Services. Finally, the author compares WIMA with SNMP, WBEM/CIM and JMX and concludes that WBEM could evolve into WIMA in the future. This book is primarily aimed at software architects and engineers who design and write management applications. It should also be of interest to administrators and people who are versed into management architectures.

Table of Contents:
List of Figures xv List of Tables xvii Foreword xix Preface xxi 1 Introduction 1 1.1 Background 1 1.2 Context of This Work 4 1.3 Caveat 6 1.4 Organization 7 2 Terminology 9 2.1 IP World vs. Telecommunications World 10 2.2 Different Types of Enterprise Management 10 2.3 Manager, Agent, Management Application, Management Platform, and Management System 14 2.4 Manager vs. Administrator 15 2.5 Agent, Mobile Agent, and Intelligent Agent 15 2.6 Proxy vs. Gateway 15 2.7 Delegation 16 2.8 Paradigm vs. Technology 17 2.9 Architecture vs. Framework 18 2.10 Firewalls and Trust Domains 18 2.11 Regular Management vs. Ad Hoc Management 20 2.12 Ontologies, Schemas, and Models 22 3 Problem Statement 25 3.1 Characteristics of SNMP-Based Management 25 3.2 Strengths of SNMP-Based Management 33 3.3 Problems with SNMP-Based Management 33 3.3.1 Scalability and efficiency issues 33 3.3.2 Missing features 46 3.3.3 Non-technical problems in SNMP-based management. 49 3.4 We Need a New Solution to a New Problem 53 3.5 Summary 56 4 Overview of the Solution Space 57 4.1 Simple Taxonomy of Network and Systems Management Paradigms 58 4.1.1 Centralized paradigm 60 4.1.2 Weakly distributed hierarchical paradigms 61 4.1.3 Strongly distributed hierarchical paradigms 63 4.1.4 Strongly distributed cooperative paradigms 69 4.1.5 Synthetic diagram 71 4.2 Enhanced Taxonomy of Network and Systems Management Paradigms 72 4.2.1 A stroll through organization theory 72 4.2.2 Other criteria for our enhanced taxonomy 78 4.2.3 Synthetic diagram 85 4.3 Summary 86 5 Analysis of the Solution Space 87 5.1 No Win-Win Solution 87 5.2 Do Not Focus on Technology Immediately 89 5.3 Reality Check: Support and Technical Maturity 90 5.4 The My-Middleware-Is-Better-Than-Yours Syndrome 92 5.5 Mobile Code and Security 94 5.6 Distribution 95 5.7 Web-Based Management 95 5.8 Summary 96 6 State of the Art in Web-Based Management 97 6.1 Browser-Based Usage Monitoring 98 6.2 Browser-Based Metamanagement 99 6.2.1 Online problem reporting 100 6.2.2 Online management procedures, online documentation 100 6.2.3 Online troubleshooting assistance 101 6.3 Browser-Based Management 101 6.3.1 Troubleshooting scripts executed via a Web browser 102 6.3.2 Configuration management 102 6.3.3 Java applet with an SNMP stack 103 6.4 Three-Tier Management 103 6.4.1 Deri et al.: SNMP-to-URL mapping 104 6.4.2 Kasteleijn: HTTP and SNMP 104 6.4.3 Integration of a Web browser in the SNMP-based management platform 105 6.4.4 Deri: Java RMI and SNMP 105 6.5 HTTP-Based Management 105 6.5.1 CLI wrappings and mappings 107 6.5.2 Embedded HTML pages and CGI programs 115 6.5.3 Embedded management applications 116 6.5.4 Low-footprint embedded HTTP servers 116 6.6 XML-Based Management 117 6.6.1 Web-Based Enterprise Management (WBEM) 117 6.6.2 John et al.: XNAMI 122 6.7 Distributed Java-Based Management 123 6.7.1 Java Management Application Programming Interface (JMAPI) 123 6.7.2 Java Management eXtensions (JMX) 124 6.7.3 Federated Management Architecture (FMA) 124 6.7.4 Anerousis’s Marvel 124 6.8 Commercial Products 125 6.9 Summary 126 7 A New Management Architecture: WIMA 127 7.1 Main Architectural Decisions (Analysis Phase) 128 7.1.1 One management architecture, four models 128 7.1.2 No need to define yet another information model 131 7.1.3 Dissociation of the communication and information models 131 7.1.4 A new organizational model: WIMA-OM 132 7.1.5 A new communication model: WIMA-CM 137 7.2 Main Design Decisions (Design Phase) 138 7.2.1 Web technologies 138 7.2.2 Three-tier architecture 139 7.2.3 Management server: COTS components and object-oriented frameworks 140 7.2.4 Management-data transfers across firewalls 142 7.2.5 Data repository independent of the management platform 142 7.2.6 Bulk transfers of regular management data 144 7.2.7 Dealing with legacy systems 144 7.2.8 Richer semantics 144 7.2.9 Easy to deploy 145 7.3 Regular Management and Notification Delivery: The Push Model 145 7.3.1 Publication phase 146 7.3.2 Discovery phase 146 7.3.3 Subscription phase 152 7.3.4 Distribution phase 158 7.3.5 Distribution 163 7.3.6 Migration path: four-tier architecture 168 7.4 Ad Hoc Management: The Pull Model 168 7.4.1 Two-tier architecture (no firewall) 169 7.4.2 Three-tier architecture (with firewall) 171 7.4.3 Distribution 172 7.4.4 Migration path 173 7.5 Summary 173 8 A New Communication Model: WIMA-CM 175 8.1 Two Communication Problems 175 8.1.1 Simplified terminology 176 8.1.2 Communication problem for pushed data 176 8.1.3 Communication problem for pulled data 177 8.2 Main Design Decisions 177 8.2.1 Dissociation of the communication and information models 177 8.2.2 Reliable transport protocol: TCP 178 8.2.3 Persistent TCP connections 181 8.2.4 Firewalls: persistent connections must be created by the manager 191 8.2.5 Reversed client and server roles 193 8.3 The Sockets API as a Manager-Agent Communication API 194 8.4 HTTP-Based Communication for Pushed Data: WIMA-CM-push 195 8.4.1 MIME multipart.196 8.4.2 Some notifications are more equal than others 197 8.4.3 Specifying the information model in the MIME header 200 8.4.4 Optional compression of management data 202 8.4.5 Example of HTTP and MIME-part headers 203 8.4.6 Simplifications in case we do not have a firewall 204 8.5 Timeouts and Reconnections 205 8.5.1 Timeouts by the operating systems 205 8.5.2 Timeouts by the applications 215 8.5.3 Synthesis 219 8.6 HTTP-Based Communication for Pulled Data: WIMA-CM-pull 221 8.7 Summary 222 9 XML in Integrated Management 223 9.1 Why Use XML in NSM? 223 9.1.1 Overview of XML 224 9.1.2 Who uses XML? 224 9.1.3 Advantages of using XML in general 225 9.1.4 Advantages of using XML in NSM 226 9.1.5 Advantages of using XML in integrated management 227 9.1.6 Disadvantages of using XML in general 228 9.2 XML for Representing Management Data: Model- and Metamodel-Level Mappings 228 9.2.1 Model-level mapping 229 9.2.2 Metamodel-level mapping 232 9.2.3 Comparison between model- and metamodel-level mappings 235 9.3 XML for Dealing with Multiple Information Models 235 9.4 XML for High-Level Semantics 238 9.4.1 Transfer of an entire SNMP MIB table 238 9.4.2 Suppression of “holes” in sparse SNMP tables 239 9.4.3 Time series of a MIB variable 239 9.4.4 Distributed object-oriented programming with XML 241 9.5 XML for Integrated Management: a Unified Communication Model 242 9.6 Summary 244 10 A WIMA-Based Research Prototype: JAMAP 245 10.1 Overview of JAMAP 246 10.1.1 Key design decisions at a glance 246 10.1.2 More on the design of JAMAP 247 10.1.3 Advanced Java technologies in JAMAP 248 10.1.4 Overview of the communication aspects 249 10.1.5 Distribution phase for monitoring and data collection 251 10.1.6 Distribution phase for notification delivery 252 10.2 Management Station 253 10.2.1 SNMP MIB data subscription applet 253 10.2.2 SNMP notification subscription applet 254 10.2.3 Rule edition applet and rule mapping applet 254 10.2.4 Event log applet 255 10.3 Management Server 255 10.3.1 Pushed-data collector servlet 255 10.3.2 Notification collector servlet 257 10.3.3 Event manager servlet 257 10.4 Agent 258 10.4.1 Subscription servlet 258 10.4.2 Push dispatcher servlet 258 10.4.3 Notification dispatcher servlet 259 10.5 Reusability 259 10.6 Summary 260 11 How Does Our Solution Compare with Others? 261 11.1 Comparison with SNMP-Based Management 262 11.1.1 Almost all of the problems in SNMP have been solved 262 11.1.2 Complementarities between WIMA and SNMP 264 11.2 Comparison with WBEM 264 11.2.1 Similarities: HTTP and XML 264 11.2.2 Problems with WBEM 265 11.2.3 Complementarities between WIMA and WBEM 267 11.3 Comparison with JMX 267 11.4 Known Problems with WIMA 268 11.5 Summary 270 12 Conclusion 271 12.1 Summary.271 12.2 Directions for Future Work 274 Acronyms 277 References 283 Appendix A: The Interfaces Group in SNMP MIB-II 299 Appendix B: Metamodel-Level XML Mapping of the Interfaces Group in SNMP MIB-II 309 Appendix C: Metamodel-Level XML Mapping of a Simple CIM Class 319 Appendix D: Remote Method Invocation of a CIM Object 325 Index 327