Software Testing: Principles and Practices

Software Testing is specially developed to serve as a text book for the undergraduate and postgraduate students of Computer Science Engineering and Information Technology. The book focusses on software testing as not just being the phase of software development life cycle but a complete process to fulfill the demand of quality software. Written in a very lucid style with crisp and to-the-point descriptions, the book covers chapters on the various software testing methodologies, test management, software metrics, software quality assurance, test automation, object-oriented testing and debugging. It also describes all the methods for test case design which is the prime issue for software testing. The book is interactive and includes a large number of test cases, examples, MCQs and unsolved problems for practice.

Table of Contents:
PART I TESTING METHODOLOGY ; 1. Introduction to Effective Software Testing ; 1.1 Introduction ; 1.2 Evolution of Software Testing ; 1.3 Software Testing Myths ; 1.4 Goals of Software Testing ; 1.5 Psychology for Software Testing ; 1.6 Software Testing Definitions ; 1.7 Model for Software Testing ; 1.8 Effective Software Testing vs Exhaustive Software Testing ; 1.9 Effective Testing is hard ; 1.10 Software Testing as a Process ; 1.11 Schools of Software Testing ; 1.12 Software Failure Case Studies ; Summary ; 2. Software Testing Terminology and Methodology ; 2.1 Software Testing Terminology ; 2.1.1 Definitions ; 2.1.2 Life Cycle of a Bug ; 2.1.3 States of a Bug ; 2.1.4 Why do bugs occur? ; 2.1.5 Bug affects economics of software testing ; 2.1.6 Bug Classification based on Criticality ; 2.1.7 Bug Classification based on SDLC ; 2.1.8 Testing Principles ; 2.2 Software Testing Life Cycle (STLC) ; 2.3 Software Testing Methodology ; 2.3.1 Software Testing Strategy ; 2.3.2 Test Strategy Matrix ; 2.3.3 Development of Test Strategy ; 2.3.4 Testing Life Cycle Model ; 2.3.4.1 V Testing life cycle model ; 2.3.5 Validation Activities ; 2.3.6 Testing Tactics ; 2.3.7 Considerations in developing testing methodologies ; Summary ; 3. Verification and Validation ; 3.1 Verification and Validation Activities ; 3.2 Verification ; 3.2.1 Verification Activities ; 3.3 Verification of Requirements ; 3.3.1 Verification of Objectives ; 3.3.2 How to verify Requirements and Objectives ; 3.4 Verification of High level Design ; 3.4.1 How to verify High level Design? ; 3.4.4.1 Verification of Data Design ; 3.4.1.2 Verification of Architectural Design ; 3.4.1.3 Verification of User Interface Design ; 3.5 Verification of Low level Design ; 3.5.1 How to verify Low level Design? ; 3.6 How to verify Code? ; 3.6.1 Unit Verification ; 3.7 Validation ; 3.7.1 Validation Activities ; Summary ; PART II TESTING TECHNIQUES ; 4. Dynamic Testing : Black Box Testing Techniques ; 4.1 Boundary Value Analysis ; 4.1.1 Boundary value checking ; 4.1.2 Robustness Testing method ; 4.1.3 Worst Case Testing method ; 4.2 Equivalence Class Testing ; 4.2.1 Identification of Equivalence classes ; 4.2.2 Identifying the Test cases ; 4.3 State Table based Testing ; 4.3.1 Finite State Machine ; 4.3.2 State Graphs ; 4.3.3 State Tables ; 4.3.4 State table based testing ; 4.4 Decision Table based Testing ; 4.4.1 Formation of Decision Table ; 4.4.2 Test case design using decision table ; 4.4.3 Expanding the immaterial test cases in decision table ; 4.5 Cause Effect Graphing based Testing ; 4.5.1 Basic notations ; 4.6 Error Guessing ; Summary ; 5. Dynamic Testing : White Box Testing Techniques ; 5.1 Need of White box testing ; 5.2 Logic Coverage Criteria ; 5.3 Basis Path Testing ; 5.3.1 Control Flow Graph ; 5.3.2 Flow graph notations of different programming constructs ; 5.3.3 Path Testing Terminology ; 5.3.4 Cyclomatic Complexity ; 5.3.4.1 Formulae based on Cyclomatic complexity ; 5.3.4.2 Guidelines for Basis Path Testing ; 5.3.5 Applications of Path Testing ; 5.4 Graph Matrices ; 5.4.1 Graph Matrix ; 5.4.2 Connection Matrix ; 5.4.3 Use of connection matrix in finding cyclomatic complexity number ; 5.4.4 Use of graph matrix for finding the set of all paths ; 5.5 Loop Testing ; 5.6 Data Flow Testing ; 5.6.1 State of a Data Object ; 5.6.2 Data Flow Anomalies ; 5.6.3 Terminology used in Data Flow Testing ; 5.6.4 Static Data flow testing ; 5.6.4.1 Static analysis is not enough ; 5.6.5 Dynamic Data flow testing ; 5.6.6 Ordering of Data flow testing strategies ; 5.8 Mutation Testing ; 5.8.1 Primary Mutants ; 5.8.2 Secondary Mutants ; 5.8.3 Mutation Testing Process ; Summary ; 6. Static Testing ; 6.1 Inspections ; 6.1.1 Inspection Team ; 6.1.2 Inspection Process ; 6.1.3 Benefits of Inspection Process ; 6.1.4 Effectiveness of Inspection Process ; 6.1.5 Cost of Inspection Process ; 6.1.6 Variants of Inspection process ; 6.1.7 Reading Techniques ; 6.1.8 Checklists for Inspection Process ; 6.2 Walkthroughs ; 6.3 Technical Reviews ; Summary ; 7. Validation Activities ; 7.1 Unit Validation Testing ; 7.2 Integration Testing ; 7.2.1 Decomposition Based Integration ; 7.2.1.1 Types of Incremental Integration Testing ; 7.2.1.2 Comparison between Top-Down and Bottom-Up Integration Testing ; 7.2.1.3 Practical Approach for Integration Testing ; 7.2.2 Call Graph Based Integration ; 7.2.2.1 Pair-wise Integration ; 7.2.2.2 Neighborhood Integration ; 7.2.3 Path Based Integration ; 7.3 Function Testing ; 7.4 System Testing ; 7.4.1 Categories of Systems Tests ; 7.4.1.1 Recovery Testing ; 7.4.1.2 Security Testing ; 7.4.1.3 Performance Testing ; 7.4.1.4 Load Testing ; 7.4.1.5 Stress Testing ; 7.4.1.6 Usability Testing ; 7.4.1.7 Compatibility/Conversion/Configuration Testing ; 7.4.2 Guidelines for performing the system tests ; 7.5 Acceptance Testing ; 7.5.1 Alpha Testing ; 7.5.1.1 Entry to Alpha ; 7.5.1.2 Exit to Alpha ; 7.5.2 Beta Testing ; 7.5.2.1 Entry to Beta ; 7.5.2.2 Exit criteria ; Summary ; 8. Regression Testing ; 8.1 Progressive vs Regression Testing ; 8.2 Regression testing produces quality software ; 8.3 Regression Testability ; 8.4 Objectives of Regression Testing ; 8.5 When to do regression testing? ; 8.6 Regression Testing Types ; 8.7 Defining Regression Test Problem ; 8.7.1 Regression Testing is a problem? ; 8.7.2 Regression Testing Problem ; 8.8 Regression Testing Techniques ; 8.8.1 Selective Retest Techniques ; 8.8.1.1 Strategy for Test Case Selection ; 8.8.1.2 Selection Criteria Based on Code ; 8.8.1.3 Regression Test selection Techniques ; 8.8.1.4 Evaluating Regression Test Selection Techniques ; 8.8.2 Regression Test Prioritization ; 8.9 Benefits of Regression Testing ; Summary ; PART III MANAGING THE TEST PROCESS ; 9. Test Management ; 9.1 Test Organization ; 9.2 Structure of Testing Group ; 9.3 Test Planning ; 9.3.1 Test Plan Components ; 9.3.2 Test Plan Hierarchy ; 9.3.3 Master Test Plan ; 9.3.4 Verification Test Plan ; 9.3.5 Validation Test Plan ; 9.3.5.1 Unit Test Plan ; 9.3.5.2 Integration Test Plan ; 9.3.5.3 Function Test Plan ; 9.3.5.4 System Test Plan ; 9.3.5.5 Acceptance Test Plan ; 9.4 Detailed Test Design and Test Specifications ; 9.4.1 Test Design Specifications ; 9.4.2 Test Case Specifications ; 9.4.3 Test Procedure Specifications ; 9.4.4 Test Result Specifications ; 9.4.4.1 Test Log ; 9.4.4.2 Test Incident Report ; 9.4.4.3 Test Summary Report ; Summary ; 10. Software Metrics ; 10.1 Need of Software Measurement ; 10.2 Definition of Software Metrics ; 10.3 Classification of Software Metrics ; 10.3.1 Product vs. Process Metrics ; 10.3.2 Objective vs. Subjective Metrics ; 10.3.3 Primitive vs. Computed Metrics ; 10.3.4 Private vs. Public Metrics ; 10.4 Entities to be measured ; 10.5 Size Metrics ; 10.5.1 Line of Code (LOC) ; 10.5.2 Token Count (Halstead Product Metrics) ; 10.5.2.1 Program Vocabulary ; 10.5.2.2 Program Length ; 10.5.2.3 Program Volume ; 10.5.3 Function Point Analysis ; 10.5.3.1 Process used for calculating Function Points ; 10.5.3.2 Sizing Data Functions ; 10.5.3.3 Sizing Transactional Functions ; 10.5.3.4 Calculating Unadjusted Function Point (UFP) ; 10.5.3.5 Calculating Adjusted Function Point ; Summary ; 11. Testing Metrics for Monitoring and Controlling the Testing Process ; 11.1 Measurement Objectives for Testing ; 11.2 Attributes and Corresponding Metrics in Software Testing ; 11.3 Attributes ; 11.3.1 Progress ; 11.3.2 Cost ; 11.3.3 Quality ; 11.3.4 Size ; 11.4 Estimation models for estimating testing efforts ; 11.4.1 Halstead metrics for estimating testing effort ; 11.4.2 Development ratio method ; 11.4.3 Project staff ratio method ; 11.4.4 Test procedure method ; 11.4.5 Task planning method ; 11.5 Architectural Design Metric used for Testing ; 11.6 Information Flow Metrics used for Testing ; 11.6.1 Henry & Kafura Design Metric ; 11.7 Cyclomatic Complexity measures for Testing ; 11.8 Function Point Metrics for Testing ; 11.9 Test Point Analysis ; 11.9.1 Procedure for Calculating TPA ; 11.9.2 Calculating Dynamic Test Points ; 11.9.3 Calculating Static Test Points ; 11.9.4 Calculating Primary Test Hours ; 11.9.5 Calculating Total Test Hours ; 11.10 Some Testing Metrics ; Summary ; 12. Efficient Test Suite Management ; 12.1 Why Test Suite grows? ; 12.2 Minimizing the test suite and its benefits ; 12.3 Defining the Test Suite Minimization Problem ; 12.4 Test Suite Prioritization ; 12.5 Types of Test case Prioritization ; 12.6 Prioritization Techniques ; 12.6.1 Coverage based Test Case Prioritization ; 12.6.1.1Total Statement Coverage Prioritization ; 12.6.1.2 Additional Statement Coverage Prioritization ; 12.6.1.3 Total Branch Coverage Prioritization ; 12.6.1.4 Additional Branch Coverage Prioritization ; 12.6.1.5 Total Fault-Exposing-Potential (FEP) Prioritization ; 12.6.2 Risk based Prioritization ; 12.6.3 Prioritization based on Operational Profiles ; 12.6.4 Prioritization using Relevant Slices ; 12.6.4.1 Execution Slice ; 12.6.4.2 Dynamic Slice ; 12.6.4.3 Relevant Slice ; 12.6.5 Prioritization based on Requirements ; 12.7 Measuring Effectiveness of Prioritized Test Suite ; Summary ; PART IV QUALITY MANAGEMENT ; 13. Software Quality Management ; 13.1 Software Quality ; 13.2 Quality Types ; 13.3 Broadening the concept of Quality ; 13.4 Quality Cost ; 13.5 Benefits of Investment on Quality ; 13.6 Quality Control ; 13.7 Quality Assurance ; 13.8 Quality management ; 13.9 QM and Project management ; 13.10 Quality factors ; 13.11 Methods of Quality Management ; 13.11.1 Procedural Approach to QM ; 13.11.2 Quantitative Approach to QM ; 13.12 Software Quality Metrics ; Summary ; 14. Testing Process Maturity Models ; 14.1 Need for Test Process Maturity ; 14.2 Measurement and Improvement of a Test Process ; 14.3 Test Process Maturity Models ; 14.3.1 The Testing Improvement Model (TIM) ; 14.3.1.1 Maturity Model ; 14.3.1.2 Key areas of TIM ; 14.3.1.3 The Assessment procedure of TIM ; 14.3.2 Test Organization Model (TOM) ; 14.3.2.1 Questionnaire ; 14.3.2.2 Improvement Suggestions ; 14.3.3 Test Process Improvement (TPI) Model ; 14.3.3.1 Key Process Areas ; 14.3.3.2 Maturity Levels ; 14.3.3.3 Test Maturity Matrix ; 14.3.3.4 Checkpoints ; 14.3.3.5 Improvement Suggestions ; 14.3.4 Test Maturity Model (TMM) ; 14.3.4.1 TMM Components ; 14.3.4.2 TMM Levels ; 14.3.4.3 The Assessment Model ; Summary ; PART V TEST AUTOMATION ; 15. Automation and Testing Tools ; 15.1 Need of Automation ; 15.2 Categorization of Testing Tools ; 15.2.1 Static and Dynamic Testing Tools ; 15.2.2 Testing Activity Tools ; 15.3 Selection of Testing Tools ; 15.4 Costs incurred in Testing Tools ; 15.5 Guidelines for Automated Testing ; 15.6 Overview of some commercial Testing Tools ; Summary ; PART VI TESTING FOR SPECIALIZED ENVIRONMENT ; 16. Testing Object Oriented Software ; 16.1 OOT and Structured Approach ; 16.1.1 Terminology ; 16.1.2 Object-Oriented Modelling and UML ; 16.2 Object Oriented Testing ; 16.2.1 Differences between Conventional testing Object oriented Testing ; 16.2.2 Object-Oriented Testing and Maintenance Problems ; 16.2.3 Issues in OO Testing ; 16.2.4 Strategy and Tactics of Testing OOS ; 16.2.5 Verification of OOS ; 16.2.6 Validation Activities ; 16.2.7 Testing of OO Classes ; 16.2.7.1 Feature based Testing of Classes ; 16.2.7.2 Role of Invariants in Class Testing ; 16.2.7.3 State Based Testing ; 16.2.8 Inheritance Testing ; 16.2.8.1 Issues in Inheritance Testing ; 16.2.8.2 Inheritance of Invariants of Base Class ; 16.2.8.3 Incremental Testing ; 16.2.9 Integration Testing ; 16.2.9.1 Thread based Integration Testing ; 16.2.9.2 Implicit Control Flow based Integration Testing ; 16.2.10 UML based OO Testing ; 16.2.10.1 UML diagrams in software testing ; 16.2.10.2 System Testing based on Use cases ; Summary ; 17. Testing Web based Systems ; 17.1 Web-based System ; 17.2 Web Technology Evolution ; 17.2.1 First Generation/ 2-tier Web system ; 17.2.2 Modern 3-tier & n-tier architecture ; 17.3 Differences between Traditional Software and Web-based Software ; 17.4 Challenges in Testing for Web-based Software ; 17.5 Quality Aspects ; 17.6 Web Engineering (WebE) ; 17.6.1 Analysis and Design of Web-based systems ; 17.6.1.1 Conceptual Modeling ; 17.6.1.2 Navigation Modeling ; 17.6.1.3 Presentation Modeling ; 17.6.1.4 Web Scenarios Modeling ; 17.6.1.5 Task Modeling ; 17.6.1.6 Configuration Modeling ; 17.6.2 Design Activities ; 17.6.2.1 Interface Design ; 17.6.2.2 Content Design ; 17.6.2.3 Architecture Design ; 17.6.2.4 Presentation Design ; 17.6.2.5 Navigation design ; 17.7 Testing of Web-Based Systems ; 17.7.1 Interface Testing ; 17.7.2 Usability Testing ; 17.7.3 Content Testing ; 17.7.4 Navigation Testing ; 17.7.5 Configuration/Compatibility Testing ; 17.7.6 Security Testing ; 17.7.6.1 Security Test Plan ; 17.7.6.2 Various Threat Types and their corresponding Test cases ; 17.7.7 Performance Testing ; 17.7.7.1 Performance Parameters ; 17.7.7.2 Types of Performance Testing ; Summary ; PART VII TRACKING THE BUG ; 18. Debugging ; 18.1 Debugging: An art or technique? ; 18.2 Debugging Process ; 18.3 Debugging is Difficult? ; 18.4 Debugging Techniques ; 18.4.1 Debugging with Memory Dump ; 18.4.2 Debugging with Watch Points ; 18.4.3 BackTracking ; 18.5 Correcting the Bugs ; 18.5.1 Debugging Guidelines ; 18.6 Debuggers ; 18.6.1 Types of Debuggers ; Summary ; CASE STUDIES ; REFERENCES ; APPENDICES ; APPENDIX AANSWERS TO MULTIPLE CHOICE QUESTIONS ; APPENDIX A SRS VERIFICATION CHECKLIST ; APPENDIX BHLD VERIFICATION CHECKLIST ; APPENDIX CLLD VERIFICATION CHECKLIST ; APPENDIX D GENERAL SDD VERIFICATION CHECKLIST ; APPENDIX EGENERIC CODE VERIFICATION CHECKLIST ; INDEX