Identifying Relevant Information for Testing Technique Selection: An Instantiated Characterization Schema

Engineering tasks are supposed to achieve defined goals under certain project constraints. Example goals of software engineering tasks include achieving a certain functionality together with some level of reliability or performance. Example constraints of software engineering tasks include budget and time limitations or experience limitations of the developers at hand. Planning of an engineering project requires the selection of techniques, methods and tools suited to achieve stated goals under given project constraints. This assumes sufficient knowledge regarding the process-product relationships (or effects) of candidate techniques, methods and tools. Planning of software projects suffers greatly from lack of knowledge regarding the process-product relationships of candidate techniques, methods and tools. Especially in the area of testing a project planner is confronted with an abundance of testing techniques, but very little knowledge regarding their effects under varying project conditions. This book offers a novel approach to addressing this problem: First, based on a comprehensive initial characterization scheme (see chapter 7) an overview of existing testing techniques and their effects under varying conditions is provided to guide the selection of testing approaches. Second, the optimisation of this knowledge base is suggested based on experience from experts, real projects and scientific experiments (chapters 8, 9, and 10). This book is of equal interest to practitioners, researchers and students. Practitioners interested in identifying ways to organize their company-specific knowledge about testing could start with the schema provided in this book, and optimise it further by applying similar strategies as offered in chapters 8 and 9.

Table of Contents:
1. Introduction.- 1 Basics on Software Testing.- 2 The Problem of Selecting Testing Techniques.- 3 Getting a Characterisation Schema for Testing Techniques.- 4 Organisation of the Book.- 2. State of the Practice.- 1 Areas Surveyed.- 2 Testing Area.- 3 Area of Characterisation.- 4 Conclusions on the State of the Practice.- 3. Research Goals.- 1 Description of the Problem.- 2 Problem-Solving Approach.- 3 Hypotheses.- 4. Problem Solving Process.- 1 Research Methods.- 2 Objectives of Technological Research.- 3 Applying the Scientific Method to Technological Research.- 4 Expert Peer Review versus Experimental Testing.- 5 The Problem-Solving Process Used in this Book.- 5. First Generative Iteration: Deductive Theoretical Schema.- 1 Description of the Research Stage: Generation of a Theoretical Schema.- 2 Testing Process.- 3 Stratification of Testing-Related Information.- 4 Testing Process Elements.- 5 Attributes of the Theoretical Schema.- 6 Result of the Construction of the Theoretical Schema.- 7 Use and Evolution of the Characterisation Schema.- 6. Second Generative Iteration: Inductive Empirical Schema.- 1 Description of the Research Stage: Generation of an Empirical Schema.- 2 Data Collection.- 3 Data Analysis.- 4 Result of Building the Empirical Schema.- 5 Study of the Evolution of the Empirical Schema.- 7. Synthesis of Perspectives: Proposal of the Preliminary Schema.- 1 Description of the Research Stage: Synthesis of Schemas.- 2 Rules of Synthesis.- 3 Synthesis of the Theoretical and Empirical Schemas.- 4 Result of Schema Synthesis.- 5 Study of Schema Synthesis.- 8. Improvement of the Schema: Expert Peer Review.- 1 Description of the Research Stage: Improvement.- 2 Questionnaire for Experts.- 3 Questionnaire Analysis Method.- 4 Analysis of Responses.- 5 Improved Schema.- 9. Empirical Evaluation.- 1 Objective of the Empirical Evaluation.- 2 Choice of the Workload.- 3 Analysis of the Results.- 4 Conclusions on the Empirical Evaluation.- 10. Experimental Evaluation.- 1 Objective of the Experiment.- 2 Experiment Planning.- 3 Experimental Design.- 4 Data Analysis.- 5 Conclusions on Experimental Evaluation.- 6 Characterisation Schema Improvement.- 11. Conclusions.- Appendices.- Forms Used to Obtain the Empirical Schema.- Questionnaires Used in Expert Peer Review.- Schema Instantiation.- Experiment Forms.