TMap Test Topics

This book talks about software testing. "TMap[registered] Test Topics" describes the application of the generic test methodology TMap to current IT themes. These themes range from package implementations and data warehouses, via iterative development with components, RUP[registered] en DSDM[registered], to PRINCE2[registered]. In addition, attention is given to themes in the field of testing itself: testing chains of systems and processes, exploratory testing, issues surrounding outsourcing and testing, and - in keeping with the current interest in IT governance - business-driven test management. The latter theme describes how the (TMap) test process is set up in such a way that the business as a client can control the test process on the basis of cost, end date, and risks. "TMap[registered] Test Topics" was written for (organisations with) testers who are familiar with the TMap methodology and wish to know how to apply TMap in their specific situation. This book is based on the standard work "Software Testing, a Guide to the TMap[registered] Approach", written by Martin Pol, Ruud Teunissen and Erik van Veenendaal. A wealth of additional TMap information, including the appendices of this book, can be found on the web site. By using contributions from many test experts, both authors and reviewers, this book contains the best available on the market. Final editing was in the hands of the Business Development department of Software Control, the Test and QA division of Sogeti Nederland BV. Both editors work in this department. Tim Koomen (left) focuses on developments in the field of testing, including agile testing, iterative development methods, testing of software packages, data warehousing, etc. Tim is co-author of the book "Test Process Improvement", has written various articles, regularly presents training courses, and gives presentations at international congresses. In 2003, he received the European Testing Excellence Award for his work in the field of TPI[registered], TMap[registered] and testing in general. Rob Baarda's interest in testing as a method started in 1986, after setting up automated test execution. Rob monitors the relationship between developments in the IT market and TMap[registered], and converts these into actions. He teaches and regularly speaks about TMap-oriented topics, including risk-based testing, testing as an economic process, and TMap[registered] and PRINCE2[registered]. Many colleagues throughout the world use TMap with a great deal of enthusiasm. The standard's widely praised completeness offers optimum possibilities to create a tailor-made test process. Although it originates from an administrative environment, TMap's philosophy can be applied to any object. TMap represents the basis of testing expertise and offers the most suitable test. When I witnessed the birth of TMap many years ago, I would never have surmised that the book would contribute so much to the awareness and positioning of testing. In the Netherlands, Belgium, but also elsewhere because of its availability in German and English. This requires a great deal, it requires innovation, and it requires continuous co-ordination with other developments in IT and (especially) the business. I am therefore proud that this book was created with combined powers, that it optimises the coverage of TMap and that our testers are in a position to detect risks sooner and better. - Martin Pol. From the first moment that TMap was published, many testers, companies, organisations, et cetera, started working with the - operational - application of what TMap had to offer. They did so with a great amount of eagerness and enthusiasm. The publication of the 2nd edition at the end of 1999 and the full English version in 2002 have also shown that TMap fills a need, which is the need for professionalisation of testing. I believe that the publication of "TMap[registered] Test Topics" is a very welcome complement to the standard that TMap has by now become, offering users the possibility of tailoring it optimally to their own situation. In addition to more in-depth strategy development - in fact a broadening into business-related risks - it looks closely at the most important innovations in IT and presents the application of TMap by users in specific situations, based on experience in the field. Again, TMap appears to be generic enough in order to provide a footing in new situations, which we as testers face regularly. I think that 'TMap Red' is a useful complement to the standard and contributes to the, ever-growing, professionalisation of the world of testing. - Ruud Teunissen. With the publication of this new TMap book, my thoughts go back to 1994. Working together with Martin and Ruud on "Testing according to TMap" (the Dutch TMap book), writing in the evenings and weekends, frequent and long discussions on the phone. It has been approximately 10 years since the first TMap book was published. In those 10 years, TMap has managed to survive major changes in the IT industry and has grown to be the Dutch and Belgian test standard. Many companies have defined their own organisation-specific standard, but these are almost always based on TMap. In particular the phases and the techniques are by now solidly rooted in the Dutch way of thinking regarding structured testing. Of course the landscape has changed considerably in the last few years, partly as a result of the emergence of incremental and iterative developing methods such as RUP, DSDM and XP. This book provides testers with the handles for applying TMap also in those types of environment. I believe that the in-depth description of the relation between project management (PRINCE-2) and test management (TMap) meets a need in the market. This book shows that the two methods can be applied together and in actual fact strengthen each other. I am glad that I was able to contribute also to this TMap book in the form of reviewing most of the chapters. As an author of "Software Testing, a Guide to the TMap Approach", I want to congratulate Sogeti, and especially Tim Koomen and Rob Baarda, with this book. TMap is, ten years after, again attracting a great deal of attention - on to the next ten years! - Erik van Veenendaal, May 2004. IT is inseparably linked with modern business operations. In the last few years, after the Internet hype and accountancy scandals, it has become clear that IT, just like any other part of the organisation, must be managed well. IT governance is the current term for this. It means that decision-making concerning IT projects must consider results, time, costs and risks. Decision-making is necessary both in choosing the projects to be executed and in the continuation or completion of projects being carried out. The information from testing can make a major contribution to the latter decision-making. Solid, structured testing provides the information on the risks and quality of IT solutions that have already been implemented in relation to the expected results. For years, TMap[registered] has been the leading test method, both in Europe and elsewhere. In order to increase the added value of the method for you as a user of TMap[registered], Sogeti has created a better link with the business in this edition with Business-Driven Test Management. In doing so, Sogeti has also anticipated typical IT developments. From our IT expertise, we feel responsible for sharing with you our experience gained through practice. We have brought these subjects together in a book with the title "TMap[registered] Test Topics". With this new edition, Sogeti makes the latest expansions of TMap available to you in print form. We also provide a great deal of additional information via the website. I hope you will enjoy reading this book and I am sure that it may help you! - Hans van Waayenburg, Chief Executive Officer, Sogeti Nederland BV. This book covers a wide range of topics relating to the central theme of TMap. We have worked on it with great pleasure for the past year and a half. The creation of this book, with its variety of topics, clearly demanded the involvement of a large number of people. Their involvement varied, from creating the necessary preconditions and sharing experiences to reviewing and (co-)authoring chapters. We would like to take this opportunity to thank all these people. Firstly, we would like to thank the steering committee, and in particular the director of Software Control, Ronald Spaans, for giving us the time to write this book. We are grateful that Martin Pol was willing to share his own new insights in the form of a chapter in the book. We also thank Sylvia Verschueren for describing her experiences in applying TMap in a DSDM environment. A special mention and word of thanks go to Anja Bakker, who wrote the complex chapters on business-driven test management in such a clear way. For designing and writing the chapters, we thank all those at Sogeti who dealt with the new IT developments in their daily activities and shared their knowledge and experience of testing with us. They have been mentioned in the chapter to which they contributed their knowledge and experience as authors, contributors and/or reviewers. A large number of active TMap users, many of whom are amongst our clients, have played an important role reading and commenting on the chapters, for which we are extremely grateful. They were (in alphabetical order): Evert-Jan van den Berg (Flextronics); Kris Dugardyn (IBM Rational); Cees Dulfer (Rabobank); Egbert Egberts (ING/OPS and IT); Ed van der Geest (SNS Bank); Harry Kobes (PANalytical B.V.); Sander Koopman (Fortis Bank ISE); Erik Kwast (Informatie Beheer Groep); Jan Meeuws (Interpolis); Meile Posthuma (ING Netherlands OPS and IT/Test Services); Maurice Siteur (Capgemini); Brian Taylor (FortisBank); Liduine Thorn (employed by ABN AMRO); Erik van Veenendaal (Improve Quality Services); Sylvia Verschueren (employed by ABN AMRO); and Johan Vink (Ministry of Agriculture, Nature and Food quality). We have made every effort to reflect their positive critical comments properly in the various chapters. It may sound like a cliche, but we mean it sincerely: without the contribution of all these people, this book would not have been possible. We hope that you, as a reader, will enjoy reading this book and may gain useful inspiration from it for your own test practice. - Tim Koomen, Rob Baarda, Rotterdam, May 2004. After the publication of the "Dutch TMap Test Topics" book, we received numerous requests for translations of certain chapters. In part, this is because of the increasing popularity of TMap in the rest of the world. However, we found that the topics in the book appeal to users of TMap, the central theme of the book, and many testers in general. For these reasons, Sogeti decided to publish this English translation, exactly one year after the Dutch publication. Publishing a translation, though easier than the original book, poses its own challenges: sentences perfectly readable in Dutch look horrible when translated literally; and the specific test and IT terms used are often hard to translate. We have had help from many people, including a professional translator, many Dutch colleagues, and colleagues from several other Sogeti companies, including Sweden and the USA. Although there isn't enough room to thank everybody by name, we want to give a special word of thanks to our USA colleagues who have helped with several parts of the translation: Dan Hannigan, Hans Kapteijns, Craig Mayer, and Camille Tetta. If the reader should notice any errors in the book, these are entirely our fault and not the fault of anyone else. - Tim Koomen, Rob Baarda, Rotterdam, May 2005.

Table of Contents:
Preface, 5; Acknowledgments, 8; Introduction, 17; 1.1 Background, 17; 1.2 TMap on the market, 18; 1.3 Structure of this book, 18; 1.4 Guideline, 19; Part 1: Business-Driven Test management with TMap; 2 Business-driven test management in a bird's-eye view, 23; 3 Formulating the assignment, 29; 4 Product Risk Analysis, 33. 4.1 Stakeholders, 34; 4.2 Identifying risks, 34; 4.3 Risk classification, 36; 4.4 Result, 38; 5 Test strategy, 39; 5.1 Persons involved, 39; 5.2 Relation between product risk analysis and test strategy, 40; 5.3 A test strategy for multiple test levels, 40; 5.3.1 Assigning risks to quality characteristics, 41; 5.3.2 Assigning risks and quality characteristics to test levels, 42; 5.4 One test strategy for an individual test level, 45; 5.4.1 Determining object parts, 46; 5.4.2 Determining risks per object part and quality characteristic, 46; 5.4.3 Determining the thoroughness of testing and the test approach, 47; 5.4.4 Determining test techniques to be used, 48; 5.5 Result, 52; 6 Estimation and planning, 53; 6.1 Estimation, 53; 6.1.1 Estimating based on ratios, 57; 6.1.2 Estimation based on size and strategy, 58; 6.1.3 Estimation based on size, 62; 6.1.4 Work Breakdown Structure, 62; 6.1.5 Review estimate approach, 63; 6.1.6 Proportional estimation, 64; 6.1.7 Extrapolation, 65; 6.2 Planning, 65; 7 Reporting and Managing, 67; 7.1 Feedback loop in the planning phase, 67; 7.1.1 Fix (master) test plan, 72; 7.2 Managing and feedback during the execution, 72; 7.2.1 Monitoring and managing, 72; 7.2.2 Reporting, 76; Part 2: Applying TMap; 8 TMap-Light, 85; 8.1 Introduction, 85; 8.1.1 Why TMap-light?, 85; 8.1.2 Risks of the TMap-light approach, 86; 8.2 Description of TMap-light, 86; 8.3 Life cycle, 87; 8.4 Techniques, 89; 8.4.1 Development of a test strategy, 89; 8.4.2 Test specification techniques, 90; 8.5 Infrastructure, 91; 8.6 Organisation, 91; 8.6.1 Setting up the test organisation, 91; 8.6.2 Progress control and reporting, 92; 8.6.3 Defect management, 92; 8.6.4 Testware maintenance, 92; 9 Outsourcing of testing, 93; 9.1 Introduction, 93; 9.2 Outsourcing in general, 93; 9.3 Special challenges when outsourcing, 94; 9.4 The process of outsourcing, 95; 9.4.1 Goal, scope and approach, 96; 9.5 Selection of supplier, 97; 9.5.1 Establishing the contract, 99; 9.5.2 Implementation/transition, 100; 9.5.3 Control, 102; 9.6 Success factors and risks, 106; 10 Testing software package implementations, 109; 10.1 Introduction, 109; 10.2 Implementation of software packages, 109; 10.2.1 What is a software package?, 109; 10.2.2 Advantages, 110. 10.2.3 Appropriate package?, 110; 10.2.4 Introduction of a package, 112; 10.2.5 Maintenance, 113; 10.3 Why test?, 113; 10.3.1 Testing software packages, 113; 10.3.2 Why an independent test partner?, 114; 10.4 How to test?, 115; 10.4.1 What is different?, 115; 10.4.2 Quality characteristics, 116; 10.4.3 Test levels, 119; 10.4.4 Test basis, 123; 10.4.5 Life cycle, 124; 10.4.6 Organisation, 124; 10.4.7 Test specification techniques, 126; 10.4.8 Test tools & test environment, 127; 10.4.9 Maintenance, 128; 11 Testing in iterative development (using components), 129; 11.1 Introduction, 129; 11.2 iCBD, 129; 11.2.1 Background, 129; 11.2.2 Component Based Development, 130; 11.2.3 Iterative and incremental development, 132; 11.3 iCBD and testing, 135; 11.3.1 Methodologies and Testing, 135; 11.3.2 Challenges for testing, 135; 11.4 Structured testing in iCBD, 138; 11.4.1 Introduction, 138; 11.4.2 Test strategy, 138; 11.4.3 Test levels, evaluations and the master test plan, 142; 11.4.4 Test basis, 146; 11.4.5 Life cycle, 148; 11.4.6 Techniques, 148; 11.4.7 Infrastructure, 149; 11.4.8 Organisation, 151; 12 Testing datawarehousing and business intelligence, 153; 12.1 Introduction, 153; 12.2 Data warehousing: converting data to information, 153; 12.2.1 Differences with traditional information systems, 154; 12.3 Components of data warehousing, 155; 12.3.1 Source data, 155; 12.3.2 Extraction, Transformation, Load, 156; 12.3.3 Data warehouse, 156; 12.3.4 Distribution (optional), 158; 12.3.5 Data marts (optional), 158; 12.3.6 Queries, OLAP or data mining, 158; 12.3.7 Data warehousing processes, 159; 12.4 Testing data warehousing, 159; 12.4.1 Main differences, 159; 12.4.2 Test strategy, 160; 12.4.3 Test levels, 164; 12.4.4 Evaluations, 166; 12.4.5 Test basis, 167; 12.4.6 Life cycle, 168; 12.4.7 Techniques, 168; 12.4.8 Infrastructure, 169; 12.4.9 Organisation, 171; 13 Exploratory Testing, 173; 13.1 Introduction, 173; 13.2 Exploratory testing, 174; 13.2.1 What is it?, 174; 13.2.2 When to apply it?, 177; 13.2.3 The Exploratory Tester, 180; 13.2.4 Comparison with error guessing, 181; 13.2.5 Importance of ET, 181; 13.3 Session-based test management, 182; 13.3.1 Introduction, 182; 13.3.2 Test charters, 182; 13.3.3 Test session, 183. 13.4 Fitting ET/SB into TMap, 185; 13.4.1 TMap Planning (& control) phase, 186; 13.4.2 TMap Preparation phase, 187; 13.4.3 TMap Specification phase, 188; 13.4.4 TMap Execution phase, 189; 13.4.5 TMap Completion phase, 190; 13.4.6 Organisation, 190; 13.4.7 Techniques, 191; 13.4.8 Infrastructure, 191; 13.5 Conclusion, 192; 14 System integration testing, 193; 14.1 Introduction, 193; 14.2 What is system integration testing, 193; 14.2.1 Description, 193; 14.2.2 What makes it so special, 194; 14.3 Test strategy, 195; 14.3.1 Strategy development, 195; 14.3.2 Quality characteristics, 196; 14.4 Life-cycle, 198; 14.5 Techniques, 200; 14.6 Organisation, 200; 14.6.1 Parties involved, 200; 14.6.2 Positions/roles/responsibilities, 201; 14.6.3 Budget, 201; 14.7 Test infrastructure, 202; Part 3: Methodological relationships; 15 TMap within Rational Unified Process, 205; 15.1 Introduction, 205; 15.2 RUP, 206; 15.2.1 Best practices, 206; 15.2.2 Phases, 207; 15.2.3 Disciplines, 208; 15.3 RUP Testing, 210; 15.3.1 Master Test Plan and Iteration Test Plan, 211; 15.3.2 Unit Test (UT), 211; 15.3.3 Integration Test (IT), 212; 15.3.4 System Test (ST), 212; 15.3.5 Acceptance Test (AT), 214; 15.3.6 Reviews, 215; 15.3.7 Roles, 215; 15.4 Fitting TMap into RUP, 215; 15.4.1 Master test plan, 215; 15.4.2 Life-cycle, 216; 15.4.3 Organisation, 229; 15.4.4 Techniques, 229; 15.4.5 Infrastructure, 230; 15.4.6 Acceptance test, 230; 16 Case study: TMap and DSDM,233; 16.1 Introduction, 233; 16.2 Our organisation and testing approach, 233; 16.2.1 Our organisation, 234; 16.2.2 Current testing approach, 234; 16.3 DSDM, 235; 16.3.1 Outlines of DSDM, 235; 16.3.2 Status, 236; 16.4 Process of mapping the testing approach on DSDM, 237; 16.5 First impressions, 238; 16.6 Our approach, 240; 16.6.1 Life cycle and deliverables, 240; 16.6.2 Roles, 248; 16.7 Implementation, 248; 16.8 Experiences in mapping the current approach, 249; 16.9 Results in projects, 250; 17 TMap within Prince2, 255; 17.1 Introduction, 255; 17.2 What is PRINCE2?, 256. 17.2.1 The PRINCE2 phenomenon, 256; 17.2.2 Important concepts from PRINCE2 for testing, 261; 17.3 Outlines of TMap in PRINCE2, 263; 17.3.1 The TMap cornerstones in PRINCE2, 264; 17.3.2 Main links with TMap, 265; 17.4 Test management in PRINCE2 according to TMap<^>[registered]<^*>, 268; 17.4.1 TMap processes in PRINCE2 processes in detail, 268; 17.4.2 The PRINCE2 components and TMap, 276; 17.4.3 PRINCE2 techniques and TMap, 280; 17.4.4 Product and role descriptions and TMap, 281; Appendix A: Checklist of properties and preconditions, 283; Appendix B: Checklist of risk questions, 284; Appendix C: Exit and Entry Criteria, 285; Appendix D: Negotiability principle, 287; Appendix E: Testing on the basis of Use Cases, 292; Appendix F: Master test plan according to RUP and TMap, 294; Appendix G: Detailed comparison of RUP and TMap test activities, 295; Appendix H: Comparing TMap products and RUP artifacts, 303; Appendix I: The relevant TMap concepts in PRINCE2, 305; Appendix J: Work Package types for testing, 308; Appendix K: Changes BDTM with TMap, 311; Literature, 313; Sogeti, 317; Authors, 319; Index, 321.

About the Author :
Rob Baarda Rob Baarda, having spent 7 years in IT development, started his career in testing in 1986, with automated test execution. That was the origin of his interest in testing as a method. Today, Rob works in the field of testing for the Business Development department at Sogeti. He frequently teaches and speaks about TMap-related topics, including risk-based testing, testing to prevent damage, and TMap[registered] and PRINCE2. Anja Bakker Anja Bakker has worked in the field of testing since 1995. Her experience includes leading tests, executing a large number of TPI assessments, and supervising the implementation of test improvements. In addition, Anja has developed various test training courses, including a test course for the Dutch Open University, and is closely involved in setting up a TMap certificate for EXIN. Anja also works as delivery manager for the Software Control division at Sogeti. Tim Koomen Tim Koomen works for Sogeti's Business Development department, where he is involved in the developments in the field of testing, such as agile testing, iterative development methods, testing software packages, data warehousing, etc. Tim is co-author of the book Test Process Improvement, has written various articles, regularly gives courses, as well as presentations at international congresses. In 2003, he received the "European Testing Excellence Award" for his work in the field of TPI, TMap and testing in general. Martin Pol Martin Pol is a senior test consultant for Polteq International Testing Services B.V., the company that he founded at the beginning of 2000. Since 1983, he has worked full-time in (international) testing and has a great deal of experience managing test projects and improving test processes for many organisations in various sectors. He is one of the founders of structured testing in the Netherlands. Martin Pol is co-author of the books on TMap[registered] and TPI[registered] and recently worked on the ISEB foundation standard for Dutch. He is a valued speaker at conferences and a teacher of training courses in Europe, the Americas, Asia and Australia. In 1998 Martin Pol received in Munich the "European Testing Excellence Award" for his enormous contribution to the field of testing in Europe. For some years, Martin Pol has been involved intensively in organisations to (also) successfully outsource testing and how to manage outsourced testing. Mark Tolsma Mark Tolsma has worked in IT since 1987, and has been a professional tester since 1995. He has many years of experience in test management and setting up and consolidating structured testing in projects and organisations. Mark is actively involved in innovations in the field of testing, including the recent introduction of TMap in the iterative system development method RUP. Sylvia Verschueren Sylvia has worked in IT at ABNAMRO for almost 20 years. In 1995, she changed from technical design to the department that is responsible for methods, techniques and tooling for software development. In this department, in her capacity of test consultant, she has become involved in testing. In addition to supporting projects (advising, coaching, giving training sessions & workshops, and carrying out reviews), she has been involved in the introduction of a standard test method (including later versions with regard to packages, web applications, CBD projects, and DSDM projects), an approach for automated testing and test improvement programs. Furthermore, she has acted as test manager and test co-ordinator in projects. Gerrit de Vries Gerrit de Vries is senior test consultant for Sogeti Nederland BV and has a great deal of experience in the field of structured testing. After working in test execution and test management, he is now a TPI[registered] Consultant (Test Process Improvement), providing support and advice for various test programs, both nationally and internationally (Germany and Sweden). His competencies include setting up large test projects, implementing a structured test approach, executing assessments of test maturity and optimising existing test processes.