Programme Evaluation in ELT: (Applied Language Studies)

This book equips readers to deal with the types of evaluation they may be involved in during their professional lives, as agents of insider or outsider evaluations or as subjects of evaluation. It provides a framework of the field, a series of case studies illustrating types of evaluation, guidance on essential evaluation procedures, and a comprehensive bibliography. Based on the practical experience of the authors as insider and outsider evaluators of institutions, teacher education programmes, language courses, short training courses and ELT projects, it will be of value as a guide for practitioners and as a coursebook for students of the field.

Table of Contents:
List of Figures. List of Tables. Acknowledgements. List of Acronyms. Part I: Overview: 1. Evaluation of Second Language Projects and Programmes: An Overview. 1. 1. Introductory. 1. 2. Evaluation: Accountability and development. 1. 3. A broader approach to evaluation. 1. 4. Aspects to evaluation. 1. 5. Evaluating evaluations. 1. 6. Endnote. 2. Recent Developments in Project Evaluation: The Issue of Baseline Design. 2. 1. Introductory. 2. 2. Baseline studies at the appraisal (pre-project) stage. 2. 3. Collecting baseline data at the implementation stage: The need to measure effects of a project and its impact. 2. 4. Conclusion. Part II: Case Studies: Introduction. 3. A Baseline Evaluation: Focus on Accountability. 3. 1. Introductory. 3. 2. The Nepal baseline study. 3. 3. Contracting The New Era for data collection. 3. 4. Selection of sample. 3. 5. Summary of the data available. 3. 6. Test data as indicators of achievement: The product dimension. 3. 7. Observation: The process dimension. 3. 8. Long-distance evaluation. 3. 9. Aspects of sampling. 3. 10. Lessons to be learned. 3. 11. Conclusions. 4. The Evaluation of a Language Programme: Development and Accountability. 4. 1. Introductory. 4. 2. Why evaluate? 4. 3. What should be evaluated and how? 4. 4. Who conducts the evaluation? 4. 5. Reflections on external evaluations of language programmes. 4. 6. Conclusions. 5. Evaluating for Development: An Initial Teacher Training Course. 5. 1. Terms of reference. 5. 2. The project. 5. 3. Principles in the approach to the formative evaluation. 5. 4. The conduct of the formative evaluation. 5. 5. Outcomes. 5. 6. Commentary on the evaluation. Part III: Methodology of Evaluation: Introduction. 6. Self-Report Methods: Interviews and Questionnaires. 6. 1. Introductory. 6. 2. Interviewing. 6. 3. Questionnaire. 6. 4. Qualitative and quantitative data analysis. 7. Classroom Observation. 7. 1. First Principles. 7. 2. Steps in collecting observational data. 7. 3. Steps in analysing observational data. 7. 4. Endnote. Part IV: Personal Considerations: 8. Political and Personal Dynamics of Evaluation. 8. 1. Introductory. 8. 2. The external evaluator: Independence and integrity. 8. 3. Insider evaluation: Ownership and climate. 8. 4. Final Reflections. Bibliography. Appendices: 1. 1. Project framework approach. 1. 1. 1. Project framework. 1. 1. 2. Project structures (The ODA project framework: typical paradigms). 1. 1. 3. Examples of the approach. 1. 2. Taxonomy of classroom research techniques. 1. 3. Hierarchy of evaluation accountability: Evaluating evaluation. 3. 1. Non-equivalent control group, pre-test/post-test design. 3. 2. Test instruments used in the Nepal study. 3. 3. Observation instruments used in the Nepal study. 3. 4. Sampling in the Nepal study. 3. 5. Establishing the language level of teachers in the Nepal study. 3. 6. An example of self-report lesson description. 4. 1. Guidelines for the BALEAP code of practice. 4. 2. Oral presentation. 4. 2. 1. Criteria of assessment - Presentation Skills and content. 4. 2. 2. Criteria of assessment - Language (TEEP). 4. 2. 3. Assessment marking sheet. 4. 3. CALS suggested marking scheme (writing skills). 4. 4. Course feedback questionnaires. 4. 4. 1. Spoken language classes. 4. 4. 2. Written language classes. 4. 4. 3. Project classes/Administration. 4. 5. Reading skills questionnaire. 5. 1. Evaluation activities. 5. 2. Sample certificado timetables. 6. Evaluation methods paradigms summary of emphases: Theses, antitheses, syntheses. 6. 1. Data arrays. 6. 1. 1. Subject totals of new overseas entrants. 6. 1. 2. Number of overseas students enrolled in universities, polytechnics, and further education colleges. 6. 1. 3. Total numbers of overseas students enrolled in the three largest discipline areas. 6. 1. 4. Total numbers of overseas students in universities against other sectors in tertiary education. 6. 1. 5. Student self-assessment. 6. 1. 6. Totals of return to questionnaire. 6. 1. 7. Problems encountered by overseas students in listening comprehension. 6. 1. 8. Problems encountered by British students in listening comprehension. 6. 1. 9. Comparison of problems encountered by British and overseas students in listening comprehension. 6. 2. 1. Students' final questionnaire: Undergraduate/postgraduate. 6. 2. 2. Teacher assessment of students' language proficiency. 6. 3. Sample structured interview. 7. 1. Formative observation - Sample instruments. 7. 2. Criteria for assessing observation systems. 7. 3. A sample of ELT teacher assessment criteria. 8. 1. Some needs identification and prioritizing activities. 8. 1. 1. Nominal group technique. 8. 1. 2. The use of Delphi technique for needs assesment. 8. 1. 3. Guidelines for review and internal development of schools (GRIDS). Index.

About the Author :
Cyril Weir and Jon Roberts are Lecturers at the Centre for Applied Language Studies, University of Reading.