"Proof," Policy, and Practice: Understanding the Role of Evidence in Improving Education

How can we “fix” our schools? Improve graduation rates in college? What works?These are questions that make the headlines and vex policy makers, practitioners, and educational researchers. While they strive to improve society, there are frequently gulfs of mutual incomprehension among them.Academics, longing for more influence, may wrongly fault irrationality, ideology, or ignorance for the failure of research to inform policy and practice more powerfully. Policy makers and practitioners may doubt that academics can deliver ideas that will reliably yield desirable results. This book bridges the divide. It argues that unrealistic expectations lead to both unproductive research and impossible standards for “evidence-based” policy and practice, and it offers promising ways for evidence to contribute to improvement. It analyzes the utility and limitations of the different research methods that have been applied to policy and practice, as well as the strengths and weaknesses of educational reform strategies. It explains why using evidence for “accountability” often makes things worse rather than better.Paul Lingenfelter offers educational researchers and policy makers a framework for considering such questions as: What problems are important and accessible? What methods will be fruitful? Which help policy makers and practitioners make choices and learn how to improve? What information is relevant? What knowledge is valid and useful? How can policy makers and practitioners establish a more productive division of labor based on their respective capabilities and limitations? He cautions against the illusion that straight-forward scientific approaches and data can be successfully applied to society’s most complex problems. While explaining why no single policy or intervention can solve complex problems, he concludes that determination, measurement, analysis, and adaptation based on evidence in specific situations can lead to significant improvement. This positive, even-handed introduction to the use of research for problem-solving concludes by suggesting emerging practices and approaches that can help scholars, practitioners, and policy leaders become more successful in reaching their fundamental goals.

Table of Contents:
Figures and Tables Foreword Introduction Acknowledgements Introduction 1. Proof and Policy 2. Experimentation and Program Evaluation 3. Measurement 4. Measurement. Fit for Which Purposes? 5. Getting Better. The Use of Evidence in Practice 6. Research and Evidence in Government and Organizations 7. What Works, When, and How. Reflections, Observations, and Opinions Bibliography About the Author Index

About the Author :
Paul E. Lingenfelter has had an unusually diversified career bridging research and public policy. At the University of Michigan during his doctoral studies he staffed university committees awarding faculty research grants and examining the role of the dissertation in graduate education for the Rackham School of Graduate Studies. At the Illinois Board of Higher Education from 1974 to 1985 he helped shape the state’s funding model for community colleges, planning and budgeting systems and cost studies for public colleges and universities, and state budgets and policies for all sectors of higher education. He was Deputy Director for Fiscal Affairs from 1980 to 1985. At the John D. and Catherine T. MacArthur Foundation from 1985 to 2000 he initially served as Director of Program Evaluation. In 1996 he became Vice President of the Program on Human and Community Development, which supported research and practice in economics, mental health and human development, education, and community development. From 2000 to 2013 Lingenfelter served as CEO/President of the association of State Higher Education Executive Officers (SHEEO), where he staffed the National Commission on Accountability in Higher Education, created the annual study State Higher Education Finance, and worked to build stronger relationships between K-12 and higher education. He is now Senior Scholar at the National Institute for Learning Outcomes Assessment at the University of Illinois. Lingenfelter has been a frequent participant in international meetings on higher education issues including governance, budgeting, quality assurance, and the assessment of student learning. He has published and spoken widely on issues in higher education and philanthropy, and he has served on numerous education boards and commissions.He is the 2019 recipient of the Excellence in Public Policy in Higher Education Award. Michael S. McPherson serves as the President at The Spencer Foundation in Chicago. Prior to joining t

Review :
"This book, whose author has been deeply involved in enacting and analyzing higher education policy in the United States for over two decades, makes an important contribution to our understanding of a key question: How do we know what policies actually work? The volume articulately describes the disconnect between academic researchers, who often pursue their own questions of interest, and the policy world, which is desperate for usable knowledge to help guide important decisions. Drawing on examples from education and other disciplines, it helps us understand why this chasm exists and how it best can be bridged." Donald E. Heller, Dean, College of Education Michigan State University "Understanding public policy -- not just what it is, but how it is created -- is essential knowledge for college leaders. Paul Lingenfelter's book provides new insights in how to construct evidence-based public policies based on solid research and evaluation." Michael N. Bastedo Professor; Director, Center for the Study of Higher and Postsecondary Education University of Michigan "To improve educational outcomes, policymakers and practitioners need high-quality, relevant data and research that informs understanding of what works, for which students, in what contexts. Lingenfelter illuminates the constructive tension among practitioners, policymakers and researchers that is needed to design and answer important questions of educational improvement, cautions against ideological singularity in research methods and “silver bullet” studies or solutions, and offers useful recommendations for ensuring that research is consequential." Laura W. Perna and Joni E. Finney, Graduate School of Education The University of Pennsylvania, and co-authors of The Attainment Agenda: State Policy Leadership in Higher Education “Lingenfelter takes on the longstanding and highly problematic relationship among research, policy and practice. He unmasks modern-day shibboleths about how performance management and randomized field trials are the new answers. While highly respectful of practitioner wisdom and judgment, he marks out clear limits here too. He argues persuasively that those engaged in the work of education must become active agents of its continuous improvement, and sketches out how policymakers can foster an environment where such systematic gathering and use of evidence is more likely to happen. This is a very wise book!” Anthony S. Bryk, President Carnegie Foundation for the Advancement of Teaching “In education, policy makers and Practitioners look to social scientists and researchers to tell them ‘what works.’ While each of these professional specialties share the goal of improving educational outcomes, their working relationship is uneasy, and the resulting educational initiatives and interventions rarely deliver the promised results, and often have unintended, and even undesirable consequences. Lingenfelter hopes to improve the situation by showing how evidence is used (and misused) in forming and applying educational policy, and by helping each of the professional specialties better understand each other’s needs, methods, capabilities, and concerns so, working together, they might become more successful. Whether you are professionally involved in educational reform, or are just interested in a better understanding of the issues, “Proof,” Policy, & Practice has much to offer.” Technical Communications “The interactions between policy making and research were once vividly described by scholars in a metaphor of dancing in the dark, where dancers do not completely see each other, the movements are complex, and the environment influences the flow of the dance. “Proof,” Policy, & Practice: Understanding the Role of Evidence in Improving Education by Paul E. Lingenfelter attempts to dismantle this black box and decode the complex dancing movements in a critical and reflective approach to analysis by disclosing the role of evidence in the interactions among research, policy, and practice. [This] book is a thoughtful and critical analysis of the significant role of evidence in improving education policy and practice in situations filled with complexities. The text awakens the idealist believers of improvement science and confronts them with the harsh reality that, ‘no single program, no single policy, no simple pill-like interventions can solve complex problems of policy and practice’. It urges all scholars, policy makers, and practitioners to reflect on whether and how research evidence works to promote improvement. The key for seeking evidence is to be critical and the objective should be held for probing, rather than proving.” Teachers College Record "Paul Lingenfelter has produced a 'must read' for the layman and reflective fodder for the seasoned policymaker. This common sense and thorough walk through the development of assessment practice and policy, punctuated with diligent attention to research, should become required reading for educational policymakers. Few people could have written this book. Paul's wisdom and keen observations are clearly the result of a lifetime of learning and reflection from multiple experiences and schooled by his persistent quest for knowledge. That rich set of life's experiences come together to produce an outstanding contribution to education policy. Throughout the book Paul takes clear stances on controversial and perplexing issues and does not shy away from specific advice on the uses and role of assessment practice and policy." Gene Wilhoit, CEO,Center for Innovation in Education former Executive Director of CCSSO and former education commissioner in Kentucky and Arkansas “[The] problem of how to act effectively in the world knowing that there is never enough time or evidence to perfect one’s decisions is as old as Aristotle and as contemporary as reforming the U.S. financial aid system for higher education. This book is richly informed both by Lingenfelter's extensive experience in scholarship, foundation leadership, and public affairs and his exceptionally perceptive reading of a wide range of relevant literature. His writings range over problems of inference, of measurement, of governmental, institutional, and individual decision-making. He draws on examples in and out of education, in the U.S. and abroad. This is a fascinating, well argued, and yet in some ways personal book. You have in your hands a deeply informed and instructive book that is also a pleasure to read. I expect that you will enjoy and profit from it.” Michael S. McPherson The Spencer Foundation "Now that everything worth investing in (from skin lotions to strategies to reduce poverty) must be 'evidence-based,' the question of what should be considered credible evidence has become critical. To the rescue comes Paul Lingenfelter’s book, “Proof,” Policy and Practice. His writing is unique in its clarity, insight and wisdom about the weaknesses of current approaches to collecting and applying evidence, and how to do it better – especially when it comes to solving complex social problems." Lisbeth B. Schorr, Senior Fellow Center for the Study of Social Policy