Annotated Bibliography on Program Evaluation

Alkin, M.C. (Ed.) (2004). Evaluation roots: Tracing theorists’ views and influences. Thousand Oaks, CA: Sage.
A history of theoretical perspectives in evaluation, this book essentially updates the seminal Foundations of Program Evaluation (Shadish, Cook, & Leviton, 1991) by tracking the evolution of contemporary evaluation theory and practice. A metaphorical evaluation theory “tree” is used to track theorists regarding their ideas on use, methods and valuing in evaluation. Not as in-depth and comprehensive in theories as its predecessor, it nevertheless is more inclusive of ideas that have emerged in the last twenty years.

American Educational Research Association, American Psychological Association, & National Council in Measurement in Education, (1999). Standards for educational and psychological testing. Washington, DC: AERA
Part III of these standards is devoted entirely to the use of educational testing in program evaluation and public policy analysis and provides a needed ethical code for practitioners. The standards warn against the use of inappropriate testing practices in the name of “proving” institutional effectiveness, such as teaching to the test items in advance, modifying test administration procedures, discouraging certain students from participating in testing sessions or focusing exclusively on test-taking procedures. “In every case, it is important to consider the potential impact of the testing process itself” (p.166).

Bernhardt, V.L. (1998). Data analysis for comprehensive school improvement. Larchmont, NY: Eye On Education
This primer on K-12 educational data collection and analysis concentrates on four types of data important to school administrators: demographics, student learning, perceptions, and school processes. It is easy to read and offers useful examples for visually presenting data. An appendix provides helpful advice on creating building-level questionnaires.

Campbell, D.T., & Stanley, J.C. (1963). Experimental and quasi-experimental designs for research. Dallas, TX: Houghton Mifflin.
This classic monograph describes the experimental ideal as well as less rigorous designs that still may provide strong correlations for supporting hypotheses. The authors describe original notions of internal and external threats to validity. This work has renewed interest for educational evaluators in light of the emphasis on “scientific designs” insisted upon in the No Child Left Behind legislation.

Creighton, T.B. (2001). Schools and data: The educator’s guide for using data to improve decision making. Thousand Oaks, CA: Corwin Press.
The use of statistics, experimental, and quasi-experimental designs for educational studies is reviewed. Special attention to SPSS Windows is highlighted throughout the book, which is meant to be a user-friendly companion to this software. Useful explanations of what SPSS can do as a tool in the context of a school is featured.

Epstein, M.H., Kutash, K., & Duchnowski, A. (Eds.) (1998). Outcomes for children and youth with behavioral and emotional disorders and their families: Programs and evaluation best practices. Austin, TX: Pro-Ed.
Devoted to social programs designed specifically for students with behavioral and emotional disorders, this book provides multiple case studies that describe evaluations of these programs across the country. State, local, and family programs are highlighted and various evaluation and research approaches are described and recommended. The book covers a wide-range of services and studies, and is a helpful reference for program administrators and evaluators attempting to measure impact.

Fetterman, D.M. (2001). Foundations of empowerment evaluation. Thousand Oaks, CA: Sage.
Empowerment evaluation is a form of program evaluation that promotes an advocacy role for the evaluator, usually on behalf of stakeholders who are marginalized by the program through its implementation. It discounts the more traditional omnipotent external evaluator role and sees the evaluator as more than a data collector and analyzer, but also a coach and facilitator within the organization. Four case examples are provided, including one in an educational setting.

Glanz, J. (2003). Action research: An educational leader’s guide to school improvement. Norwood, MA: Christopher-Gordon Publishers.
This introduction to action research is specifically written for practitioners. Action research is seen as a practitioner’s professional responsibility in their day-to-day efforts and the book reviews basic quantitative and qualitative methods to be used. It might be particularly helpful at the Master’s level for thesis writing that addresses small, local studies.

Guskey, T.R. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin Press.
This is a practical guide for establishing an evaluation model for professional development activities in schools. The author describes well the breadth of the typical project in terms of measuring teacher satisfaction, teacher learning, administrative support, change in teacher practice, and student outcomes. Each of these components is unpacked in chapters that are easy-to-read and provide excellent examples.

Hemmeter, M.L., Joseph, G.E., Smith, B.J., & Sandall, S. (2001). Division of Early Childhood recommended practices program assessment. Longmont, CO: Sopris West
This program assessment tool is directed toward early intervention and early childhood special education programs and is based upon the Council for Exceptional Children’s Division of Early Childhood’s recommended practices. The document is complimentary to the National Association for the Education of Young Children’s accreditation document. It is practitioner-friendly and a potentially useful administrative tool for reform.

Lincoln, Y.S., & Guba, E.G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage Publications.
This is a seminal work in qualitative research that influences Guba and Lincoln’s Fourth Generation Program Evaluation approach. It reads as a counterpoint to more positivist approaches and recalls the yet to be extinguished quantitative/qualitative battle.

Patton, M.Q. (1997). Utilization-focused evaluation. Beverly Hills, CA: Sage Publications.
This is Patton’s compilation of evaluation ideas that stress the utility of evaluation efforts for stakeholders. He is at times a compelling and thoughtful author and the book forces the evaluator to think about his or her own practices. The book is at times trite, even a little silly, but the author’s points are generally a useful challenge to evaluation efforts that don’t amount to much.

Ryan, K.E., & Schwandt, T.A. (eds.) (2002). Exploring evaluator role and identity. Greenwich, CT: Information Age Publishing.
An interesting set of essays on the roles that evaluators take on as they work with stakeholders to examine programs. Identity issues and conflicts are highlighted. This is a reflective work from some of the great thinkers in the field presently.

Schalock, R.L. (2001). Outcome-based evaluation. New York: Kluwer Academic/Plenum Publishers
The author applies outcome-based evaluation specifically to special education cases in this evaluation guide. He concentrates on multiple theories and approaches, including effectiveness, impact, and policy evaluations.

Shadish, W.R., Cook, T.D., & Leviton, L.C. (1991). Foundations of program evaluation. Newbury Park, CA: Sage Publications.
A concise history of program evaluation, this book concentrates on the major theorists of the evaluation era beginning in the 1960’s through the early 90’s. Chapters are devoted to Campbell, Weiss, Scriven, Wholey, Cronbach, Stake, and Rossi. The authors provide an excellent synthesis of the theories set forth by these trailblazers.

Wholey, J.S., Hatry, H.P., & Newcomer, K.E. (eds.) (1994). Handbook of practical program evaluation. San Francisco, CA: Jossey-Bass.
A series of chapters covering several different methods to be used in program evaluation.