Program Evaluation

Course Description:

A systematic approach to program evaluation in government and nonprofit organizations is the primary focus of this course. Topics will include theories, research, and practice related to program evaluation. The skills learned from this course will assist the practitioner in determining the effectiveness of new or existing programs in organizations. The summary and communication of results to constituent groups will be a key part of the reporting process.

Required Texts:

Rossi, Peter H., Lipsey, Mark W., & Freeman, Howard E. (2003). Evaluation: A Systematic Approach, 7th edition. Thousand Oaks , CA : SAGE Publications. (ISBN: 9780761908944)

Ingraham, Patricia W., Joyce, Philip G., & Donahue, Amy Kneedler (2003). Government Performance: Why Management Matters . Baltimore : Johns-Hopkins University Press. (ISBN: 978-0-8018-7228-0)

Herr, Kathryn G. & Anderson, Gary L. (2005). The Action Research Dissertation: A Guide for Students and Faculty. Thousand Oaks , CA : SAGE Publications. (ISBN: 9780761929918)

Course Objectives:

Students will demonstrate skills associated with the program evaluation process by planning, implementing, and evaluating a project. Each person will develop both written and oral reports and deliver a summary presentation.

Upon completing this course, each student will be able to:

•  Explain program evaluation and provide information about the basic process.

•  Discuss the importance of program evaluation in relation to justification of programs and management decision making.

•  Describe the options in terms of possible methods of evaluating a program under different circumstances.

•  Present a proposal to a review group outlining the project and method of evaluation.

•  Conduct a project that involves planning and implementing a simple change activity and evaluate the outcome.

•  Summarize the project results and provide a written report and deliver an oral report to the class. This might be through video or other means of asynchronous delivery.

Assessment Measures:

•  Research Paper 20%

•  Project Proposal 10%

•  Case Study Project 30%

•  Presentation of Project Results 10%

•  Participation, Discussions 30%

Course Outline:

Unit 1: Introduction to Program Evaluation

•  Rossi text, Chapter 1. An Overview of Program Evaluation

•  Ingraham text, Chapter 1 Management, Capacity, and Performance

•  Herr text, Chapter 1. Introduction: What is an Action Research Dissertation?

Unit 2: Which Evaluation Method is Right for Your Program?

•  Rossi Chapter 2. Tailoring Evaluations

•  Ingraham Chapter 2 Dissecting Management

•  Herr Chapter 2. Action Research Traditions and Knowledge Interests

Unit 3: Getting Started with Program Evaluation

•  Rossi Chapter 3. Identifying Issues and Formulating Questions

•  Ingraham Chapter 3 Assessing Management

•  Herr Chapter 3. The Continuum of Positionality in Action Research

Unit 4: Does the Organization Need a Program?

•  Rossi Chapter 4. Assessing the Need for a Program

•  Ingraham Chapter 4 Research Methodology

•  Herr Chapter 4. Quality Criteria for Action Research

Unit 5: Putting Program Evaluation to Work

•  Rossi Chapter 5. Expressing and Assessing Program Theory

•  Ingraham Chapter 5 State and Local Findings

•  Herr Chapter 5. Designing the Plane While Flying it: Proposing and Doing the Dissertation

Unit 6: Evaluation During Program Implementation

•  Rossi Chapter 6. Assessing and Monitoring Program Process

•  Ingraham Chapter 6 Federal Results

•  Herr Chapter 6. What Does an Action Research Dissertation Look Like?

Unit 7: Data Collection for Program Evaluation

•  Rossi Chapter 7. Measuring and Monitoring Program Outcomes

•  Ingraham Chapter 7 The Big Lessons

•  Herr Chapter 7. Action Research, Ethics, and the Institutional Review Board

Unit 8: Evaluation of the Program after Completion

•  Rossi Chapter 8. Assessing Program Impact: Randomized Field Experiments

•  Ingraham Chapter 8 Next Steps

Unit 9: Other Program Evaluation Methods

Rossi Chapter 9. Assessing Program Impact: Alternative Designs

Unit 10: Dealing with Results

Rossi Chapter 10. Detecting, Interpreting, and Analyzing Program Effects

Unit 11: Efficiency as a Variable

Rossi Chapter 11. Measuring Efficiency

Unit 12: External Factors in Program Evaluation

Rossi Chapter 12. The Social Context of Evaluation

Units 13-14: Presentations & Review/Evaluation of Projects