bg-cover

Program Evaluation

Evaluating the educational programs that Extension professionals deliver is the key to success in all Extension Program. These resources, while extensive can assist in developing qualitative and quantitive program evaluation.

Speaker: Mary Arnold, Ph.D.

http://breeze.wsu.edu/p32528684/
COMPREHENSIVE APPROACHES TO EVALUATION

  1. W.K. Kellogg Foundation Evaluation Handbook.  http://www.wkkf.org/knowledge-center/resources/2010/W-K-Kellogg-Foundation-Evaluation-Handbook.aspx
  2.  UNFPN The Programme Managers Planning, Monitoring and Evaluation tool kits specifically go to: http://www.seachangecop.org/node/1512 (there are six tool kits and links are listed there). The first one has a glossary. The fourth one can also be found here: http://betterevaluation.org/sites/default/files/stakeholder.pdf
  3.  My M & E – A platform for sharing knowledge and practice amongst M & E practitioners worldwide—multiple links on multiple topice. (www.mymande.org)
  4. Evaluate”, Evaluation centre operated by Western Michigan University, Specializing in National Science Foundation (NSF) Evaluations. (www.evalu-ate.org)

 IMPLEMENTATION, MONITORING AND DELIVERY

  1.  World Bank Overview of Monitoring and Evaluation. http://siteresources.worldbank.org/EXTEVACAPDEV/Resources/4585672-1251481378590/MandE_tools_methods_approaches.pdf
  2. Questionnaire Design: Asking questions with a purpose, Ellen Taylor-Powell. http://learningstore.uwex.edu/Assets/pdfs/G3658-02.pdf
  3.  Sampling, Ellen Taylor-Powell. http://learningstore.uwex.edu/Assets/pdfs/G3658-03.pdf
  4. Collecting evaluation data: An overview of sources and materials, Ellen Taylor-Powell & Sara Steele. http://learningstore.uwex.edu/Assets/pdfs/G3658-04.pdf
  5. Collecting evaluation data: Direct observation, Ellen Taylor-Powell & Sara Steele. http://learningstore.uwex.edu/Assets/pdfs/G3658-05.pdf
  6. Collecting evaluation data: Surveys, Ellen Taylor-Powel &, Carol Hermann.  http://learningstore.uwex.edu/Assets/pdfs/G3658-10.pdf
  7. Collecting evaluation data: End-of-session questionnaires, Ellen Taylor-Powell & Marcus Renner. http://learningstore.uwex.edu/Assets/pdfs/G3658-11.pdf
  8. Summary of evaluation models in Payne, D.A. (1994). Designing educational project and program evaluations: A practical overview based on research and experience. Boston: Kluwer Academic Publishers.www.southalabama.edu/coe/bset/johnson/660lectures/Lect2.doc
  9. Ellen Taylor-Powell, formerly of UWEX, has  written a series of “Quick Tips” on the Retrospective Pretest Design, which can be accessed here:

#27 – Using the Retrospective Post-then-Pre Design (2 pages, 178 KB) http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet27.pdf
#28 – Designing a Retrospective Post-then-Pre Question (2 pages, 183 KB) http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet28.pdf
#29 – When to Use the Retrospective Post-then-Pre Design (1 page, 177 KB) http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet28.pdf
#30 – Analysis of Retrospective Post-then-Pre Data (2 pages, 166 KB) http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet30.pdf

  1. The University of Wisconsin Extension Service has a series of “Quick Tips” on program evaluation, which can be accessed here: : http://extension.psu.edu/evaluation
  2.  Pennsylvania State University Extension service has a series of “Tipsheets” on program evaluation, which can be accessed here: http://extension.psu.edu/evaluation

QUANTITATIVE DATA MANAGEMENT AND ANALYSIS
Speaker: Marc Braverman, Ph.D.
http://breeze.wsu.edu/p7y22remyn1/

  1.  Analyzing quantitative data, Ellen Taylor-Powell. http://learningstore.uwex.edu/Assets/pdfs/G3658-06.pdf
  2. Using Excel for analyzing survey results, Jennifer Leahy.  http://learningstore.uwex.edu/Assets/pdfs/G3658-14.pdf
  3. Conduit, the quarterly newsletter for EvaluATE, the evaluation resource center for advanced technology education, discusses. statistical significance and the use of descriptive statistics   http://www.evalu-ate.org/resource_types/newsletter-pdfs/page/2/
  4. Data Analysis Basics: Variables and Distribution, https://nciph.sph.unc.edu/focus/vol3/issue5/3-5DataBasics_issue.pdf
  5. Data Analysis: Simple Statistical Tests  https://nciph.sph.unc.edu/focus/vol3/issue6/3-6DataTests_issue.pdf  (Offers discussion questions and slides.

QUALITATIVE DATA MANAGEMENT AND ANALYSIS
Speaker: Darlene Russ-Eft, Ph.D.
http://breeze.wsu.edu/p8e0bhb52az/

  1. Analyzing qualitative data, Ellen Taylor-Powell & Marcus Renne http://learningstore.uwex.edu/Assets/pdfs/G3658-12.pdf
  2.  Compiled by Donald Ratcliff, offers a list of 15 approaches to qualitative data analysis based largely  on the volume by, Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis, 2nd ed. Newbury Park, Cal.:Sage  http://www.psychsoma.co.za/files/15methods.pdf

OTHER EVALUATION RESOURCES

  1. Guidelines And Checklist For Constructivist (A.K.A. Fourth Generation) Evaluation, Egon G. Guba & Yvonna S. Lincoln.(Checklist for designing a qualitative study.) http://dmeforpeace.org/sites/default/files/Guba%20and%20Lincoln_Constructivist%20Evaluation.pdf
  2. W.K. Kellogg Foundation Logic Model Development Guide (Logic model development guide for foundations/non-profits) http://www.smartgivers.org/uploads/logicmodelguidepdf.pdf
  3. Developmental Evaluation:  Applying complexity concepts to enhance innovation and use.  Michael Quinn Patton.  2011.  Guilford Press. http://www.guilford.com/cgi-bin/cartscript.cgi?page=pr%2Fpatton.htm&dir=research%2Fres_eval&cart_id=611080.27904 
  1. SAMMIE
    SAMMIE stands for Successful Assessment Methods and Measurement In Evaluation and is a one stop site on the web to link you to valuable resources. Through SAMMIE you can – access resources on 22 evaluation-related topics; read the best literature on the web related to program evaluation; ask an expert your questions about program evaluation; and develop a personalized program evaluation plan. Whatever your level of experience evaluating programs, SAMMIE can be the web portal to help you expand your skills.
  2. American Evaluation Association (AEA)
    The American Evaluation Association is an international professional association of evaluators devoted to the application and exploration of program evaluation, personnel evaluation, technology, and many other forms of evaluation. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness.
  3. Evaluation and Accountability Resources (Southern Region)
    A set of compiled resources for program development and evaluation. This is a project of the Southern Region Extension Program and Staff Development Committee.
  4. Program Development and Evaluation, University of Wisconsin
    Information on Program Development and Evaluation through the University of Wisconsin. Program Development is an ongoing systematic process that UW-Extension professionals follow as they plan, implement and evaluate their educational programs. The program development process is captured in the UW-Extension program development model that includes situational analysis, priority setting, program action – the logic model – and evaluation.
  5. Program Evaluation, Penn State
    This site, hosted by Penn State College of Agriculture Sciences/ Cooperative Extension & Outreach provides information to design and implement a useful program evaluation to improve a program, compare delivery methods, respond to stakeholders and advocate. The site hosts a series of Tip Sheets that WSU Extension educators will find helpful. The site is maintained by Nancy Ellen Kiernan, Ph.D., Program Evaluator.
  6. On-Line LOGIC Model Course, University of Wisconsin Extension
    This site is an on-line course that introduces a holistic approach to planning and evaluating education and outreach programs.