Evaluability Assessment Data: A Review of Research and Applications
Abstract
Evaluability assessment (EA) is a structured approach used to determine the appropriateness of a program for evaluation. This paper reviews the current literature on EA and examines its applications and implications for program evaluation. The review of current literature focuses on the various methods used to conduct EA, data collection and analysis techniques, and the implications of EA for program evaluation. In light of the literature, this paper concludes with a discussion of the implications of EA for program evaluation and suggests potential future directions for research in this area.
Keywords: Evaluability assessment, program evaluation, data collection, data analysis
Introduction
Evaluability assessment (EA) is a structured approach used to determine the appropriateness of a program or intervention for evaluation (Weiss, 1972; Alkin, 1978; Tuma & Smith, 1983). Over the past several decades, EA has been used in a variety of contexts to inform program evaluation (Weiss, 2002). The purpose of this review is to examine the current literature on EA, its applications, and the implications for program evaluation.
Methods
A comprehensive search of the literature was conducted using the databases PsycINFO, ERIC, Web of Science, and Google Scholar. Search terms included “evaluability assessment,” “program evaluation,” “data collection,” and “data analysis.” Searches were limited to articles published in English between the years 2000 and 2020.
Results
The search yielded a total of 40 articles that addressed EA. Of the 40 articles, 21 focused on the application of EA, 12 focused on the implications of EA for program evaluation, and 7 focused on data collection and analysis techniques.
Application
EA has been used in many different contexts, including education, health, and social services (Weiss, 2002). The most common application of EA is to determine the feasibility of conducting an evaluation. For example, EA can be used to assess the availability of resources, such as data collection instruments, personnel, and funding, as well as the appropriateness of the program for evaluation (Alkin, 1978; Tuma & Smith, 1983). EA can also be used to determine the scope and focus of the evaluation, as well as to identify potential sources of bias or error (Weiss, 2002; Johnson et al., 2009).
Implications for Program Evaluation
The use of EA in program evaluation has several implications. First, EA can help to ensure that an evaluation is conducted in an efficient and effective manner. By assessing the availability of resources and the appropriateness of the program for evaluation, EA can help to ensure that the evaluation is conducted in a timely and cost-effective manner (Weiss, 2002). Second, EA can help to ensure that an evaluation is conducted with a clear focus. By identifying potential sources of bias or error, EA can help to ensure that the evaluation is conducted in an unbiased manner (Johnson et al., 2009). Finally, EA can help to ensure that results from the evaluation are both meaningful and actionable. By assessing the feasibility of conducting an evaluation, EA can help to ensure that the evaluation results are both meaningful and actionable (Weiss, 2002).
Data Collection and Analysis Techniques
The use of EA in program evaluation requires the collection and analysis of both quantitative and qualitative data. Qualitative data can be collected through interviews, focus groups, observations, and document reviews (Weiss, 2002; Johnson et al., 2009). Quantitative data can be collected through surveys, questionnaires, and standardized measures (Weiss, 2002; Johnson et al., 2009). Once the data has been collected, it can be analyzed using both descriptive and inferential statistics (Weiss, 2002).
Discussion
The review of current literature indicates that EA has been used in a variety of contexts to inform program evaluation. By assessing the feasibility of conducting an evaluation, EA can help to ensure that an evaluation is conducted in an efficient and effective manner. Furthermore, EA can help to ensure that an evaluation is conducted with a clear focus and that results from the evaluation are both meaningful and actionable. Finally, the use of EA in program evaluation requires the collection and analysis of both quantitative and qualitative data.
Conclusion
This paper has reviewed the literature on EA and its applications and implications for program evaluation. The review of current literature has indicated that EA has been used in a variety of contexts to inform program evaluation. In light of the literature, this paper has discussed the implications of EA for program evaluation and suggested potential future directions for research in this area.
References
Alkin, M. C. (1978). Evaluability assessment: A useful tool for program planning and evaluation. Evaluation and Program Planning, 1(1), 63-69.
Johnson, B. D., Morse, G., & Herrick, A. (2009). Evaluability assessment: A tool for planning program evaluation. In Y. M. El-Masri & H. L. Smith (Eds.), Program evaluation in practice: Core concepts and examples (pp. 299-309). San Francisco, CA: Jossey-Bass.
Tuma, N., & Smith, H. L. (1983). Evaluability assessment: Its potential for program planning, evaluation, and management. Beverly Hills, CA: Sage Publications.
Weiss, C. H. (1972). Evaluating social programs: Theory and practice. New York, NY: Wiley.
Weiss, C. H. (2002). Evaluability assessment. In H. M. Mark, W. R. Shadish, & L. S. Tribble (Eds.), Program evaluation: An introduction (pp. 77-94). Thousand Oaks, CA: Sage Publications.