Compare and Contrast the Three Main Types of Evaluation

Question

Examlex · Accepted Answer

The Answer is Process evaluation documents the implementation of an innovation (intervention program, policy, or practice). Process evaluation data are collected to monitor the activities that take place as an innovation is put into effect. In the health service delivery system, the innovation might be a treatment program, a clinic standing order, or a medical procedure. In the health service support system, the innovation could be a health education course for health care providers or a new quality improvement approach. In both the delivery and support system, a process evaluation can determine the extent to which an intervention was delivered as intended. It records the types of variations that occur and quantifies the magnitude of those variations. More than just auditing of services and outputs (a checklist approach to progress monitoring), when done well, a process evaluation helps to uncover implementation-related factors that may have facilitated or hindered the achievement of targeted outcomes. While a process evaluation is highly useful for understanding implementation issues, by design, it does not measure outcomes associated with an innovation. Additionally, the practical value of a process evaluation may be limited if data analyses do not occur until well after implementation. Therefore, this type of evaluation is almost always used in conjunction with other methods.
Summative evaluation is used to provide evidence about the worth or merit of an intervention and is typically conducted after an innovation has been implemented. It focuses on products, results, or impact. A summative evaluation, when used in conjunction with process evaluation data, can also address the bottom-line question of how well an intervention worked. Addressing these bottom-line questions requires that comprehensive process and outcome data of high quality are available.
Summative data are critical for accountability. However, there can be major challenges in using summative evaluation in health settings, particularly for evaluations pertaining to delivery system services. The complexity of the human condition makes it extremely challenging to isolate the effect of an intervention on an individual or population, given the multitude of factors that impact health and well-being. The types of evaluation designs that would allow one to examine such effects can be cumbersome, expensive, resource intensive, and in some cases, unethical (e.g., using a randomized control design that withholds a potential treatment from a group that could benefit from it).
Another significant consideration in health service settings is the time lag between health activities and health outcomes. We can measure what happened in a program, but may not be fully able to track whether or not the program ultimately made a difference in the target population without a significant investment in evaluation resources to track these changes over long periods of time. Potentially, it could be months or years before a program has its intended impact. Even in the interim, individual and community-level influences could moderate the effectiveness of what was developed and implemented.
Despite the limitations and challenges of summative evaluations, this type of evaluation bears its own merit. A summative evaluation answers the important question of "What actually works?," a critical input in promoting health outcomes and wellness. Using summative evaluation in many health service settings requires considering whether the goals of an initiative are feasible (i.e., Can these be achieved?), evaluable (i.e., Can we measure these?), and plausible (i.e., Is the attribution chain between intervention and outcomes realistic?).
A formative evaluation is designed to create information that can be used for program improvement. While a formative evaluation relies upon process evaluation data, it differs by objective. The aim of a formative evaluation is improvement, whereas the aim of a process evaluation is simply to monitor implementation activities. When done well, a formative evaluation illuminates areas and issues impacting outcomes as an intervention unfolds. A formative evaluation helps with identifying and addressing emergent challenges. As such, formative approaches are particularly useful for measuring and responding to the complexity of health service systems. Formative data can be collected and used by health service delivery and support systems to make evidence-informed decisions and mid-course corrections as needed. Formative evaluations can be used to complement process evaluations. For example, a process evaluation might first surface issues with implementation fidelity; then, a formative evaluation could be conducted with intervention stakeholders to identify opportunities for improvement.

Compare and Contrast the Three Main Types of Evaluation

Definitions: