Your organization has decided to try to change the way primary care practices deliver care, in the hope of improving important outcomes. The first question to ask is whether you should evaluate the intervention.
Not all primary care interventions require an evaluation. When it is clear that a change needs to be made, the practice may simply move to adoption. For example, if patients are giving feedback about lack of evening hours, and business is being lost to urgent care centers, then a primary care practice might decide to add evening hours without evaluating the change. You may still want to track utilization and patient feedback about access, but a full evaluation of the intervention may not be warranted. In addition, some operational and quality improvement changes can be assessed through locally managed Plan-Do-Study-Act cycles. Examples of such changes include changing appointment lengths and enhancing educational materials for patients. Finally, when previously published studies have provided conclusive evidence in similar settings with similar populations, you do not need to re-test those interventions.
A more rigorous evaluation may be beneficial if it is costly to adopt the primary care intervention and if your organization is considering whether to spread the intervention extensively. An evaluation will help you learn as much as possible about how best to implement the intervention and how it might affect outcomes. You can examine whether it is possible for practices to make the changes you want, how to roll out this (or a refined intervention) more smoothly, and whether the changes made through the intervention sufficiently improve outcomes to justify the effort. You also may be able to ascertain how outcomes varied by practice, market, and patient characteristics. Outcomes of interest typically include health care cost and quality, and patient, clinician, and staff experience. Results from the implementation and impact analyses can help make a case for refining the intervention, continuing to fund it, and/or spreading it to more practices, if the effects of the intervention compare favorably to its costs.
Figure 1 summarizes the steps involved in planning and implementing an evaluation of a primary care intervention; the two boxes on the right-hand side show the evaluation's benefits.
Figure 1. Steps in Planning and Implementing an Evaluation of a Primary Care Intervention
Common Challenges in Evaluating Primary Care Interventions
Timeframes are too short or intervention too minor to observe changes in care delivery and outcomes. Small numbers of practices make it hard to detect effects statistically due to clustering.
Data are limited, of poor quality, or have a significant time lag.
Results are not generalizable because practices participating in intervention are different from other practices (e.g., participants may be early adopters).
Outcomes may improve or decline for reasons other than participation in the intervention and the comparison group or evaluation design may not adequately account for this.
Differences exist between intervention practices and comparison practices even before the intervention begins. Comparison practices get some form or level of intervention.
