Evidence-Based Interventions: Replicate or Adapt?

First posted on www.medicaring.org on June 14, 2011.

In a complex system such as transitions of sick and fragile patients from one setting to another, we are often so grateful for the few carefully done and reported research endeavors that funders and researchers easily fall into the trap of insisting upon slavish replication, assuming that this is the way to achieve the same results. If we were working with a highly standardized “system,” such as how heart cells respond to a drug, then we could reasonably assume that the curve of responses in Maine would be just about the same as the curve of responses in Arizona, and that what works for a dozen will work as well for a hundred. Sometimes, of course, even those assumptions are wrong, but it is rare for an unmeasured characteristic of the population to greatly alter drug effects or metabolism. 

However, there is every reason to assume that carefully done research on small numbers in a few settings will not be enough to guide practical implementation of process redesign. There are two main reasons for this. First, our paradigm for good studies is the randomized controlled trial (RCT), but some of its characteristics actually undercut the utility of the findings for guiding replication. Specifically, the effective restrictions (stated and unstated) for eligibility make it likely that only a small sub-set of actual patients will be eligible for the trial. Second, the fact that one is willing to randomize within one setting is good for blinded trials, but undercuts the galvanizing of the will that is often essential in fueling system reform. Consider this example – could you really generate the outrage that allows a nursing unit to make changes to stop repeated mistakes in transitions to stop the suffering of their discharged patients — and simultaneously be expected to continue to do it wrong for all but a few of the patients? 

Another challenge in the usual RCT is that the numbers affected are small — often only a small subset of the patients in the test site. While this works for a proof of concept, improvement experts quickly note that scaling up is never just a matter of applying the same changes to a lot more people! Instead, scaling up poses its own problems. As one scales up improvements in care transitions, one has to work on incorporating many elements of the work into job descriptions and job routines so that the workflow is smooth. One has to figure out fail-safe strategies, develop broad consensus in the community as to standards, train a populace to take a more active role in managing transitions for themselves and their loved ones, right-size the community’s supportive services, and a dozen additional elements. The research model is usually a discrete “add-on” patch to a dysfunctional system. 

Indeed, an RCT relies upon not changing the underlying dysfunctional system. As one tries to implement the improvement approach more broadly, efficiency dictates that it become part of the system wherever possible. Often, this also means that the highly skilled and motivated people involved in the research are replaced by less skilled, and, often, less motivated personnel providing routine services, with lower pay and more stresses. Adapting the work of a research nurse practitioner to a regular home care RN, or of a skilled professional to a retiree volunteer, is real work that takes testing, innovation, and creativity. In the work of the Quality Improvement Organizations (QIOs), for instance, as they implemented evidence-based interventions, many substantial adaptations were required. One team trained certain nurses in a home health agency to be the bridging nurses in an adaptation of Naylor’s model. One team used senior volunteers as trained coaches for patient activation in an adaptation of the Coleman model. I don’t believe that any of the 14 communities were able to implement a research-based intervention exactly as it had been done in the research report. The research was still quite important for laying down the path, but following the path with larger numbers in varied contexts required adaptations. 

Perhaps the most substantial challenge in our work is that small numbers do not threaten the hospitals’ overall patient flow, while broad implementation could cut into occupancy rates and cause serious financial problems, especially if done too quickly for the system to adapt and right-size its services. Scaling up requires considering the financial impact. The good news is that there are usually good reasons to absorb this impact, including the fact that most rehospitalizations and medical hospitalizations of Medicare patients do not make the hospital money, or at least not much money. 

Joanne Lynn, M.D. is the director of Altarum Institute’s Center for Elder Care and Advanced Illness which is researching and demonstrating new models to help an aging U.S. population live meaningfully and comfortably with serious chronic illnesses associated with advancing age, at a sustainable cost to our families and society.  

__________

As a leading nonprofit health care research and consulting institute dedicated to improving human health, Altarum encourages open discussion and debate about the many challenges in health care today. All postings to the Health Policy Forum (whether from employees or those outside the Institute) represent the views of the individual authors and/or organizations and do not necessarily represent the position, interests, strategy, or opinions of Altarum Institute. Altarum is a nonprofit, nonpartisan organization. No posting should be considered an endorsement by Altarum of individual candidates, political parties, opinions, or policy positions. Read more.