When the best is the enemy of the good: The Nature of Research Evidence Used in Systematic Reviews and Guidelines

This paper, written by Marcel P. J .M. Dijkers for the Task Force on Systematic Review and Guidelines, examines the approach in evidence-based practice of using only the "best available" evidence and argues that this strongly disadvantages rehabilitation programs because of their nature. He goes on to state that the focus on the use of only the 'strongest' research designs shifts the focus away from researchers asking 'What is the best design to answer this research question?'


"Unfortunately, many systematic reviews and guidelines published in recent years have adopted an all-or-nothing approach to the evidence base. Cochrane review groups may be the most extreme; in many instances only evidence for therapeutic interventions resulting from randomized clinical trials (RCTs) is accepted. If that level of evidence is lacking, "more research" is recommended, and no recommendations for practice are made. Other groups follow a similar practice, although they may draw the line at a different level in the evidence hierarchy. For instance, AAN guidelines specify that no recommendation should be made if there is not at least one Class II study or two consistent Class III studies, and that the recommendation to be made when this minimum level of evidence is available is to be phrased in terms of "may be considered" or "may not be considered" as appropriate (Edlund et al., 2004).

When a well-respected statistician-methodologist like Douglas Altman goes on record stating,

Only randomised trials allow valid inferences of cause and effect. Only randomised trials have the potential directly to affect patient care — occasionally as single trials but more often as the body of evidence from several trials, whether or not combined formally by meta-analysis (Altman, 1996, p. 570)

it is not surprising that the misunderstanding spreads in EBP circles that only RCTs can contribute information that is of use in clinical decision making. This is also reflected in the following: "Treatment decisions in clinical cardiology are directed by results from randomized clinical trials (RCTs)" (Hernandez, Boersma, Murray, Habbema, & Steyerberg, 2006, p. 257)."


Dijkers, M. P. J. M. for the NCDDR Task Force on Systematic Review and Guidelines. (2009). When the best is the enemy of the good: The nature of research evidence used in systematic reviews and guidelines. Austin, TX: SEDL. Retrieved from: http://ktdrr.org/ktlibrary/articles_pubs/ncddrwork/tfsr_best/