- Open Access
The best evidence for alcohol screening and brief intervention in primary care supports efficacy, at best, not effectiveness: You say tomāto, I say tomăto? That’s not all it’s about
Addiction Science & Clinical Practice volume 9, Article number: 14 (2014)
The review related to this manuscript is available at http://www.ascpjournal.org/content/9/1/13.
Several meta-analyses find that patients with nondependent, unhealthy, alcohol use (identified by screening) who are randomized to receive brief counseling subsequently report drinking less than those randomized to control groups [1–4]. Most studies do not have biological confirmation, and those that do generally do not find effects on such outcomes, nor on any hard outcomes (though some do ). The lack of evidence for effects on these outcomes raises the possibility that alcohol screening and brief intervention efficacy remains unknown, since the modest effects on drinking could easily be due to social desirability bias  (i.e., patients are told they are in a study of counseling to reduce their drinking, they are randomly assigned to brief counseling with that goal, and then they are asked whether they reduced their drinking, so they may therefore report less drinking to please the interviewer; this would be expected to occur more often in an intervention than in a control group). This possibility should be resolved by further study (i.e., randomized trials with biological outcomes).
But if we take the positive trial findings as being valid—and many scientists and practitioners have —the question is whether they represent efficacy or effectiveness. This question is important because if alcohol screening and brief intervention (ASBI) has efficacy, the next steps would be to determine how to implement it and retain effectiveness in real-world practice. But if the evidence supports effectiveness, little further study would be needed, and efforts would turn to dissemination and implementation.
In Addiction Science & Clinical Practice, Heather  makes the case that extant trials suggest that ASBI is effective. His case relies largely on two arguments. First and foremost, Heather observes that based on an effectiveness scale, randomized trials of ASBI can be characterized mainly as effectiveness studies. He does concede that the scale may have minor, and mainly methodological, limitations. But I believe the scale to be highly flawed for evaluating whether a trial of a preventive service like ASBI is an effectiveness or an efficacy study.
The scale as reported in Kaner et al.  emphasizes patients, practitioners, and intervention content. Such a scale could be useful for describing studies of lengthy interventions delivered by specialists developed for selected patients. Then, when such an intervention is somehow adapted for use in less selected patients to be delivered by generalists in general care settings, the trial will be appropriately characterized as an effectiveness study. But this conceptual approach does not apply to ASBI. ASBI was designed and developed as a brief service for all patients (i.e., unselected) to be delivered by practitioners who are not highly specialized, in general care settings. So even the most tightly controlled efficacy study would score as an effectiveness study on the scale.
More specifically, ASBI is designed for all primary care patients and starts with universal screening. It would not make sense to test a universal preventive service in selected or referred patients in an alcohol specialist clinic. If one did so, it would no longer be ASBI. By definition, any study of ASBI will score as effectiveness on “patients and problems” and “practice context”. Similarly any study of “brief” intervention will score high for effectiveness on “intervention content” because the timing could fit with general practice (e.g., 15 minutes). To be characterized as an efficacy study, the intervention would need to be much longer, but then of course it would no longer be brief intervention; it would be something else. ASBI based on personalized feedback and/or motivational interviewing is by design therapeutically flexible; so again, it will score high for effectiveness (by definition) since flexibility is characteristic of ASBI.
Thus, because of the nature of ASBI, just about any study of it would be characterized as an effectiveness study by the scale, and the only variability might be whether the ASBI or some component of it was performed by research staff, and whether it was monitored or supported by research staff. On these features, many ASBI studies in fact do look like efficacy studies. Practitioner training was usually implemented and orchestrated by researchers championing the effort. ASBI would not have occurred in the studies were it not for substantial researcher involvement. For example, researchers often performed the screening and provided the results to those conducting the BI. Investigators have been generally available to support ongoing ASBI implementation in these studies. And studies often monitored and assured fidelity of the BI. Although ASBI studies often begin with universal screening, many patients with unhealthy drinking are excluded for clinical and research reasons, such as drinking too little or too much, having alcohol dependence , having significant psychiatric or medical comorbidity, or for research reasons, such as being difficult to follow up, in addition to the fact that many simply choose not to participate in the trials.
The fact that ASBI trials have been conducted in primary care, often by primary care practitioners as interventionists in initially unselected patients, should not lead us to conclude that they are effectiveness trials, even if a scale would count them as such for those reasons. They are still studies that were controlled by researchers much more than usual interventions would be in routine clinical practice.
Evidence for the case that these have been largely efficacy studies is that two of the studies designed as effectiveness studies found no intervention effects [9–11]. They differed primarily from the rest of the ASBI trial literature in that the researchers had less control over intervention implementation (though even in the SIPS trial  when practitioners wouldn’t do the BI, researchers stepped in), fidelity, support, monitoring, and patient selection, yet they were similar to the rest of the literature in that they took place in primary care settings and used a flexible intervention delivered by real practitioners. These two studies clearly were effectiveness studies, very different from most other ASBI trials, yet they would score similarly on the effectiveness scale.
The second argument made by Heather  and Flay  is that in treatment effectiveness studies, the available intervention is optimized but acceptability could be variable, whereas the latter would be optimized in efficacy studies. But for ASBI, the distinction between these two (availability and acceptability) is minimal. Most ASBI intervention studies make BI “available in uniform fashion, within standardized contexts/setting, to a specified target group” (characteristic of efficacy studies according to Heather ). The distinguishing feature for Heather is whether or not it is delivered in a “manner acceptable to them [the patients]”. But once one has a patient who has agreed to be in a trial of an alcohol BI in primary care, it is likely that it will be acceptable (they will participate in the brief counseling), particularly if the intervention is flexible and provided on the spot, which ASBI usually is by design. ASBI as a service conceptualized and designed for primary care is brief, delivered in primary care at the time of the visit.
In sum, because of the very nature of the intervention, the only way to test ASBI is in studies that would have the characteristics of effectiveness studies according to the scale used by Kaner et al. . I suggest that despite this, there are meaningful distinctions that should be made in characterizing these studies on the spectrum of efficacy versus effectiveness that are not captured by the scale for this preventive intervention. They are distinctions that primary care clinicians would instantly recognize—e.g., having special staff for training and implementation who are not usually in their practice taking steps to monitor and support fidelity, and having patients identified by research staff who prompt an intervention. Just because the study is completed in primary care and the intervention designed for that setting is (by definition) acceptable, does not make it an effectiveness study. Or, if we wish to say that based on the scale, they are effectiveness studies, then I suggest that for ASBI the distinction between effectiveness and efficacy studies is not meaningful. We then need another kind of study—a real-world  “does it work on a wet Wednesday in Wigan” study, which most ASBI studies are not.
If the current literature were truly replete with real-world ASBI effectiveness studies, then we would likely see ASBI being widely disseminated (it is not), and implemented successfully. Yet in one of the only large health systems with widespread ASBI implementation, the benefit of ASBI is undetectable . Regardless of whether we agree to call ASBI studies effectiveness research (you say tomāto, I say tomăto), the bottom line is that the current literature is not very informative about whether ASBI works under real-world conditions.
Jonas DE, Garbutt JC, Amick HR, Brown JM, Brownley KA, Council CL, Viera AJ, Wilkins TM, Schwartz CJ, Richmond EM, Yeatts J, Evans TS, Wood SD, Harris RP: Behavioral counseling after screening for alcohol misuse in primary care: a systematic review and meta-analysis for the U.S. preventive services task force. Ann Intern Med. 2012, 157: 645-654. 10.7326/0003-4819-157-9-201211060-00544.
O’Donnell A, Anderson P, Newbury-Birch D, Schulte B, Schmidt C, Reimer J, Kaner E: The impact of brief alcohol interventions in primary healthcare: a systematic review of reviews. Alcohol Alcohol. 2014, 49: 66-78. 10.1093/alcalc/agt170.
Kaner EF, Dickinson HO, Beyer F, Pienaar E, Schlesinger C, Campbell F, Saunders JB, Burnand B, Heather N: The effectiveness of brief alcohol interventions in primary care settings: a systematic review. Drug Alcohol Rev. 2009, 28: 301-323. 10.1111/j.1465-3362.2009.00071.x.
Kaner EF, Dickinson HO, Beyer FR, Campbell F, Schlesinger C, Heather N, Saunders JB, Burnand B, Pienaar ED: Effectiveness of brief alcohol interventions in primary care populations. Cochrane Database Syst Rev. 2007, 18 (2): CD004148. doi:10.1002/14651858.CD004148.pub3
Crowne DP, Marlowe D: A new scale of social desirability independent of psychopathology. J Consult Psychol. 1960, 24: 349-354.
Moyer VA, on behalf of the U.S. Preventive Services Task Force: Screening and behavioral counseling interventions in primary care to reduce alcohol misuse: U.S. preventive services task force recommendation statement. Ann Intern Med. 2013, 159: 210-218.
Heather N: The efficacy-effectiveness distinction in trials of alcohol brief intervention. Addict Sci Clin Pract. 2014, 9: 13-10.1186/1940-0640-9-13.
Saitz R: Alcohol screening and brief intervention in primary care: absence of evidence for efficacy in people with dependence or very heavy drinking. Drug Alcohol Rev. 2010, 29: 631-640. 10.1111/j.1465-3362.2010.00217.x.
Kaner E, Bland M, Cassidy P, Coulton S, Dale V, Deluca P, Gilvarry E, Godfrey C, Heather N, Myles J, Newbury-Birch D, Oyefeso A, Parrott S, Perryman K, Phillips T, Shepherd J, Drummond C: Effectiveness of screening and brief alcohol intervention in primary care (SIPS trial): pragmatic cluster randomised controlled trial. BMJ. 2013, 346: e8501-10.1136/bmj.e8501.
Hilbink M, Voerman G, van Beurden I, Penninx B, Laurant M: A randomized controlled trial of a tailored primary care program to reverse excessive alcohol consumption. J Am Board Fam Med. 2012, 25: 712-722. 10.3122/jabfm.2012.05.120070.
van Beurden I, Anderson P, Akkermans RP, Grol RP, Wensing M, Laurant MG: Involvement of general practitioners in managing alcohol problems: a randomised controlled trial of a tailored improvement programme. Addiction. 2012, 107: 1601-1611. 10.1111/j.1360-0443.2012.03868.x.
Flay BR: Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med. 1986, 15: 451-474. 10.1016/0091-7435(86)90024-1.
Williams EC, Rubinsky AD, Chavez LJ, Lapham GT, Rittmueller SE, Achtmeyer CE, Bradley KA: An early evaluation of implementation of brief intervention for unhealthy alcohol use in the US veterans health administration. Addiction. 2014, 109: 1472-1481. 10.1111/add.12600.http://dx.doi.org/10.1111/add.12600,
About this article
Cite this article
Saitz, R. The best evidence for alcohol screening and brief intervention in primary care supports efficacy, at best, not effectiveness: You say tomāto, I say tomăto? That’s not all it’s about. Addict Sci Clin Pract 9, 14 (2014) doi:10.1186/1940-0640-9-14