Providing access to a personalized feedback intervention website does not appear to have an impact on college student drinking. The design of this study varied from other randomized controlled trials evaluating the efficacy of personalized feedback interventions because most other trials either automatically provided the feedback after participants recorded their baseline data or ensured that the participant received the feedback through some other means. In order to test the effectiveness of a web-based intervention when participation is voluntary, it is necessary to set up the evaluation trial so that the use of the intervention is voluntary (in this case by recruiting for a survey about drinking and offering a randomized half access to a website that would let them compare their drinking to other university students if they were interested). In addition, compensation for participating was minimal. When access is voluntary, only a minority of participants (less than 20%) who were offered access reported actually using the intervention, thus obviating the potential mean group effect for the intervention condition (i.e., those provided access to the intervention).
There were several limitations to this trial. The first is that there is no previous research evaluating whether the CYDU has an impact in a situation where all participants in the intervention condition actually receive the intervention. Thus, it is possible that the intervention itself is not effective. However, we judge this possibility to be unlikely as the CYDU is very similar to the general population version (the Check Your Drinking screener) which has been subjected to four randomized trials to-date including two with college students[20–23]. However, there is merit in conducting an efficacy trial with the CYDU in order to rule out this possibility. In addition, it is possible that the CYDU has an impact but that there is something about the way the CYDU was presented (or with the introductory page of the CYDU) that acted as a barrier to participants actually completing this web-based intervention. In the present study, 128 participants were redirected to the CYDU. However, only 37 participants actually reported completing the CYDU. Finally, some participants (27%, 115/425) were responding to the baseline survey using a mobile phone platform making it possible that they would be less likely to complete the CYDU which has been set up to be completed in a computer-based environment (thus contributing to the low completion rate).
The other primary limitation was that the study was underpowered. Based on one of the authors experience in other research trials (and from reports of the prevalence of problem drinking on university campuses), we estimated that sending email invitations to 10,000 college students would be sufficient to garner 2,000 participants completing the baseline survey. Of these, we estimated that 1,200 might be risky drinkers and that 1,000 would agree to participate in the trial. With a hoped for follow-up rate of 80%, we powered the study to detect a small difference on the AUDIT-C. In the current study, 10,000 email invitations were sent out, 1,768 responded and only 450 met criteria for risky drinking leading to a much lower than anticipated sample size. There are a number of possibilities for this low response rate. The first is that email invitations with links to websites might be met with growing suspicion given the possibility of computer viruses. Another possibility is that the incentive to participate was too small ($5 certificate). Still, the proportion of participants to the baseline survey was not that much lower than anticipated but rather the proportion of those who participated who were risky drinkers was overestimated. This may reflect the fact that the university campus under study had a substantial commuter population and comprised of a diverse population with a relatively low proportion of students who drank alcohol. Subsequent investigation and communication with campus administrators has confirmed that drinking rates on this campus are considerably lower than average. This limitation points to the need to evaluate web-based personalized feedback interventions in a variety of different college settings. Irrespective of the smaller than anticipated sample size, the results were clear enough to establish that, if there was any difference between the groups at all, it was very minor and would not have reached statistical significance without a much larger sample size. A post hoc power calculation (80% power to detect a difference with an alpha of .05) revealed that roughly 2,400 participants per condition would be needed to detect the difference observed in this trial.
While no overall effects were observed in this trial, it is likely that CYDU or similar web-based feedback interventions might be most effective among specific subgroups of students. The present study included an extremely minimal assessment which precluded our ability to evaluate potential moderators. Future research might consider whether voluntary access to web-based feedback interventions might be most effective for students who view their drinking as problematic and/or who are considering changing their drinking.
Despite limitations, the negative results observed in this trial establishes the need to conduct more pragmatic trials of the potential real-world influence of web-based personalized feedback interventions before we can confidently make the claim that these interventions will have an impact on problem drinking in college students when these interventions are offered in a voluntary participation manner.