Estimating the impact of stimulant use on initiation of buprenorphine and extended-release naltrexone in two clinical trials and real-world populations
Addiction Science & Clinical Practice volume 18, Article number: 11 (2023)
Co-use of stimulants and opioids is rapidly increasing. Randomized clinical trials (RCTs) have established the efficacy of medications for opioid use disorder (MOUD), but stimulant use may decrease the likelihood of initiating MOUD treatment. Furthermore, trial participants may not represent “real-world” populations who would benefit from treatment.
We conducted a two-stage analysis. First, associations between stimulant use (time-varying urine drug screens for cocaine, methamphetamine, or amphetamines) and initiation of buprenorphine or extended-release naltrexone (XR-NTX) were estimated across two RCTs (CTN-0051 X:BOT and CTN-0067 CHOICES) using adjusted Cox regression models. Second, results were generalized to three target populations who would benefit from MOUD: Housed adults identifying the need for OUD treatment, as characterized by the National Survey on Drug Use and Health (NSDUH); adults entering OUD treatment, as characterized by Treatment Episodes Dataset (TEDS); and adults living in rural regions of the U.S. with high rates of injection drug use, as characterized by the Rural Opioids Initiative (ROI). Generalizability analyses adjusted for differences in demographic characteristics, substance use, housing status, and depression between RCT and target populations using inverse probability of selection weighting.
Analyses included 673 clinical trial participants, 139 NSDUH respondents (weighted to represent 661,650 people), 71,751 TEDS treatment episodes, and 1,933 ROI participants. The majority were aged 30–49 years, male, and non-Hispanic White. In RCTs, stimulant use reduced the likelihood of MOUD initiation by 32% (adjusted HR [aHR] = 0.68, 95% CI 0.49–0.94, p = 0.019). Stimulant use associations were slightly attenuated and non-significant among housed adults needing treatment (25% reduction, aHR = 0.75, 0.48–1.18, p = 0.215) and adults entering OUD treatment (28% reduction, aHR = 0.72, 0.51–1.01, p = 0.061). The association was more pronounced, but still non-significant among rural people injecting drugs (39% reduction, aHR = 0.61, 0.35–1.06, p = 0.081). Stimulant use had a larger negative impact on XR-NTX initiation compared to buprenorphine, especially in the rural population (76% reduction, aHR = 0.24, 0.08–0.69, p = 0.008).
Stimulant use is a barrier to buprenorphine or XR-NTX initiation in clinical trials and real-world populations that would benefit from OUD treatment. Interventions to address stimulant use among patients with OUD are urgently needed, especially among rural people injecting drugs, who already suffer from limited access to MOUD.
Concomitant use of opioids and stimulants, primarily methamphetamine and cocaine, increased by over 80% in the past decade [1,2,3], contributing to an overdose epidemic that killed over 100,000 Americans in 2021 . While overdose deaths due to prescription opioids and heroin are declining, use of and mortality from highly potent synthetic drugs such as fentanyl and methamphetamine (MA) are steeply rising [4,5,6]. Co-use of opioids and cocaine, especially co-injection (“speedballs”), are also associated with increased risk of overdose . Medications for opioid use disorder (MOUD), including methadone, buprenorphine, and extended-release naltrexone (XR-NTX), are effective treatments for reducing opioid use, achieving remission from OUD, preventing overdose, and saving lives [8,9,10].
However, concomitant stimulant use may reduce rates of MOUD initiation [11, 12], inpatient and outpatient opioid treatment engagement, and retention in treatment . Stimulant use may also be associated with decreased opioid abstinence during MOUD treatment, although results are mixed and may depend on type of stimulant [13,14,15]. In addition, most of these data come from retrospective reviews of medical records, which have low sensitivity to detect stimulant use , or studies with small samples and substantial confounding concerns . As the prevalence of co-use of opioids with MA or cocaine increases [3, 17], overdose deaths involving stimulants and opioids follow. Methodologically robust, novel strategies to understand the role of stimulant use in opioid treatment initiation, engagement, and outcomes remain urgently needed .
Randomized clinical trials (RCTs) typically collect extensive covariate data that investigators can use to better control bias (i.e., increase internal validity) when investigating nonrandomized secondary questions. However, RCT samples are usually not representative of the larger populations intended to receive treatments under study , threatening external validity, and RCTs of treatments for substance use disorders (SUDs) are no exception . SUD treatment trial samples tend to differ from real-world SUD patient populations on age , sex [21, 22], race [21, 22], education , income [21, 24, 25], and SUD severity [21, 23,24,25,26], among other dimensions. People living in rural areas, who already suffer from limited access to medical care including SUD treatment , are also underrepresented in RCTs . Further, trials may limit enrollment to participants with a single SUD or fail to recruit representative patients with multiple SUDs, limiting the ability to answer questions about typical clinical patients who use multiple substances . Conversely, large-scale “real-world” data sources may be representative, but often lack the data elements, validity, or specificity to identify and estimate unbiased exposure effects . In particular, few SUD RCTs are implemented to test the real-world effectiveness of SUD treatment. The National Drug Abuse Treatment Clinical Trials Network (CTN) is the only government sponsored entity in the US that carries out large-scale pragmatic SUD treatment clinical trials, and the large datasets collected within the CTN contain standardized data elements and similarities in design that presented an opportunity to study SUD treatment in a broader context.
Transportability analysis is a novel analytic solution to this problem, formally defining when and how statistical estimates may be extrapolated from a given source population to a different target population [31,32,33]. When transportability analysis is conducted with nested populations, i.e., the source population is a subset of the target, it is called generalizability analysis. In a generalizability analysis, detailed exposure, outcome, and covariate data from an RCT can be fused with broad, representative data sources to support the projection of the exposure effect from the RCT onto a real-world target population of interest. The theoretical framework and assumptions underlying transportability/generalizability analysis are well-described [31,32,33]. We use generalizability analysis to estimate the association between stimulant use and MOUD initiation in three real-world populations of interest. This technique allows us to leverage the strengths of narrowly focused, precise clinical trials and broad, widely representative datasets to answer questions each data source independently cannot.
The primary aim of this study is to estimate the association between stimulant use and initiation of buprenorphine or XR-NTX for the treatment of OUD. To complete this aim, we conducted a two-part analysis. First, we estimated associations in pooled data of two NIDA-CTN clinical trials. Second, we generalized results to three target populations of interest: (1) civilian, noninstitutionalized, housed adults identifying the need for OUD treatment, as characterized by the National Survey on Drug Use and Health (NSDUH); (2) adults entering OUD treatment, as characterized by Treatment Episodes Dataset (TEDS); and (3) adults with opioid misuse and high rates of injection drug use (IDU) living in rural regions of the U.S., as characterized by the Rural Opioids Initiative cohort (ROI). Figure 1 provides a visual depiction of our study aims and procedures. We hypothesize that (1) stimulant use reduces the likelihood of initiating MOUD in all populations, and (2) the magnitude of this reduction is larger in real-world populations of adults with opioid misuse than in highly-selected trial participants.
Data sources and populations
The source (or study) population for our analyses was the pooled study sample of participants in two multisite CTN treatment trials: 0051 (X:BOT ; 2014–2017) and 0067 (CHOICES ; 2018–2019). Information on each study, including inclusion/exclusion criteria and recruitment and enrollment data are presented in Additional file 1: Table S1. Briefly, X:BOT randomized participants to XR-NTX vs. sublingual buprenorphine in inpatient, medically monitored opioid treatment facilities. CHOICES randomized to XR-NTX vs. treatment as usual in outpatient HIV clinics. However, 93% of treatment-as-usual participants who initiated treatment received buprenorphine; to create a more homogeneous pooled source population, eight CHOICES participants receiving methadone and three receiving only oral naltrexone were excluded. All X:BOT participants were included (corresponding to the original intent-to-treat sample). In both studies, participants provided urine drug screen (UDS) and harmonized questionnaire, and clinical, pharmacy, or laboratory data weekly (X:BOT) or every 4 weeks (CHOICES) for 24 weeks.
We defined three target populations for our analyses. The first is a nationally-representative sample of civilian, housed, noninstitutionalized adults with OUD identifying the need for treatment, as characterized by the NSDUH household 2018 and 2019 survey years . Among all NSDUH respondents, we included those who had OUD, were age 18 or older, and (1) received treatment in the past year, but used nonprescribed opioids in the past month or (2) identified the need for, but had not received, treatment. The second target population was people receiving or planning to receive MOUD treatment, as captured in the 2018 TEDS admissions dataset . Treatment episodes were included if the individual entering treatment met diagnostic criteria for OUD based off the Diagnostic and Statistical Manual of Mental Disorder—5th edition (DSM-5), used heroin or other illicit opioids in the past month, was 18 or older, and had MOUD as part of their intended treatment plan. The third target population was people who inject drugs or misuse opioids in rural areas of the U.S. as characterized by eight sites participating in the ROI . Briefly, the ROI is a consortium of harmonized studies of people who use drugs in rural counties with high overdose rates covering ten states (Illinois, Kentucky, North Carolina, New England [Massachusetts, New Hampshire, Vermont], Ohio, Oregon, West Virginia, Wisconsin). Inclusion criteria varied slightly by study site, but generally, people were eligible for the ROI cohort if they reported IDU or use of opioids to “get high” in the past month. ROI participants were included in this analysis if they were 18 or older and used opioids to get high in the past month. In order to meet positivity assumptions (i.e., to avoid generalizing to people who are strictly excluded from the source population of clinical trials), we also restricted all three target populations to individuals who were English-speaking, not currently pregnant, and without suicidal ideation.
Exposure, outcome, and covariates
The primary exposure of interest, stimulant use, was defined according to UDS positivity for MA, other amphetamines, or cocaine in clinical trial data. UDS were collected with an FDA-approved one-step temperature-sensitive test cup; a further validity check was performed using a commercially available adulterant test strip. Stimulants are detectable in urine for approximately 3 days to 1 week, depending on dose, route of administration, and type of stimulant . Participants provided UDS samples weekly (X:BOT) or every 4 weeks (CHOICES) and stimulant use was treated as time-varying.
The outcome was time from randomization to MOUD initiation, defined as the date of first prescription of buprenorphine or injection of XR-NTX. Participants who did not initiate MOUD were censored on the last day of study participation.
Demographics, use of other substances, housing status, and depression history from both clinical trial and target population data were included in analyses as covariates as well as in generalizability formulas. Some covariates were measured in different ways across datasets. “Heavy alcohol/benzodiazepine use” was defined as DSM-5 use disorders in the clinical trials and the NSDUH. In TEDS, heavy use was defined by the appearance of benzodiazepines or alcohol on the list of reasons for entering treatment. In the ROI, use of benzodiazepine or alcohol 15 or more days in the past month was classified as heavy use. Injection drug use was past 30 days in clinical trials, TEDS, and ROI, and the past year in NSDUH. Experience of recent homelessness was the past month in the CTN trials, past 6 months in the ROI, current in TEDS, and not measured in the NSDUH household survey. Depression history was not measured in TEDS or ROI. Some covariate data was missing in all sources: 6% of cases from the clinical trials contained missing data, 2% of NSDUH surveys, 9% of TEDS episodes, and 3% of ROI surveys. Missing covariate data were multiply imputed using chained equations .
Creation of a selection diagram
Prior to analyses, a causal “selection diagram” (Additional file 1: Fig. S1) was created by the study team using a combination of subject-matter expertise, protocols, previous literature, and data. Selection diagrams assist investigators who conduct transportability analyses in identifying variables that modify the effect of the exposure (stimulant use) and differ in distribution between source and target populations. All such variables must be included in transportability analyses to identify valid transportable exposure effects . In addition, in an observational setting (i.e., stimulant use is not randomized), the selection diagram is used to identify important covariates to include in outcome models. Trial protocols informed the selection diagrams by explicitly outlining inclusion and exclusion criteria for each trial. Subject matter expertise and prior literature were used to identify determinants of study outcomes (e.g., factors that influence or are associated with MOUD initiation other than through stimulant use) and thus potential key variables that may modify the effect of stimulant use on the outcome. Authors with expertise in addiction medicine reviewed selection diagrams to assess their completeness and appropriateness. To describe differences between CTN trial samples and the three target populations, we calculated standardized mean differences between CTN trial samples and each target population for all variables included in the transportability analyses.
Stage 1: estimation of stimulant use associations in the clinical trials
Among participants in the clinical trials, cumulative incidences of MOUD initiation by baseline stimulant use were described using Kaplan–Meier curves. Cox proportional hazards models were used to estimate associations between time-varying stimulant use and initiation on MOUD. Analyses controlled for CTN trial (X:BOT vs. CHOICES), treatment arm (buprenorphine vs. XR-NTX), age, sex, race, education, employment, depression history, past 30-day homelessness, alcohol use disorder, past 30-day IDU, and time-varying opioid and benzodiazepine use (we focused on benzodiazepines specifically owing to their increasing co-involvement in opioid overdose deaths ). Analyses were repeated, stratified by type of MOUD (i.e., buprenorphine and XR-NTX), controlling for the same covariate set. To be able to continue to stage 2 and generalize results from these pooled clinical trials conducted in heterogeneous populations, we assume a common effect of stimulant use on MOUD initiation (conditional on measured covariates) across populations. We tested this assumption by fitting a model with an interaction between stimulant use and trial. We also tested this explicitly using a binary outcome of initiation . Neither test was statistically significant (p > 0.3 for both), supporting that our assumption was reasonable.
Stage 2: estimation of transported associations in real-world populations
Using variables identified in the selection diagrams, we calculated stabilized inverse probability of selection weights (IPSW) . The probability of selection into the CTN trials was calculated conditional on selection variables (Table 1; Additional file 1: Fig. S1) using logistic regression models. Probabilities were averaged across 10 datasets generated with multiple imputation of covariate data. Three sets of weights were generated, one for each target population (NSDUH, TEDS, ROI). Additional file 1: Table S2 details the three sets of IPSW that were used in the transportability analyses. In the analysis of NSDUH data, the IPSW model was calculated incorporating sampling weights to account for the complex survey design . Each set of weights was trimmed at the 1st and 99th percentiles, as is generally recommended in inverse probability weighting procedures to balance the bias-variance tradeoff due to extreme observations [44, 45]. Additional file 1: Table S3 presents descriptive statistics and shows bias and relative precision of generalized stimulant use associations using trimmed vs. untrimmed IPSW.
To complete the transportability analyses, we re-fit Cox regression models incorporating IPSW with robust standard errors (sandwich estimator) . By incorporating the weights, these models estimate the association between stimulant use and MOUD initiation in the three target populations. As detailed above, the same covariate sets were used for confounder adjustment (“Stage 1: estimation of stimulant use associations in the clinical trials”). As in the clinical trials, we first present overall estimates, and then estimates stratified by type of MOUD. All analyses were conducted in R v.4.0.5 with the ‘mice’, ‘survival’, ‘emmeans’, and ‘ggplot2’ packages.
Characteristics of RCT versus real-world target populations
This analysis included 673 clinical trial participants (n = 570 X:BOT and n = 103 CHOICES), 139 NSDUH respondents (weighted to represent 661,650 people), 71,751 TEDS treatment episodes, and 1933 ROI participants. Table 1 details the characteristics of the CTN trial participants and the three target populations. The majority of all included participants/respondents were between the ages of 30–49, male, and non-Hispanic White. Most had achieved a high school diploma or GED or completed some college, but fewer than half were employed. The age distribution was similar between clinical trial and target populations, but trial participants were substantially more likely to be male and Non-Hispanic Black or Hispanic than all target populations. There were differences in education levels, with ROI and TEDS participants generally having the lowest levels of formal education, clinical trial participants in the middle, and NSDUH respondents having the highest levels of education. Employment rates were about half as high among clinical trial participants and TEDS patients compared to the NSDUH and ROI samples. About a third of trial participants had a history of depression compared to 44% of the NSDUH sample (TEDS and ROI datasets did not contain depression data). Of ROI participants, 53% reported experiencing homelessness compared to 27% of clinical trial participants and 11% of the TEDS sample. Around 10% of each population used alcohol heavily. Rates of heavy benzodiazepine use were also around 10%, except for 16% in the NSDUH. IDU rates varied widely between populations: Two-thirds of clinical trial participants reported recently injecting drugs compared to 96% of ROI participants, 36% of the NSDUH respondents, and 54% of the TEDS sample. Overall, the clinical trial population was most like the TEDS population (average SMD = 0.21), less similar to the NSDUH respondents (average SMD = 0.27), and least so to the ROI participants (average SMD = 0.36).
Rates of stimulant use and treatment initiation or engagement
All individuals in the CTN studies had access to and were expected to initiate buprenorphine or XR-NTX as part of their study participation. A total of 80% of participants initiated MOUD during the trials (543/673). Rates were higher in X:BOT (83%) than CHOICES (68%), explained by their inpatient vs. outpatient treatment initiation settings. Initiation on buprenorphine was more likely than XR-NTX (90% vs. 71%). Similarly, all included TEDS treatment episodes had MOUD as part of the treatment plan, but TEDS does not report whether or not MOUD treatment was actually initiated. The NSDUH does not ask about MOUD treatment specifically, but 58% of included NSDUH respondents accessed some form of SUD treatment in the past year (but were still actively using opioids at the time of survey response). Among those who had not, 45% reported trying but being unable to access treatment and 55% had not tried. Among ROI participants, 38% reported engagement in inpatient or outpatient SUD treatment in the past 6 months and 64% of those without recent treatment engagement indicated that they had tried but were unable to obtain treatment.
At baseline, 105/673 (16%) CTN trial participants tested UDS positive for stimulants (26 MA/amphetamines only, 75 cocaine only, and four used both). Monthly stimulant use rates at follow-up ranged from 15 to 30% (mean 25%, standard deviation 5%).
Stage 1: associations between stimulant use and MOUD initiation in the clinical trials
Figure 2 displays the cumulative incidence of MOUD initiation in each CTN trial. Although fewer CHOICES participants initiated MOUD overall, there was little evidence that the associations between stimulant use and MOUD initiation differed by trial (p value for interaction = 0.884). In an analysis of the CTN trials, stimulant use reduced the likelihood of MOUD initiation by 32% (adjusted HR [aHR] = 0.68, 95% CI 0.49 to 0.94, p = 0.019). The impact of stimulant use on XR-NTX was larger (51% reduction, aHR = 0.49, 95% CI 0.28 to 0.86, p = 0.013) than on buprenorphine (33% reduction, aHR = 0.67, 95% CI 0.48 to 0.92, p = 0.015).
Stage 2: associations between stimulant use and MOUD initiation in the target populations
Table 2; Fig. 3 summarize results of our generalizability analyses. When adjusting for differences in demographics, substance use behaviors, and depression history between clinical trial participants and people needing treatment, as characterized by the NSDUH population, stimulant use reduced the likelihood of MOUD initiation by 25% overall (aHR = 0.75, 95% CI 0.48 to 1.18, p = 0.215). Estimates were similar when examining buprenorphine (23% reduction, aHR = 0.77, 0.48 to 1.23, p = 0.275) and XR-NTX independently (26% reduction, aHR = 0.74, 0.31 to 1.76, p = 0.496). However, all estimates of association displayed substantial uncertainty and none met the threshold for statistical significance.
Adjustment for differences between clinical trial participants and individuals entering treatment, as captured in the TEDS admissions dataset, suggested little change in the point estimate of stimulant use as observed in the trials (aHR = 0.72, 95% CI 0.51 to 1.01, p = 0.061). Medication-specific effect sizes in this population were attenuated compared to the trials, but similarly larger for XR-NTX (32% reduction, aHR = 0.68, 95% CI 0.34 to 1.06, p = 0.273) than buprenorphine (20% reduction, aHR = 0.80, 95% CI 0.55 to 1.16, p = 0.241), although confidence intervals were large.
Transportability analyses suggested a larger impact of stimulant use on MOUD initiation among rural people with high rates of IDU as compared to the clinical trials; we estimate that people using stimulants would be 39% less likely than non-users to initiate MOUD in this population (aHR = 0.61, 95% CI 0.35 to 1.06, p = 0.081). Additionally, larger differences were noted by type of MOUD: Stimulant use reduced the likelihood of XR-NTX initiation by as much as 76% (aHR = 0.24, 95% CI 0.08 to 0.69, p = 0.008), whereas a reduction of 37% was estimated for buprenorphine (aHR = 0.63, 95% CI 0.33 to 1.20, p = 0.161).
Our study suggests that stimulant use is a barrier to buprenorphine or XR-NTX initiation. Whether associations were estimated in randomized trials or generalized to people needing SUD treatment, people entering SUD treatment, or people from rural communities with high rates of IDU, concurrent stimulant use was associated with a 25–40% reduction in the likelihood of initiating MOUD. Although many of our generalized results failed to reach statistical significance, previous research has generally shown a negative association between stimulant use, especially MA, and MOUD initiation, with effect sizes similar to those observed in our study [11,12,13, 46, 47]. However, most of these studies have been cross-sectional, retrospective chart reviews, or included stimulant use as a self-reported, time-fixed exposure. By grounding our generalizability analysis in rich clinical trials data, our work adds valuable evidence to support the importance of addressing stimulant use among people with OUD seen in different settings and the need to develop strategies to improve MOUD initiation, such as outreach to needle exchanges or communities of people experiencing homelessness, given the higher rates of overdose and mortality in those with concomitant stimulant use.
Understanding differences between source (i.e., clinical trial participants) and target populations (people identifying the need for treatment, people entering treatment, and rural people with high rates of IDU) is critical to interpreting and applying results. RCTs frequently do not represent real-world populations of clinical relevance, and the degree to which exposure effects estimated from clinical trials are generalizable depends on differences in variables that modify the treatment effect . In this study, we observed that CTN trial participants were most similar to those included in the TEDS dataset, less so to NSDUH respondents, and least so to members of the ROI cohort. Therefore, we also observed that associations between stimulant use and MOUD initiation were more likely to change when generalized to people identifying a need for SUD treatment (NSDUH) and people from rural communities with high rates of IDU (ROI) compared to people entering treatment (TEDS). Likely, a key similarity between CTN and TEDS populations is that both were engaging in SUD treatment. In the trials, participants were extensively supported and encouraged to initiate MOUD as part of their participation, and TEDS treatment episodes were included if MOUD was part of the treatment plan. On the other hand, only about half of NSDUH respondents and 40% of ROI participants had recently engaged in any form of SUD treatment, and many others attempted to, but did not access treatment. Furthermore, the majority of ROI participants were persons who injected drugs, who may have more severe addiction and would be less likely to engage in treatment.
Stimulant use may inhibit MOUD initiation through several causal pathways, both structural and personal. Historically, clinical guidelines discouraged buprenorphine treatment for people who use other substances along with opioids; these were updated in 2020 with the caveat that this population may require additional support and more intensive care [48, 49]. Stigma from providers, communities, and other people who use drugs remains a substantial barrier to medication treatment engagement among people using both opioids and stimulants [50, 51]. Stimulant use may also affect one’s motivation, resourcefulness, social network, or physical and psychological capability to initiate MOUD , which may be differentially affected by access. Yet, although the size of the stimulant use association varied across target populations, it was universally negative, even if many of the generalized estimates were insufficiently precise to be conclusive.
NSDUH respondents were more likely to be white, highly educated, employed, and female than trial participants. The NSDUH is also a household survey, failing to reach the substantial number of people with SUDs who experience homelessness (nearly 30% of clinical trial participants had been recently homeless). Also, rates of IDU, a marker of the severity of substance use disorder, were about half as high in the NSDUH compared to the clinical trials. Results suggest that the impact of stimulant use may have been reduced in this population compared to that observed in the clinical trials, although results failed to reach statistical significance. Differences may be attributable to more support structures, resources, and/or other advantages available to racial/ethnic majority, educated, employed, and housed people with less severe substance use disorders responding to the NSDUH.
The CTN trials seem to be a better approximation of the population entering SUD treatment, as characterized by TEDS. This is supported by a previous analysis of TEDS data showing that people admitted for OUD treatment who also used MA were 35% less likely to have MOUD as part of the treatment plan, an effect size similar to that observed in the trials . Stimulant use still negatively impacted treatment uptake, but estimates of association were similar between TEDS and the clinical trials.
On the other hand, the negative impact of stimulant use on treatment initiation was nearly doubled in the population from rural areas, which were generally non-treatment seeking, had lower levels of education, and much higher rates of homelessness and IDU than the clinical trial participants. Stimulant use may be associated with as much as a 40% reduction in the likelihood of MOUD initiation in this population, which already suffers from limited access to SUD treatment , although the result failed to reach statistical significance. However, the negative impact of stimulant use on XR-NTX initiation was especially strong and statistically significant; given limited access to medically-monitored detoxification facilities in rural areas and the challenges people injecting opioids face when initiating a full opioid antagonist , our data adds further evidence that the real-world impact of XR-NTX may be limited in this population. Recent data showing extremely high rates of MA use in rural areas , rapid eastward spread of MA , and westward spread of fentanyl  are likely to worsen overdose risk and MOUD initiation for people who use both stimulants and opioids, nationally .
Our findings highlight the need for evidence-based interventions to reduce stimulant use among people from rural communities with OUD, who are rarely included in clinical trials . For example, contingency management interventions demonstrated to improve treatment engagement and decrease methamphetamine use in clinical trials, could be adapted and integrated with OUD treatment . While currently no pharmacotherapies are approved for treatment of stimulant use disorders, rigorously studied emerging therapies with modest benefit such as mirtazapine  and combined bupropion/extended-release naltrexone for methamphetamine use disorder  could be integrated with MOUD treatment, when appropriate. Structural interventions such as peer support services that have been used to improve treatment retention , and behavioral therapy (potentially delivered via telehealth) could be adapted to focus on people using both stimulants and opioids.
Generalizability and transportability analyses offer several advantages when answering questions about “real-world” impacts of treatment compared to using a single dataset, even if that dataset is large and representative of the target population . Notably, to conduct generalizability analyses, target population datasets do not always need to contain exposure or outcome information. In this case, the NSDUH did not ask about MOUD treatment until 2019, and even then, in a single question. The NSDUH and ROI are cross-sectional surveys and substance use data, including use of opioids and stimulants, are not longitudinal. TEDS admission data does not confirm MOUD receipt or collect quantitative substance use data. Therefore, the ability to answer our study question of interest in these datasets alone would be very limited. Additionally, secondary analyses of data not collected for research purposes may be especially subject to bias. For example, large national studies of VA electronic health record data  and private insurance claims data  showed small but positive associations between stimulant use and receipt of MOUD (RR ~ 1.2–1.3), in contrast to several other studies finding the opposite. On the other hand, data collected in clinical trials usually have higher validity; in these CTN studies, substance use data were collected via a biomarker in a time-varying fashion and the exact day a participant initiated MOUD was recorded. Thus, even though clinical trials are usually incompletely representative, the extensive datasets allow for less biased inference about the relationship between exposure and outcome, even if the exposure under study was not randomized. Generalizability analysis allows one to combine these strengths with broader, more representative data sources to answer questions about treatment effects in real-world populations of interest—something that each dataset alone could not.
Our study is subject to several limitations. Most notably, generalizability analysis relies on untestable assumptions about the causal mechanisms giving rise to differential treatment effects across populations. All observational studies rely on similar untestable assumptions; we utilized a combination of subject-matter expertise, protocols, previous literature, and data to select a covariate set for IPSW estimation, and we publish our assumptions in the form of a “selection diagram” (Additional file 1: Fig. S1). However, not all key variables were available in the data; we identify two missing items, engagement in general medical care (e.g., primary care) and psychiatric comorbidities other than depression, in the diagram. There are likely other unidentified effect-modifying variables that we were unable to account for. Second, the ability to transport to a target population can be limited by the available data characterizing differences between the study and target populations. For example, the NSDUH has been specifically criticized for underrepresenting injection drug use , a key modifier of our findings. The NSDUH also relies on complex survey weighting; results are subject to higher uncertainty and may be strongly influenced by a select few heavily-weighted individuals. TEDS data is primarily comprised of SUD treatment episodes in facilities receiving public funding, but the scope of facilities and medical records included varies considerably by state. Thus, TEDS data may not reflect the experiences of all people entering substance use treatment . Most inference made with these data would be subject to these challenges; they are not specific to generalizability analyses. The assumption of positivity, essentially, that there is some amount of overlap in selection characteristics between source and target populations, is seldom met in finite samples. Although we are unable to definitively conclude that positivity was sufficiently met, we excluded strict positivity violations (e.g., pregnant people who were excluded from both clinical trials) and explored the impact of positivity violations through weight trimming (Additional file 1: Table S3), consistent with best practices [44, 45]. Estimators of generalized treatment effects vary in efficiency, with weighting-based strategies being relatively inefficient, leading to wide confidence intervals and possibly explaining lack of statistical significance. Unfortunately, to the best of our knowledge, more efficient estimators (g-computation, targeted maximum likelihood based-methods) have not yet been developed for time-to-event outcomes with time-varying covariates. Also, we were unable to split stimulant use into MA/amphetamines and cocaine, due to the relatively small number of MA-positive UDS in the clinical trials. The trials were primarily conducted in East Coast sites, where the prevalence of MA use was lower compared to cocaine during the study periods. Finally, we were unable to examine the impact of stimulant use on methadone treatment, which was not included in X:BOT and only initiated by eight people in CHOICES (whom we excluded). Other studies have examined the impact of stimulants on methadone initiation .
This study emphasizes the negative impact that stimulant use may have on MOUD initiation in trials and extrapolates findings to three real-world populations needing OUD treatment, where polysubstance use is the norm rather than the exception. Although broadening the availability of MOUD is expected to improve the public health burden of opioid use, rapidly increasing rates of concomitant stimulant use may reduce its impact. An analysis of urine drug screens from 150,000 buprenorphine patients showed 10% cocaine and 30% MA positivity rates , underscoring the continuing challenge of polysubstance use following MOUD initiation. Efforts to identify efficacious treatments for stimulant use disorders should be intensified and integrated with OUD treatment. Within the changing landscape of the opioid overdose crisis due to fentanyl and other synthetic opioids, which are both rapidly becoming drugs of choice as well as adulterating the stimulant supply, there is greater urgency to address stimulant use among patients with OUD.
Availability of data and materials
CTN 0051 X:BOT data are publicly available on the NIDA datashare website (https://datashare.nida.nih.gov/study/nidactn0051). NSDUH and TEDS are publicily available from the SAMHSA website, https://www.datafiles.samhsa.gov/dataset/national-survey-drug-use-and-health-2019-nsduh-2019-ds0001 and https://www.datafiles.samhsa.gov/dataset/treatment-episode-data-set-admissions-2018-teds-2018-ds0001. CTN 0067 data will be posted to the NIDA datashare website upon de-identification and harmonization; interim requests for data should be directed to the corresponding author (email@example.com). Requests for ROI data should be directed to the data coordinating center (https://ruralopioidinitiative.org/studies.html).
Medications for opioid use disorder
Randomized controlled trial
Substance use disorder
National Drug Abuse Treatment Clinical Trials Network
National Survey on Drug Use and Health
Treatment Episodes Dataset
Injection drug use
Rural Opioids Initiative
Urine drug screen
Inverse probability of selection weights
Ellis MS, Kasper ZA, Cicero TJ. Twin epidemics: the surging rise of methamphetamine use in chronic opioid users. Drug Alcohol Depend. 2018;193:14–20.
Strickland JC, Havens JR, Stoops WW. A nationally representative analysis of “twin epidemics”: rising rates of methamphetamine use among persons who use opioids. Drug Alcohol Depend. 2019;204:107592.
Goodwin RD, Moeller SJ, Zhu J, Yarden J, Ganzhorn S, Williams JM. The potential role of cocaine and heroin co-use in the opioid epidemic in the United States. Addict Behav. 2021;113:106680.
Ahmad FB, Rossen LM, Sutton P. Provisional drug overdose death counts, for 12 months ending November 2021, updated April, 2022. National Center for Health Statistics; 2022. https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm.
Gladden RM, O’Donnell J, Mattson CL, Seth P. Changes in opioid-involved overdose deaths by opioid type and presence of benzodiazepines, cocaine, and methamphetamine—25 states, July–December 2017 to January–June 2018. MMWR Morb Mortal Wkly Rep. 2019;68(34):737–44.
Shover CL, Falasinnu TO, Dwyer CL, Santos NB, Cunningham NJ, Freedman RB, et al. Steep increases in fentanyl-related mortality west of the Mississippi river: recent evidence from county and state surveillance. Drug Alcohol Depend. 2020;216:108314.
Kerr T, Fairbairn N, Tyndall M, Marsh D, Li K, Montaner J, et al. Predictors of non-fatal overdose among a cohort of polysubstance-using injection drug users. Drug Alcohol Depend. 2007;87(1):39–45.
Wakeman SE, Larochelle MR, Ameli O, Chaisson CE, McPheeters JT, Crown WH, et al. Comparative effectiveness of different treatment pathways for opioid use disorder. JAMA Netw Open. 2020;3(2):e1920622.
Sordo L, Barrio G, Bravo MJ, Indave BI, Degenhardt L, Wiessing L, et al. Mortality risk during and after opioid substitution treatment: systematic review and meta-analysis of cohort studies. BMJ. 2017;357:j1550.
Lee JD, Nunes EV Jr, Novo P, Bachrach K, Bailey GL, Bhatt S, et al. Comparative effectiveness of extended-release naltrexone versus buprenorphine–naloxone for opioid relapse prevention (X:BOT): a multicentre, open-label, randomised controlled trial. Lancet. 2018;391(10118):309–18.
Xu KY, Mintz CM, Presnall N, Bierut LJ, Grucza RA. Comparative effectiveness associated with buprenorphine and naltrexone in opioid use disorder and cooccurring polysubstance use. JAMA Netw Open. 2022;5(5):e2211363.
Ford BR, Bart G, Grahan B, Shearer RD, Winkelman TNA. Associations between polysubstance use patterns and receipt of medications for opioid use disorder among adults in treatment for opioid use disorder. J Addict Med. 2021;15(2):159–62.
Frost MC, Lampert H, Tsui JI, Iles-Shih MD, Williams EC. The impact of methamphetamine/amphetamine use on receipt and outcomes of medications for opioid use disorder: a systematic review. Addict Sci Clin Pract. 2021;16(1):62.
Cunningham CO, Giovanniello A, Kunins HV, Roose RJ, Fox AD, Sohler NL. Buprenorphine treatment outcomes among opioid-dependent cocaine users and non-users. Am J Addict. 2013;22(4):352–7.
Bunting AM, Krawczyk N, Choo TH, Pavlicova M, McNeely J, Tofighi B, et al. Polysubstance use before and during treatment with medication for opioid use disorder: prevalence and association with treatment outcomes. J Subst Abuse Treat. 2022;143:108830.
Rowe CL, Santos GM, Kornbluh W, Bhardwaj S, Faul M, Coffin PO. Using ICD-10-CM codes to detect illicit substance use: a comparison with retrospective self-report. Drug Alcohol Depend. 2021;221:108537.
Jones CM, Underwood N, Compton WM. Increases in methamphetamine use among heroin treatment admissions in the United States, 2008–17. Addiction. 2020;115(2):347–53.
Chan B, Freeman M, Ayers C, Korthuis PT, Paynter R, Kondo K, et al. A systematic review and meta-analysis of medications for stimulant use disorders in patients with co-occurring opioid use disorders. Drug Alcohol Depend. 2020;216:108193.
Kennedy-Martin T, Curtis S, Faries D, Robinson S, Johnston J. A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials. 2015;16:495.
Susukida R, Crum RM, Stuart EA, Ebnesajjad C, Mojtabai R. Assessing sample representativeness in randomized controlled trials: application to the National Institute of drug abuse clinical trials network. Addiction. 2016;111(7):1226–34.
Humphreys K, Weisner C. Use of exclusion criteria in selecting research subjects and its effect on the generalizability of alcohol treatment outcome studies. Am J Psychiatry. 2000;157(4):588–94.
Humphreys K, Weingardt KR, Harris AH. Influence of subject eligibility criteria on compliance with national institutes of health guidelines for inclusion of women, minorities, and children in treatment research. Alcohol Clin Exp Res. 2007;31(6):988–95.
Storbjork J, Garfield JB, Larner A. Implications of eligibility criteria on the generalizability of alcohol and drug treatment outcome research: a study of real-world treatment seekers in Sweden and in Australia. Subst Use Misuse. 2017;52(4):439–50.
Kelpin SS, Ondersma SJ, Weaver M, Svikis DS. Representativeness of patients enrolled in a primary care clinical trial for heavy/problem substance use. Subst Abus. 2018;39(4):469–75.
Blanco C, Olfson M, Okuda M, Nunes EV, Liu SM, Hasin DS. Generalizability of clinical trials for alcohol dependence to community samples. Drug Alcohol Depend. 2008;98(1–2):123–8.
Okuda M, Hasin DS, Olfson M, Khan SS, Nunes EV, Montoya I, et al. Generalizability of clinical trials for cannabis dependence to community samples. Drug Alcohol Depend. 2010;111(1–2):177–81.
Lister JJ, Weaver A, Ellis JD, Himle JA, Ledgerwood DM. A systematic review of rural-specific barriers to medication treatment for opioid use disorder in the United States. Am J Drug Alcohol Abuse. 2020;46(3):273–88.
Seidler E, Keshaviah A, Brown CL, Wood E, Granick L, Kimball A. Geographic distribution of clinical trials may lead to inequities in access. Clin Invest. 2014;4:373–80.
McCabe SE, West BT, Jutkiewicz EM, Boyd CJ. Multiple DSM-5 substance use disorders: a national study of US adults. Hum Psychopharmacol. 2017;32(5):e2625.
Kim HS, Kim JH. Proceed with caution when using real world data and real world evidence. J Korean Med Sci. 2019;34(4):e28.
Bareinboim E, Pearl J. Causal inference and the data-fusion problem. Proc Natl Acad Sci. 2016;113(27):7345–52.
Pearl J, Bareinboim E. Transportability across studies: a formal approach. California: University of Los Angeles; 2011.
Pearl J, Bareinboim E. External validity: from Do-Calculus to transportability across populations. Stat Sci. 2014;29(4):579–95.
Korthuis PT, Cook RR, Lum PJ, Waddell EN, Tookes H, Vergara-Rodriguez P, et al. HIV clinic-based extended-release naltrexone versus treatment as usual for people with HIV and opioid use disorder: a non-blinded, randomized non-inferiority trial. Addiction. 2022;117(7):1961–71.
National Center for Health Statistics. Data hosting: national survey on drug use and health. https://www.cdc.gov/rdc/b1datatype/nsduh.htm.
The Substance Use and Mental Health Services Administration. Treatment episode dataset: admissions. https://www.datafiles.samhsa.gov/study-series/treatment-episode-data-set-admissions-teds-nid13518.
Jenkins RA, Whitney BM, Nance RM, Allen TM, Cooper HLF, Feinberg J, et al. The rural opioid initiative consortium description: providing evidence to understand the fourth wave of the opioid crisis. Addict Sci Clin Pract. 2022;17(1):38.
Vandevenne M, Vandenbussche H, Verstraete A. Detection time of drugs of abuse in urine. Acta Clin Belg. 2000;55(6):323–33.
Azur MJ, Stuart EA, Frangakis C, Leaf PJ. Multiple imputation by chained equations: what is it and how does it work? Int J Methods Psychiatr Res. 2011;20(1):40–9.
Liu S, O’Donnell J, Gladden RM, McGlone L, Chowdhury F. Trends in nonfatal and fatal overdoses involving benzodiazepines—38 states and the District of Columbia, 2019–2020. MMWR Morb Mortal Wkly Rep. 2021;70(34):1136–41.
Luedtke A, Carone M, van der Laan MJ. An omnibus non-parametric test of equality in distribution for unknown functions. J R Stat Soc Ser B (Stat Methodol). 2019;81(1):75–99.
Cole SR, Stuart EA. Generalizing evidence from randomized clinical trials to target populations: the ACTG 320 trial. Am J Epidemiol. 2010;172(1):107–15.
Ackerman B, Lesko CR, Siddique J, Susukida R, Stuart EA. Generalizing randomized trial findings to a target population using complex survey population data. Stat Med. 2020;40(5):1101–20.
Cole SR, Hernán MA. Constructing inverse probability weights for marginal structural models. Am J Epidemiol. 2008;168(6):656–64.
Austin PC, Stuart EA. Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Stat Med. 2015;34(28):3661–79.
Englander H, King C, Nicolaidis C, Collins D, Patten A, Gregg J, et al. Predictors of opioid and alcohol pharmacotherapy initiation at hospital discharge among patients seen by an inpatient addiction consult service. J Addict Med. 2019;14:415.
Lin LA, Bohnert ASB, Blow FC, Gordon AJ, Ignacio RV, Kim HM, et al. Polysubstance use and association with opioid use disorder treatment in the US Veterans Health Administration. Addiction. 2021;116(1):96–104.
Center for Substance Abuse T. SAMHSA/CSAT treatment improvement protocols. Clinical guidelines for the use of buprenorphine in the treatment of opioid addiction. Rockville: Substance Abuse and Mental Health Services Administration; 2004.
Crotty K, Freedman KI, Kampman KM. Executive summary of the focused update of the ASAM national practice guideline for the treatment of opioid use disorder. J Addict Med. 2020;14(2):99–112.
Lopez AM, Dhatt Z, Howe M, Al-Nassir M, Billing A, Artigiani E, et al. Co-use of methamphetamine and opioids among people in treatment in Oregon: a qualitative examination of interrelated structural, community, and individual-level factors. Int J Drug Policy. 2021;91:103098.
Russell C, Ali F, Nafeh F, LeBlanc S, Imtiaz S, Elton-Marshall T, et al. A qualitative examination of substance use service needs among people who use drugs (PWUD) with treatment and service experience in Ontario, Canada. BMC Public Health. 2021;21(1):2021.
Paulus MP, Stewart JL, Neurobiology. Clinical presentation, and treatment of methamphetamine use disorder: a review. JAMA Psychiatry. 2020;77(9):959–66.
Gauthier P, Greco P, Meyers-Ohki S, Desai A, Rotrosen J. Patients’ perspectives on initiating treatment with extended-release naltrexone (XR-NTX). J Subst Abus Treat. 2021;122:108183.
Korthuis PT, Cook RR, Foot CA, Leichtling G, Tsui JI, Stopka TJ, et al. Association of methamphetamine and opioid use with nonfatal overdose in rural communities. JAMA Netw Open. 2022;5(8):e2226544.
Jones CM, Compton WM, Mustaquim D. Patterns and characteristics of methamphetamine use among adults—United States, 2015–2018. MMWR Morb Mortal Wkly Rep. 2020;69(12):317–23.
Cook RR, Torralva R, King C, Lum PJ, Tookes H, Foot C, et al. Associations between fentanyl use and initiation, persistence, and retention on medications for opioid use disorder among people living with uncontrolled HIV disease. Drug Alcohol Depend. 2021;228:109077.
Brown HD, DeFulio A. Contingency management for the treatment of methamphetamine use disorder: a systematic review. Drug Alcohol Depend. 2020;216:108307.
Coffin PO, Santos GM, Hern J, Vittinghoff E, Walker JE, Matheson T, et al. Effects of mirtazapine for methamphetamine use disorder among cisgender men and transgender women who have sex with men: a placebo-controlled randomized clinical trial. JAMA Psychiatry. 2020;77(3):246–55.
Trivedi MH, Walker R, Ling W, Dela Cruz A, Sharma G, Carmody T, et al. Bupropion and naltrexone in methamphetamine use disorder. N Engl J Med. 2021;384(2):140–53.
Stack E, Hildebran C, Leichtling G, Waddell EN, Leahy JM, Martin E, et al. Peer recovery support services across the continuum: in community, hospital, corrections, and treatment and recovery agency settings—a narrative review. J Addict Med. 2022;16(1):93–100.
Inoue K, Hsu W, Arah OA, Prosper AE, Aberle DR, Bui AAT. Generalizability and transportability of the national lung screening trial data: extending trial results to different populations. Cancer Epidemiol Biomark Prev. 2021;30(12):2227–34.
Manhapra A, Stefanovics E, Rosenheck R. Initiating opioid agonist treatment for opioid use disorder nationally in the Veterans Health Administration: who gets what? Subst Abus. 2020;41(1):110–20.
Morgan JR, Schackman BR, Leff JA, Linas BP, Walley AY. Injectable naltrexone, oral naltrexone, and buprenorphine utilization and discontinuation among individuals treated for opioid use disorder in a United States commercially insured population. J Subst Abus Treat. 2018;85:90–6.
Han B, Volkow ND, Compton WM, McCance-Katz EF. Reported heroin use, use disorder, and injection among adults in the United States, 2002–2018. JAMA. 2020;323(6):568–71.
Batts K, Pemberton M, Bose J, Weimer B, Henderson L, Penne M, et al. Comparing and evaluating substance use treatment utilization estimates from the national survey on drug use and health and other data sources. CBHSQ data review. Rockville: Substance Abuse and Mental Health Services Administration; 2012. p. 1–120.
Saloner B, Whitley P, LaRue L, Dawson E, Huskey A. Polysubstance use among patients treated with buprenorphine from a national urine drug test database. JAMA Netw Open. 2021;4(9):e2123019.
The authors would like to thank the many research staff and participants involved in CHOICES, X:BOT, the ROI, and the NSDUH for making this work possible.
X:BOT and CHOICES were supported by the NIDA CTN, NIH/NIDA UG1DA013035. The ROI is supported by UG3DA044829/UH3DA044829, UG3DA044798/UH3DA044798, UG3DA044830/UH3DA044830, UG3DA044823/UH3DA044823, UH3DA044822/UH3DA044822, UG3DA044831/UH3DA044831, UG3DA044825, UG3DA044826/UH3DA044826, U24DA044801, and UL1TR002369 co-funded by NIDA, ARC, CDC, and SAMHSA. RC was supported by NIDA K01DA055130 and AHRQ/PCORI K12HS026370. Funders had no role in the design of the study, collection, analysis, and interpretation of data, or in writing the manuscript.
Ethics approval and consent to participate
All included studies were conducted with ethical approval and all included participants provided informed consent prior to participation. CHOICES was approved by the Advarra Institutional Review Board (IRB00000971), with participating sites deferring to its regulatory role, while each site in X:BOT and the ROI received local IRB approval prior to trial initiation. The NSDUH and TEDS are collected by the U.S. Substance Abuse and Mental Health Services Administration under federal privacy and confidentiality laws and regulations (https://www.samhsa.gov/about-us/who-we-are/laws-regulations).
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Selection diagram fortransportability analysis of the effects of stimulant use on initiation ofmedication for opioid use disorder (MOUD). Variables in gray text were notmeasured in one target population; those in gray boxes were not measured in anytarget population. Table S1. Inclusion andexclusion criteria, recruitment, screening and enrollment data for CTN clinicaltrials. Table S2. Inverse probabilityof selection weight models. Table S3.Distributions of inverse probability of selection weights and bias-variance tradeoffin weight trimming. Bias and precision of the stimulant use estimates arecomputed relative to untrimmed weights.
About this article
Cite this article
Cook, R.R., Foot, C., Arah, O.A. et al. Estimating the impact of stimulant use on initiation of buprenorphine and extended-release naltrexone in two clinical trials and real-world populations. Addict Sci Clin Pract 18, 11 (2023). https://doi.org/10.1186/s13722-023-00364-3