Implementation model

Implementation will be guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework to study how best to implement the bundled treatment approach in juvenile justice settings and CMHCs [38]. We will utilize the Stages of Implementation Completion (SIC) implementation approach to measure inner and outer context factors that affect implementation of best-practices [38,39,40,41]. The outer context, in this case, includes state and local policies regarding funding and provision of addiction services for youth in general and, specifically, for justice involved youth. Inner context includes the organizations climate, culture, staffing as well as characteristics of juvenile justice and community mental health center (CMHC) staff towards adolescents (see Fig. 1 for illustration of EPIS model). The EPIS framework also explicitly identifies project phases, including exploratory phases, which is useful in the conceptualization of our pilot project. To guide and assess implementation we will collect data from self-report surveys and interviews conducted with substance use providers, parents, youth and court personnel across the study period.

Fig. 1 EPIS implementation model Full size image

Study setting

Two counties in a Midwest state were chosen based on high rates of substance-related arrests and rates of substance use among adolescents in the community. Counties were chosen given their small size as well as the presence of one primary community mental health center (CMHC) that is designated to serve the community. This allows for feasibility of building and strengthening relationships between juvenile justice and community mental health professionals/agencies.

Participants

YIJJ and caregivers

We will recruit YIJJ aged 14–17 in one of two counties who are either detained in the detention center and returning to the community or currently in the community. Eligible youth will be those who are identified as having a history of substance use, based on screening done by the juvenile justice staff (see below for explanation). Both youth and their legal guardian must agree to participate. Only youth and caregivers who are fluent in English will be eligible to participate.

Community partner participants

Community partner participants will be clinicians, case managers, and supervisors working in the CMHC in each county (range n = 10–30). Bachelor’s level case managers from each CMHC site (n = 2 case managers for each county) will be recruited to act as intervention specialists. They will be trained in the brief intervention and provide treatment to YIJJ and their caregivers. Master’s level licensed clinicians from each CMHC (n = 2 for each county) will be trained in the comprehensive substance use intervention and provide treatment to YIJJ.

The juvenile justice system in each county have agreed to participate and implement substance use screening. Designated juvenile probation officers/intake officers will be identified and will be responsible for conducting universal substance use screening with all youth coming through the detention center at intake.

In addition to these primary individuals who will provide interventions, we will seek participation from other personnel (e.g., juvenile justice intake staff, CMHC supervisors) who are involved in the implementation and sustainability of the interventions to complete qualitative interviews and quantitative measures to assess feasibility and implementation in these settings.

Training and fidelity monitoring

Juvenile justice substance use screening

We will conduct a one-time training for juvenile probation officers at each of the JJ system sites to train them on the substance use screening tool (see below for details). Training is minimal compared to the intervention training given the ease/usability of the measure.

Interventions

Prior to the start of the trial, therapists and intervention specialists at each of the two sites will be trained by respective intervention training teams (i.e., training team for Teen Intervene and ENCOMPASS). Bachelor’s level brief intervention specialists will be trained in the brief interventions. For the purposes of wider dissemination of these evidence-based brief interventions, non-research providers at each CMHC also will have the opportunity to receive initial training. Following a one-time initial training, brief intervention specialists will have monthly meetings with the training team for ongoing training, consultation, and fidelity monitoring. Master’s level therapists who will be designated as ENCOMPASS therapists at each of the sites will be trained in ENCOMPASS by the ENCOMPASS master trainers. Following initial in-person training, ENCOMPASS therapists will engage in weekly meetings with the ENCOMPASS trainers for ongoing training, consultation, and fidelity monitoring. Psychotropic medication prescribers are also engaged in the weekly meetings and supervision is provided by an adolescent addiction psychiatrist.

Fidelity monitoring

Fidelity monitoring for the brief intervention and ENCOMPASS will be conducted by the respective intervention trainers.

Teen Intervene Brief intervention specialists will complete self-rating checklists following each session and also audiotape sessions to be rated by Teen Intervene trainers for fidelity [28]. Therapists and raters will complete the same checklists, which include items related to module-specific activities as well as items related to adherence to principles of motivational interviewing (e.g., reinforcing change talk) and cognitive behavioral techniques (e.g., assigning homework). Items are rated on a 5-point scale, with higher scores denoting greater fidelity/adherence and competency in skills. Mean scores of 3 across all items denote sufficient competency and fidelity.

ENCOMPASS ENCOMPASS therapists will complete self-rating treatment adherence fidelity checklists following each session as well as audio record sessions to be reviewed by ENCOMPASS trainers for fidelity checks. The same checklist will be used by therapists and raters to measure adherence and consists of six items related to principles of cognitive behavioral therapy six items related to principle motivational interviewing and motivational enhancement techniques, as well as items related to module-specific tasks and activities. Therapists self-rate their performance for each session in each of these areas on a 6-point scale (0 = not completed, 1 = low skill level to 5 = high skill level); thus, fidelity to the treatment can be quantified as an overall mean score for each session. Overall mean scores of 3 or above on the self-rating forms and/or trainer ratings of audiotaped sessions will be considered meeting minimum fidelity/adherence standards [32, 33].

Procedures

Screening and referral

Juvenile probation officers will screen all youth who come through the intake process at the detention center using the CRAFFT [42]. The CRAFFT consists of six yes or no questions regarding substance use (e.g., do you ever use alcohol or drugs to relax; has family ever been concerned about your alcohol or drug use) as well as questions regarding frequency of past year substance use, with higher scores (range 0–6) denoting more severe substance use problems (see Additional file 1: Appendix S1 for complete measure). The CRAFFT has been shown to be an effective clinical tool in identifying substance use risk level among children and adolescents [41]. The research team will be notified of any youth scoring at least a 1 on the CRAFFT measure. Participants also will be notified of the opportunity to participate in an intervention and will be told that they might be contacted regarding voluntary participation in a research study. The research team will contact potentially eligible participants.

Enrollment and baseline assessment

Following initial recruitment, research assistants will complete consent/assent with the caregiver/youth at their home or preferred location. As part of the consent process, participants will be told that this is an intervention study and that they might be assigned to different interventions. Participants and their caregivers also will complete a baseline assessment, which includes self-report measures of demographics, family functioning, mental health, substance use (see Table 1 for overview of all measures).

Table 1 Overview of data collection Full size table

Risk level determination

Based on the youth’s initial CRAFFT score as well as their scores on initial baseline measures collected after consent at the first/baseline visit, youth will be categorized as either low to mid-risk or high-risk based on criteria from the American Society of Addiction Medicine (ASAM). ASAM criteria utilizes six dimensions of an individual’s substance use and overall functioning to determine appropriate level of care [25]. We will determine risk level based on youth’s pattern of substance use, such as frequency and quantity of use, as well as problematic use based on DSM-5 substance use disorder symptoms. Table 2 describes specific risk determination criteria we will determine based on assessment. Generally, high-risk youth will be those who have identified past year repeated use of more lethal substances with a high risk for overdose and withdrawal potential (e.g., opioids, methamphetamine) and those who endorse three or more DSM-5 SUD symptoms, which is consistent with a moderate to severe SUD. All other youth initially will be randomized to one of the brief interventions. Based on preliminary data collected from each county, it is estimated that about 10% of youth will be in the high-risk group.

Table 2 Risk level determination Full size table

Following determination of risk level, high-risk youth will be assigned to complete ENCOMPASS, the comprehensive substance use treatment. Youth who are categorized as low- or mid-risk will be randomly assigned to one of the two brief interventions. Following randomization, youth will engage in either Teen Intervene or Family Check-Up with a brief intervention specialist.

Measuring treatment response

Following completion of the first round of brief interventions (three sessions of Teen Intervene), we will measure youth’s response to treatment. Treatment response will be measured utilizing the Brief Situational Confidence Questionnaire (BSCQ), an eight item self-report measure of one’s level of confidence in resisting urges to use substances in eight situations: unpleasant emotions, physical discomfort, pleasant emotions, testing of control, urges and temptations, conflict with others, social pressures, and pleasant times with others [42]. Individuals rate their confidence in abstaining from substance use in each of the situations on a 0% (not at all confident) to 100% (completely confident), with higher scores denoting greater confidence in abstaining from substance use. Treatment response will be determined utilizing a cut-off overall mean score of 70% on the BSCQ (i.e., average of 70% confidence in abstaining from using substance use across all eight scenarios), based on results from previous studies utilizing the BSCQ as a measure of symptom improvement [43]. The cutoff of 70% has two advantages: it is the average treatment response expected for this intervention, and it yields (approximately, if the distribution is symmetric) an anticipated equal sample-size split for responders and non-responders for the second randomization. Those with a post-intervention score of 70% or higher will be considered “treatment responders” and those whose BSCQ post-intervention score is lower than 70% will be considered “treatment non-responders.” Those youth who are considered “responders” will be told that they have successfully completed the intervention; they will complete additional post-treatment measures at 3 and 6 month follow-up. Non-responders will continue to the second randomization.

Following completion of treatment response measures using the BSCQ, those who are identified as “non-responders” (i.e., mean post-test BSCQ score lower than 70) will complete a second round of brief interventions. If youth continue to not respond, clinicians will make recommendations and referrals as appropriate. Following completion of the second round of brief intervention, treatment response will be measured again using the BSCQ and participants will complete 3- and 6- month follow-up (see Fig. 2).

Fig. 2 Overview of study design. YIJJ who score 1 or more on the CRAFFT will be recruited for the study. If interested and consent to treatment, they will be assessed to determine risk level. YIJJ identified as high-risk youth will be assigned to complete ENCOMPASS. Low to mid risk youth will be assigned to the brief intervention, Teen Intervene. Following the brief intervention, we will measure treatment response using the BSCQ. YIJJ who score below 70% will be considered “non-responders” and referred to ENCOMPASS Full size image

Ongoing data collection

In addition to the intervention, participants and their caregivers will complete data collection at three time points: baseline, 3 months, and 6 months. Parents and youth each will be compensated $50 at each data collection point. In addition to measures of treatment response as described above, participants will complete self-report measures of family functioning, mental health symptoms, other risk behaviors, and psychosocial functioning (see Table 1 for overview of measures). These additional measures will help to explore additional characteristics related to treatment response.

Community partners will complete pre- and post-intervention measures related to feasibility of implementation and sustainability (e.g., organizational climate; see Table 1 for complete list of measures).

Data analysis

Measurement of intervention effectiveness and treatment response

Preliminary analyses

Preliminary analyses will be used to compare the primary treatment response outcome (confidence of abstaining measured by the BSCQ) and substance use outcome across the responders and non-responders of the brief intervention and ENCOMPASS at baseline, 3 months, and 6 months. A separate model will be utilized to measure effectiveness among those assigned to ENCOMPASS. Substance use will be calculated as number of days in the past month of using alcohol, marijuana, and other substances (range 0–90). These outcomes will be compared between groups with longitudinal generalized linear models (GLM) using the GENMOD procedure with a linear link and normal error distribution. The BSCQ and substance use outcome will be analyzed in separate models. The baseline measure of BSCQ or substance use days will be adjusted as covariates and the repeated measurements of each outcome at 3 months and 6 months will be analyzed simultaneously in the model.

The GLMs are easily fit for non-Gaussian distributions by specifying appropriate link functions and error distributions. If substance use days exhibit sparse counts or large frequency of zeros, we will consider using logistic, Poisson, or zero-inflated Poisson regression models, depending on the observed distributions of outcome behaviors. We also will explore analyzing substance use as time-to-event data using a Cox proportional hazards regression model with the SAS PHREG procedure with censoring if loss to follow-up occurs. Effect sizes will be computed from the model as differences between groups on adjusted means divided by SDs for the linear models, or differences between odds for logistic models or hazards for Poisson models.

The median and relevant percentile (e.g. 25th or 75th) of time to events will be presented by group with associated confidence intervals. All models will include an indicator variable for whether the subject was enrolled as a detained or probation youth. Other covariates will be included based on their theoretical importance as potential confounders and on their imbalance between randomized arms at baseline. We also will examine moderators that we have hypothesized to be potential determinants of treatment response (e.g., family functioning).

Analyses

The specific analyses and hypothesis tests (which are preliminary because this is a feasibility study) will assess effectiveness of the brief intervention and ENCOMPASS on substance use outcomes. These data will be used to generate hypotheses about brief and comprehensive interventions for a future larger study.

Sample size and power

We plan to recruit n = 80 youth and caregiver dyads per county (160 total). We estimate that no more than 10% of youth in each county will be high-risk, resulting in a sample size of n = 16 assigned to ENCOMPASS and n = 144 to first-stage brief intervention (BI). We anticipate 20% loss at 6-month follow-up resulting in n = 12 in the ENCOMPASS group and n = 116 in the first-stage BI with 6 month data (58 responders and 58 non-responders, assuming a 50% response rate. We have accounted for a sample size inflation factor related to potential within-county intraclass correlation (ICC) of outcomes. Specifically, in the first step of the back-calculation, we determined that a sample size of n = 40 per responders and non-responders to the first-stage intervention (with 6-month data) is required under the “independence of observations” assumption (i.e., before accounting for clustering) to provide 80% power for the two-sided model-based tests to detect an effect size (0.63) that is between medium (0.50) and large (≥ 0.80). For a first-stage effective sample size of 40 in each group (responder vs. non-responder), there is a 1.39 sample size inflation factor (IF) when the ICC = 0.01 and the number of clusters (i.e., counties) = 2: IF = [1 + (average 40 patients per cluster − 1) × ICC of 0.01]. Therefore, an actual sample size of 56 per first-stage brief intervention (i.e., 40 × 1.39) with 6-month data is required (i.e., an effective sample size of 40 per first-stage brief intervention with 6-month data if observations were not clustered). The sample of 56 in each group will provide adequate power for this feasibility study and its preliminary hypothesis tests, given that we anticipate n = 58 per first-stage group (responder vs. non-responder) with 6-month data. This power calculation is conservative because the longitudinal models will have slightly increased power due to incorporating all available data (baseline, 3-months, 6-months), including data from persons missing one but not all times points. The goal will be to estimate effect sizes, for which the planned sample size will be adequate. A minimum of 12 per group is recommended for estimating effects sizes of continuous outcomes in pilot or feasibility studies [44].

Measurement of implementation in juvenile justice and CMHC settings

We will utilize both qualitative and quantitative methods to assess implementation of substance use screening in the juvenile justice systems and implementation of interventions in the CMHCs.

Stages of implementation completion exploratory analyses

We will utilize the stages of implementation completion (SIC) tool to track and assess implementation progress separately for the CMHCs and juvenile justice settings. Utilizing the online SIC tracker, we will log data occurring at each of the implementation stages: pre-implementation (exploration and preparation), implementation, and sustainability. The online SIC tracker computes SIC scores at each implementation stage for measures such as proportion (percentage of planned activities performed) and duration (number of days between activities), which have been shown to predict successful achievement of implementation milestones [45, 46]; data also provides insights into organizational implementation behavior predicting sustainment.

Quantitative analysis

We will conduct separate analyses for juvenile justice systems and CMHCs based on CMHC and juvenile justice staff completion of implementation measures (see Table 1 for complete list of measures). We will use linear mixed models to analyze repeatedly measured implementation variables (baseline/pre-implementation, 6 months/post-implementation) to determine whether implementation variables change over time. In one model measuring implementation of substance use screening in juvenile justice settings, organizational readiness to implement change (based on juvenile justice staff self-reports) will serve as the dependent variable and staff and setting characteristics (e.g., demographics, years of experience, see Table 1) will serve as independent variables. Similar analyses will be conducted to assess implementation with the CMHCs.

Qualitative analysis

We will conduct separate qualitative analyses for implementation of substance use screening in the juvenile justice systems and implementation of interventions in the CMHCs. We will separately analyze qualitative interviews with juvenile justice staff and CMHC staff using an inductive, interpretive approach based on grounded theory [47,48,49]. Two coders will initially review transcripts and complete open coding to identify emergent themes. Following open coding, coders will discuss findings and then complete focused coding utilizing a determined coding scheme based on emergent themes. This qualitative analysis can be viewed as a developmental approach where findings will both improve understanding of implementation and inform refinement of implementation strategies.