Abstract Objective A central goal of health care is to improve patient outcomes. Although several studies have demonstrated the effectiveness of therapist guided internet-based cognitive behaviour therapy (ICBT) for social anxiety disorder (SAD), a significant proportion of patients do not respond to treatment. Consequently, the aim of this study was to identify individual characteristics and treatment program related factors that could help clinicians predict treatment outcomes and adherence for individuals with SAD. Method The sample comprised longitudinal data collected during a 4-year period of adult individuals (N = 764) treated for SAD at a public service psychiatric clinic. Weekly self-rated Liebowitz Social Anxiety Scale (LSAS-SR) scores were provided. Rates of symptomatic change during treatment and adherence levels were analysed using multilevel modelling. The following domains of prognostic variables were examined: (a) socio-demographic variables; (b) clinical characteristics; (c) family history of mental illness; and (d) treatment-related factors. Results Higher treatment credibility and adherence predicted a faster rate of improvement during treatment, whereas higher overall functioning level evidenced a slower rate of improvement. Treatment credibility was the strongest predictor of greater adherence. Having a family history of SAD-like symptoms was also associated with greater adherence, whereas Attention-Deficit/Hyperactivity Disorder (ADHD)-like symptoms, male gender, and family history of minor depression predicted lower adherence. Also, the amount of therapist time spent per treatment module was negatively associated with adherence. Conclusions Results from a large clinical sample indicate that the credibility of ICBT is the strongest prognostic factor explaining individual differences in both adherence level and symptomatic improvement. Early screening of ADHD-like symptoms may help clinicians identify patients who might need extra support or an adjusted treatment. Therapist behaviours that promote adherence may be important for treatment response, although more research is needed in order to determine what type of support would be most beneficial.

Citation: El Alaoui S, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, et al. (2015) Predictors of Symptomatic Change and Adherence in Internet-Based Cognitive Behaviour Therapy for Social Anxiety Disorder in Routine Psychiatric Care. PLoS ONE 10(4): e0124258. https://doi.org/10.1371/journal.pone.0124258 Academic Editor: Melissa Norberg, Macquarie University, AUSTRALIA Received: November 4, 2014; Accepted: March 11, 2015; Published: April 20, 2015 Copyright: © 2015 El Alaoui et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited Data Availability: Study data are stored at the repository of Karolinska Institutet. Data are not freely accessible, but each request is assessed by Karolinska Institutet ethics committee, and approval of data access can be given after this assessment. To request data, contact either Samir El Alaoui at samir.el.alaoui@ki.se, or Karolinska institutet by e-mail at info@ki.se, or at Karolinska Institutet, SE-171 77, Stockholm, Sweden. Funding: This study was supported by research grants from Stockholm County Council (Nils Lindefors). The main funding organization is a public institution, and neither of the funding organizations had any role in the design and conduct of the study; in the collection, management, and analysis of the data; or in the preparation, review, and approval of the manuscript. Competing interests: The authors have declared that no competing interests exist.

Introduction Social anxiety disorder (SAD) is one of the most common anxiety disorders [1] and is often described as following a chronic course if untreated. SAD is characterized by an intense fear in social situations and is associated with significant costs, both for the affected individuals and for society [2]. The effectiveness of cognitive behavioural therapy (CBT) in the treatment of SAD is well established [3]. To increase accessibility to CBT, guided internet-based CBT (ICBT) has been developed with outcomes comparable with face-to-face CBT [4]. However, a significant proportion of patients do not achieve clinically significant improvement, irrespective of treatment modality. For example, it has been reported that up to 50% of patients do not respond to first-line treatments such as CBT or pharmacotherapy [5]. Also, in a study on ICBT, 23% of subjects did not improve on any measure [6]. When ICBT is considered for dissemination within regular clinical practice, identifying prognostic factors that explain individual differences in how patients respond to ICBT when administered within the context of routine care could lead to improvements in treatment delivery. Specifically, this may help clinicians better predict how different subgroups of patients with SAD are likely to respond to ICBT. Such knowledge could contribute to treatment modifications that decrease outcome variability and increase treatment effectiveness for a wider range of patients. There are few studies on outcome predictors of ICBT for SAD and little consensus regarding the evidence of these. This might be due to several reasons such as differences in samples size, sample characteristics and statistical methods for analysing the prognostic value of baseline variables. Therefore, our selection of outcome predictors was not restricted to internet interventions but refers to the larger literature of CBT for SAD [7,8] and is guided by previous research in this area [9,10]. In line with a number of recent outcome prediction studies on SAD [11–14] and constrained by available patient data routinely collected within the clinic, variables with potential prognostic value were categorized into four domains: (1) socio-demographic variables (e.g. age, gender), (2) clinical characteristics (e.g. illness severity, comorbidity), (3) family history of mental illness and (4) treatment-related factors (e.g. adherence, treatment credibility and amount of therapist support). Examples of identified outcome predictors within the socio-demographic domain that have been found to be associated with a positive outcome in ICBT for SAD are employment status (i.e. working full-time) and having children [13]. In regard to clinical characteristics, illness severity and number of comorbid diagnosis have also been identified as significant outcome predictors [15]. Also, although comorbid depression has been related to lower treatment response during both CBT [16,17] and ICBT [13] for SAD and higher illness severity after treatment [17,18], several studies have not been able to link comorbid depression to the rate of symptomatic change during treatment (e.g. [18,19]), and a recent study found no predictive value of comorbid depression [15]. In regard to treatment-related factors, treatment credibility and adherence have been identified as being of special relevance to the effectiveness of self-help interventions such as ICBT [20,21]. Several studies have found that credibility of ICBT [13,14] and adherence to either face-to-face CBT or ICBT [6,13,14,22,23] is associated with treatment outcome. However, since adherence is measured during or after treatment completion, it may be of limited clinical use as a pre-treatment prognostic predictor. Therefore, examining adherence not only as an outcome predictor but also as an outcome itself has been identified as an aspect of prediction research on ICBT that needs more attention [24]. We are only aware of one study that has examined this relationship in regard to ICBT for SAD specifically, where treatment credibility was found to predict adherence in unguided self-help, but not in guided self-help. The aim of this study was to examine individual differences in response to ICBT for SAD and to facilitate possible future identification of patients who might need additional support during treatment. To identify predictors of symptomatic change and adherence of ICBT within the context of a routine clinical setting, we explored effectiveness data obtained at a publicly funded psychiatric clinic specialized in delivering ICBT. Based on findings in previous research on ICBT, we hypothesized that treatment credibility and adherence would have prognostic value also in a naturalistic setting, where higher levels of these factors would likely be associated with a more favourable outcome. To analyse the data, multilevel modelling techniques were used. These statistical methods take advantage of all available data collected during the study period and have been shown to produce more precise parameter estimates and require fewer statistical assumptions [25,26]. A stepwise procedure adopted by Fournier et al. [27] was followed where predictor domains (socio-demographic variables, clinical variables, family history and treatment factors) were first examined separately. This allowed us to determine the predictive value of each variable while controlling for the other variables within each domain. Then, a final model was tested where only those variables identified as significant outcome predictors within each domain were simultaneously included.

Materials and Methods Participants The study sample consisted of 764 adult patients, all of which had a principal diagnosis of SAD and had undergone treatment as part of routine outpatient psychiatric care at a specialized public ICBT clinic at the time of data extraction (i.e. patients having completed treatment between October 2010 and October 2014). Patients were admitted to treatment either through self-referral or through referral from psychiatric or primary care. All patients first completed an on-line screening battery of self-report questionnaires. Then, clinician administered assessments were conducted before admittance where the Mini-International Neuropsychiatric Interview (MINI) [28] diagnostic interview was used by psychiatrists (or resident physicians supervised by psychiatrists) to diagnose Axis I DSM-IV disorders. Ethics Statement The study, including its consent procedure, was approved by the Regional Ethical Review Board in Stockholm, Sweden (no 2011/2091-31/3). Since this research was conducted as a retrospective cohort study, active informed consent was not required and therefore not obtained. However, as required by the ethics committee, a letter explaining the study was sent to all participants prior to data collection providing a choice of participation. Thus, participation in the study was obtained through an opt-out methodology. This approach (i.e. passive consent) is considered to be the most efficient procedure for registry data that does not violate the option of providing choice [29] and also increases the likelihood of having a more representative sample [30]. Study Design and Treatment Over the study period, approximately 25 different psychologists delivered the treatments. These were trained in CBT and either licensed or occasionally resident psychologists under clinical supervision. Therapists followed a treatment manual originally developed by Andersson and colleagues [31] with documented effects in several randomized controlled trials [32] adhering to Clark and Wells’ [33] cognitive model of social phobia developed for individual therapy of SAD. Non-inferiority of the treatment compared to conventional group-based CBT has previously been documented within the context of routine psychiatric care [34]. Treatment modules consisted of psycho-educative text and supporting figures, complemented with homework assignments. This is similar to self-help books used in bibliotherapy, with the difference that the material was accessed online after logging in to a secure treatment website. In ICBT, treatment modules are the equivalence of sessions in conventional CBT. However, since the treatment content was presented in text form, patients were able to learn at their own convenience. Similar to sessions in CBT, modules were administered sequentially; after completion of a module and its homework assignment, the subsequent module was activated by the therapist. This meant that patients had to complete a module in order to get access to the following module. The first module introduced patients to CBT for SAD providing rationale for the treatment. This was followed by exercises in cognitive restructuring to help patients understand how SAD is maintained and learn how to recognize maladaptive or negative automatic thoughts with the aid of a detailed review of Clark & Wells’ cognitive model. Patients were also taught how to challenge negative thoughts and encouraged to reflect on treatment goals. In the next phase of treatment, behavioural experiments were introduced, including gradual exposure to feared situations. During these modules, patients learned about the function of safety behaviours and experimented with attentional refocussing to enhance the effect of exposure. The final modules contained material on developing communication and social skills and relapse prevention strategies. The last module encouraged patients to review what they had learned throughout the treatment, and to make a plan for continued improvements and how to manage setbacks. The role of the therapists was to guide patients through each treatment module. This was conducted through online written conversations within the secure treatment platform. This included not only providing feedback on homework assignments, but prompting patients when they were inactive or helping them with problem solving when experiencing difficulties. Thus, more therapist time tended to be allocated to patients having a greater need for support and guidance. Outcome Measures The primary outcome measure was the self-rated version of the Liebowitz Social Anxiety Scale, LSAS-SR [35]. The LSAS-SR has reported strong psychometric properties with an internal consistency of α =. 95 and a 12-week test-retest reliability of r =. 83 [35,36]. The LSAS has also been evaluated for internet administration with a reported internal consistency of α =. 94 [37]. For the current sample, Cronbach's alpha of the LSAS-SR at pre-treatment was. 95. Besides at pre- and post-treatment, social anxiety was measured weekly throughout the active treatment phase. The secondary outcome measure was treatment adherence, operationalized as the number of activated modules. Potential Predictor Variables All potential predictors of symptomatic change and adherence were measured at baseline except for treatment program factors, which were measured during treatment (e.g. treatment credibility was assessed during the second week) or after treatment (e.g. adherence). Each variable was assigned to one of the following four domains: socio-demographic variables, clinical characteristics, family history of mental illness and treatment-related factors. Socio-demographic variables. This domain comprised 6 variables: age, gender, level of education, employment status (dichotomized; i.e. working part- or full-time or student vs. unemployed, part- or full-time sick leave or disability retirement), cohabitation status (dichotomized; i.e. single, divorced, widowed, or separated vs. married or living with partner) and having children. Clinical characteristics. The clinical domain consisted of 14 variables: Clinical Global Impression—Severity Scale [38] which was rated on a 7-point scale; co-morbid depressive symptoms were measured at baseline using the self-rated Montgomery-Åsberg Depression Rating Scale (MADRS-S) [39]; the Alcohol Use Disorders Identification Test (AUDIT) [40] was used prior to treatment to screen for alcohol problems and the Drug Use Disorders Identification Test (DUDIT) [41] was used to screen for drug misuse; the Adult ADHD Self-Report Scale-V1.1 (ASRS) was used as a short screening instrument for adult ADHD-like symptoms [42]; the number of co-morbid diagnoses was assessed during the clinician diagnostic interview using the MINI; the Global Assessment of Functioning (GAF) was used to assess social, occupational, and psychological functioning; years since onset of symptoms was assessed during the clinical interview; age of onset of symptoms was calculated by subtracting the reported number of years since onset of symptoms from the patient’s age at the time of treatment; generalized beliefs of self-efficacy was assessed using the General Self-Efficacy Scale (GSES) [43]; concurrent psychotropic medication, history of depression, history of inpatient psychiatric care and history of attempted suicide were assessed during the diagnostic interview. Family history of mental illness. The family history domain consisted of 14 variables reporting occurrence of family history of mental illness: family history of social anxiety disorder, social anxiety disorder-like symptoms, anxiety, depression, minor depression, panic disorder, neuropsychiatric condition, psychosis, bipolar disorder, dependence / substance abuse, suicide attempts and family history of completed suicide. These were assessed during the diagnostic interview by a clinician prior to treatment as part of the intake assessment process at the clinic in the form of a checklist. As such, the measure of family history was created for this study as a categorical variable. Treatment-related factors. The treatment domain consisted of 9 variables: adherence, operationalized as the number of activated treatment modules; treatment credibility, operationalized as the total score of the credibility/expectancy scale [44] where patients’ attitudes to the credibility of the treatment and expectancy regarding treatment effectiveness were rated on a 10-point scale (0 = not at all to 10 = very much) after the first week of the treatment; therapist time (hours); therapist time per module (minutes); patients’ logged time online; patients’ number of logins; patients’ number of mouse clicks; patients’ number of sent messages and patients’ number of posted messages on a common discussion forum within the treatment platform. Data Analysis Multilevel model of symptomatic change. When analysing predictors of symptomatic improvement during the course of treatment, a longitudinal multilevel framework was applied using the linear mixed-effects models procedure in SPSS version 21. All available data was used in a full intent-to-treat analysis approach with full maximum likelihood estimation procedures for all multilevel models. The objective of using multilevel modelling was to model change in social anxiety over the treatment period and to examine the influence of predictors on this trajectory. This was studied by testing the interaction effect of each predictor × time product; i.e. whether the effect of time on social anxiety varied depending on the value of the predictor variable. Due to the relatively large number of potential predictor variables, these were grouped into the four domains described above. As such, all variables were first tested in separate models before building a final model of change that included significant variables from all domains. Each domain of predictors were explored in a stepwise manner following the procedure adopted by Fournier et al. [27] and Amir et al. [11]. In Step 1, each model therefore included all variables of that domain, which were then screened in regard to their significance levels. Since the main interest of the study was in the factors that have an effect on the rate of symptomatic improvement, potential prognostic variables were required to have a significant interaction with time in order to progress to the next step. However, each consecutive model still included both the main effect (i.e. the effect on post-treatment level of social anxiety) of each added predictor as well as its interaction with time (i.e. the effect on the rate of change). Those that were found to be significant at p <. 2 in Step 1 were tested again in a second model (Step 2). Increasingly stricter significance levels were applied at each step and this screening procedure was repeated at p <. 10 (Step 3) and at p <. 05 (Step 4). Only those variables that remained significant at Step 4 (p <. 05) within each domain were included in a final model where the effect of each predictor was tested while simultaneously controlling for the effects of the others. Each multilevel model consisted of two levels, where Level 1 was the repeated measurements describing the linear change of social anxiety within patients over the course of treatment (a quadric change trajectory was also tested, but because it did not significantly describe the shape of change over time it was not included in the models) and Level 2 consisted of the patients with their socio-demographic, family history, clinical and treatment-related characteristics. In line with the approach described by Smits and colleagues [12], the time variable was re-centred at post-treatment; estimates of the intercept therefore reflect the predicted level of social anxiety at post-treatment. We also chose to recode the time variable between -1and 0 as proposed by Heck et al. [45]. This coding has the benefit that the estimated coefficients reflect the magnitude of change in social anxiety achieved over the entire treatment period (as opposed to the amount of weekly change). Therefore, since there were 12 measurement occasions, the distance between each step was 0.083; the time variable was therefore coded as -1, -0.92, -0.83, -0.67, -0.58, -0.50, -0.42, -0.33, -0.25, -0.17, -0.08 and 0. Finally, to facilitate comparison of coefficients for predictors measured on different scales, we followed the approach by Smits and colleagues [12] where predictor variables were standardized into z-scores prior to analysis. Analysing adherence as outcome. In the case of analysing adherence as an outcome, selection of variables to enter into the mixed-effects regression model was based on the same stepwise procedure described above. However, in the treatment factors domain, only treatment credibility and therapist time per module were tested since patient activity such as number of logins, mouse clicks, messages and time online were variables highly correlated with adherence (p <. 001).

Discussion The aim of the present study was to identify predictors of symptomatic improvement and adherence to ICBT for SAD within the context of routine psychiatric outpatient care. On average, patients improved at a rate of 20 points on the LSAS-SR during the treatment. Although this demonstrates significant symptomatic improvements, it is worth noting that the average post-treatment LSAS-SR score was approximately 50 (see Table 5), which is well over the LSAS-SR total score ≤ 30 criteria for remission from SAD as proposed by Mennin [46] and Rytwinski [47]. This indicates that most patients still had at least mild SAD after treatment. Treatment credibility and adherence emerged as significant predictors of a faster rate of improvement during the course of treatment, whereas those patients presenting with a higher overall level of functioning prior to treatment experienced a slower rate of improvement, possibly reflecting a floor effect. Further, treatment credibility and having earlier experience of SAD-like symptoms within the family was associated with increased treatment adherence, whereas ADHD-like symptoms, being male and having a family history of minor depression emerged as factors associated with lower adherence. Finally, the amount of time therapists spent with patients on each treatment module was negatively associated with adherence; however, the most likely explanation for this observation is simply that therapists spent more time with patients who had difficulties complying with the treatment rather than that therapist contact in itself would be a predictor of poor adherence. Probably the most interesting finding in this study was that patients’ beliefs and expectations towards the treatment seem to be more critical for a successful intervention than clinical or demographic factors. The association between treatment credibility and adherence might indicate that when patients feel confident that the treatment will be helpful, they may be more willing to engage in the treatment, consequently being more exposed to effective components of the CBT protocol (e.g. confronting feared thoughts and situations). Indeed, treatment credibility has previously been identified as a factor that plays a significant role in the effectiveness of ICBT interventions [13,14]. Although a high level of treatment credibility appears to be the most important prognostic factor for adherence, one must be cautious to conclude that this is a causal relationship, given that treatment credibility was not measured until the second week of treatment; it might have been influenced by early response which also seems to be a strong outcome predictor. The relation between early response and outcome of ICBT for SAD has recently been studied in detail elsewhere [48]. One final remark on treatment credibility is that since this study was done on a sample where participants had specifically applied for an ICBT interventions, it is interesting that the effect of treatment credibility still appear to be such a strong predictor. It would be interesting to compare its effect with a sample at a regular primary care clinic where patients have no preference on the delivery modality of CBT. Strengths and Limitations The main benefit of this study was the size of the sample and the routine care setting of treatment delivery. This increases the likelihood of the results being generalizable to similar clinical contexts were ICBT for SAD is being considered for dissemination. Additionally, outcome assessments were administered systematically and frequently and inclusion of patients spanned over several years. A discussion on the limitations of the study is warranted. Probably the most evident limitation is the lack of a comparison group, which introduces the inability to control for natural temporal trends as a potential explanation for the observed change in SAD symptoms occurring during the course of treatment. This is often a common issue in effectiveness studies in clinical settings. In this case, the aim of the study was to explore differences in treatment outcome within the context of a regular care clinic, effectively evaluating all patients who had received treatment for SAD over the course of a longer time frame. In such a naturalistic setting, there would have been additional ethical considerations of implementing a wait-list condition since there are also operational demands in regard to process lead times and improvement efforts of health care quality (e.g. decreasing time to wait between applying for treatment and receiving adequate care). Also, to measure adherence, we used the number of activated treatment modules, which means that we do not know whether patients actually completed the last activated module or not. This is apparently a limitation in measurement reliability. Since adherence in conventional face-to-face CBT typically refer to both in-session and out of session behaviour, the remote therapist guidance inherent in the ICBT delivery format makes it more challenging to have full insight into the actual (as opposed to reported) level of compliance in each module. Of course, tracking actual homework compliance in face-to-face CBT may be equally challenging, given the dependency on patients’ self-reports of out of session behaviour. Another limitation concerns the heavy reliance on subjective self-report data, since this could potentially introduce response bias. As was reported above, self-reported LSAS scores were systematically higher that clinician ratings. However, such a small mean difference (2.59 points) might not be clinically relevant. Further, clinician- and self-rated versions of the LSAS were found to be highly correlated (see also Fresco et al. [49]), which strengthens the assumed validity of the used self-report measurements. Finally, a note on generalizability of these results is that although the generalizability may be higher than would be the case of an RCT, these results might not apply to all primary care clinics settings, since patients included in this sample have actively sought out this treatment rather than being referred by a GP or primary care. Therefore, it might be argued that patients recruited in internet-based intervention studies may not be representative for the typical routine mental health care patient. Clinical Implications The findings of the present study highlight the importance of not only therapeutic compliance but also patients’ beliefs and expectations about the treatment. This study, then, raises the question of how to promote these factors. For example, could the provision of extra support to identified subgroups at risk of low adherence increase the probability of therapeutic compliance (and ultimately treatment response)? Although patients who showed signs of inactivity and lower adherence generally would receive reminders and supportive messages from their therapist in order to help them to stay engaged in the treatment, it is not clear whether these therapist behaviours significantly increased compliance. [14]. Indeed, the importance of therapist support in ICBT is not yet established [50]. For example, it has been observed in several studies of ICBT that more therapist support is associated with both increased adherence and larger treatment effects [51]. On the other hand, in a study where effectiveness and adherence was compared between guided and unguided ICBT for SAD, it was found that therapist guidance increased the likelihood of completing the treatment and that treatment effects were similar among those who completed the treatment regardless of whether they had received guidance or not [52]. Also, in a study comparing three levels of therapist guidance in ICBT for SAD, it was observed that the treatment was equally effective regardless of the amount of guidance patients received [50]. Although it cannot be concluded from the present study that the amount of time that therapists invest in guidance increases patients’ compliance or response to treatment in general, therapists might still be able to better support those patients seeking treatment for SAD who also presents with ADHD-like symptoms (e.g. by using similar organizational strategies that are used to aid patients with ADHD [53]).

Conclusions Prediction research is ultimately a tool to help clinicians make informed decisions prior to treatment. Results from this relatively large clinical sample indicate that expectation of treatment effectiveness and adhering to the treatment are critical factors for treatment success in ICBT for SAD. Clinicians might therefore benefit from early identification of probable “low-responders” and to test added measures that could increase therapeutic compliance and credibility. For example, clearly explaining the rationale for and communicating confidence in the treatment might increase patients’ willingness to engage and consequently increase the likelihood of treatment effectiveness. Also, since having ADHD-like symptoms seems to be associated with somewhat lower adherence, patients presenting with these additional challenges might benefit from extra support when working with CBT components central for treatment success. However, more research is needed in order to determine not only what type of support that would be most useful but also what level of guidance would be optimal in terms of providing effective treatments with efficient use of healthcare resources [54]. In sum, ICBT is an effective treatment for individuals suffering from SAD and high treatment credibility and adherence are strong predictors for greater rate of improvement. With the current dissemination of ICBT in regular care as an innovative and cost-effective way of increasing access to evidence based psychotherapy, the online delivery-format of ICBT may have several advantages over conventional CBT, especially for individuals with more severe social anxiety who might fear face-to-face interactions [6,54]. ICBT may therefore be a preferable treatment option for individuals who might otherwise avoid seeking help.

Acknowledgments We wish to express our great appreciation of Monica Hellberg for administrative assistance.

Author Contributions Conceived and designed the experiments: SEA BL EH VK EA CR GA NL. Performed the experiments: SEA BL EH VK EA CR GA NL. Analyzed the data: SEA BL EH VK EA CR GA NL. Contributed reagents/materials/analysis tools: SEA BL EH VK EA CR GA NL. Wrote the paper: SEA BL EH VK EA CR GA NL.