We analysed the progress of students using two primary dependent variables: number of ECTS credits earned (as described previously) and retention rate.

In line with our expectations, analysis of variance (ANOVA) between cohorts revealed (1) a significant increase for the intervention cohort in number of credits earned for male majority students [M control cohorts =33.32 (SD=20.45), M intervention cohort =40.90 (SD=18.04), Cohen’s d=0.39, F(1, 1506)=37.96, P=0.000] and (2) a significant increase in number of credits earned for male minority students [M control cohorts =26.42 (SD=20.01), M intervention cohort =37.95 (SD=20.49), Cohen’s d=0.57, F(1, 336)=19.85, P=0.000].

No significant increase was apparent, however, for either female majority students [M=40.77 (SD=19.09), M intervention cohort =42.87 (SD=20.80), F(1, 544)=1.10, P=0.294] or female minority students [M control cohorts =28.24 (SD=19.78), M intervention cohort =34.06 (SD=22.26), F(1, 216)=3.22, P=0.074].

Within the cohorts, separate univariate ANOVA revealed that the intervention reduced or virtually eliminated many of the academic achievement differences evident in the subgroups within the control cohorts. In the control cohorts, for example, there were significant main effects of gender [M male =32.07 (SD=20.54), M female =37.24 (SD=20.08), Cohen’s d=0.25, F(1, 2016)=16.30, P=0.000] and ethnicity [M majority =35.30 (SD=20.36), M minority =27.13 (SD=19.92), Cohen’s d=0.41, F(1, 2016)=71.55, P=0.000], as well as a significant interaction between gender and ethnicity [M majority male =33.32 (SD=20.45), M majority female =40.77 (SD=19.09), Cohen’s d=0.38; M minority male =26.42 (SD=20.01), M minority female =28.24 (SD=19.78), Cohen’s d=0.09, F(1, 2016)=6.03, P=0.014].

As predicted, however, the gender effect was no longer significant in the intervention cohort [M male =40.34 (SD=18.62), M female =40.25 (SD=21.56), Cohen’s d=0.01, F(1, 592)=0.23, P=0.633]. Furthermore, the effect for ethnicity was much reduced, although it remained significant [M majority =41.43 (SD=18.88), M minority =36.40 (SD=21.21), Cohen’s d=0.25, F(1, 592)=8.55, P=0.004]. The gender by ethnicity interaction was also no longer significant [M majority male =40.90 (SD=18.14), M majority female =42.87 (SD=20.80); M minority male =37.95 (SD=20.49), Cohen’s d=0.10, M minority female =34.06 (SD=22.26), Cohen’s d=0.18, F(1, 592)=2.12, P=0.146]. These results indicate that both the gender and ethnicity gap were reduced.

More detailed analyses (see Table 1; Figure 1) revealed that the remaining ethnicity effect in the intervention cohort was significant only between the majority male and females on the one hand, and the female minority students on the other. No significant differences remained between male and female majority students, between male and majority and minority students, between female majority students and male minority students, nor between male and female minority students (Fig. 1; Table 1).

Table 1 Comparison of differences between subgroups in number of credits and retention rate after Year 1 between the control cohorts (combined) and intervention cohorts Full size table

Figure 1 Number of credits (ECTS) earned after the first academic year by gender, ethnicity and cohort. (a) While the three pre-intervention control cohorts show a consistent gender gap, this gap closes almost completely in the intervention cohort, even though all students in the intervention cohort participated. (b) While the ethnicity gap seems to widen rather than close in the pre-intervention control cohorts, in the intervention cohort the gap closes significantly. (c) The interaction between gender and ethnicity shows that while both gaps diminish in the intervention cohort, the largest performance gain is achieved by male minorities. Full size image

Between cohorts, χ2 analysis with retention (percentage of cohort) as the dependent variable revealed that retention was significantly increased for the male majority students [Retention control cohorts =56.5%, Retention intervention cohort =72.9%, χ2 (1, N=1509)=29.57, P=0.000] and for the male minority students [Retention control cohorts =43.6%, Retention intervention cohort =67.1%, χ2 (1, N=338)=13.33, P=0.000]. There was no significant increase for the female majority students [Retention control cohorts =70.0%, Retention intervention cohort =77.2%, χ2 (1, N=546)=2.47, P=0.116]. Retention rate change among the female minority students was marginally significant [Retention control cohorts =44.6%, Retention intervention cohort =59.6%, χ2 (1, N=218)=3.59; P=0.058].

Within the cohorts, whereas retention rate differences between the pre-intervention groups were significant [Retention majority male =56.5%, Retention majority female =70.0%, Retention minority male =43.6%, Retention minority female =44.6%), χ2 (3, N=2017)=58.34, P=0.001], the differences in the intervention cohort were reduced below statistical significance [Retention majority male =72.9%, Retention majority female =77.2%, Retention minority male =67.1%, Retention minority female =59.6%, χ2 (3, N=593)=6.70, P=0.082]. This indicated that in the intervention cohort, the differences between the subgroups consisting of gender and ethnicity with respect to retention rate were significantly reduced. This was despite the fact that all subgroups in the intervention cohort improved to some extent, with groups that previously performed worst improving most. The increase in percentages academic performance and retention rate were impressive, ranging from 5 to 34% for number of credits earned and from 10 to 54% for retention rate (see Table 1).

Closing the gender and ethnicity gaps

Detailed analyses comparing each two subgroups within the control cohorts as well as in the intervention cohort further indicated that the goal-setting intervention reduced the gender and ethnic gap in retention. In the control cohorts, the differences between every subgroup were highly significant except for the difference between minority males and minority females. There were no longer significant differences between the subgroups In the intervention cohort, with the exception of small differences between majority males and females and minority females. However, even these differences were significantly smaller than in the pre-intervention cohorts (see Table 1; Fig. 2).

Figure 2 Closing of the gender and ethnicity gap as a function of academic year, for number of credits (ECTS) earned and retention rate. (a) With respect to gender, the gap in number of credits earned closes altogether after the first academic year, and increases slightly after Year 2. For ethnicity, the gap closes considerably after Year 1 and even more after Year 2. (b) With respect to gender, the gap in retention rate closes altogether after the first academic year, and opens slightly after Year 2. For ethnicity, the gap closes considerably after Year 1, and remains stable in Year 2. Full size image

After the first year, there was a marked reduction in the gender gap in performance with respect to the number of credits earned after 1 year. In the control cohorts, there was a difference of 5.17 ECTS between female (M=37.24 ECTS, SD=20.08) and male students (M=32.07 ECTS, SD=20.54), Cohen’s d=0.26. The intervention cohort difference shrank to a mere 0.09 ECTS between female (M=40.25 ECTS, SD=21.56) and male students (M=40.34 ECTS, SD=18.62), Cohen’s d=0.01, for a reduction of 98.25% (Fig. 2a). To check if this effect was lasting—that student performance did not decline during Year 2—we calculated the difference in the number of credits earned in Year 2. Analyses showed that this was essentially equivalent for the control cohorts (the difference was 3.68 ECTS) and the intervention cohort (the difference was 3.49 ECTS); a reduction of 5%, meaning that the gap did not widen in Year 2.

Furthermore, there was a marked reduction in the gender gap with respect to retention after 1 year. In the control cohorts, there was a difference of 8.6 percentage points between female students (Retention=62.8%) and male students (Retention=54.2%). The post-intervention retention difference shrank to 0.2 percentage points in the intervention cohort (Retention female students =72.0%; Retention male students =71.8%), a gender gap reduction of 97.67%. Thus, the goal-setting intervention virtually eliminated the gender gap in retention and number of credits earned (a reduction of approximately 98%). The effect on retention also seemed to be lasting. In the control cohorts the difference between female (59.3%) and male students (48.6%) was 10.7 percentage points after 2 years. In the intervention cohort, by contrast, the difference with regard to retention rate between female (68.0%) and male students (66.8%) was 1.2 percentage points (a gender retention difference of 1.8%). This is a reduction of 88.78% (Fig. 2b).

The ethnic performance gap in number of credits earned and retention was also reduced significantly in the intervention cohort. In the control cohorts, after 1 year, there was a difference of 8.17 ECTS between the majority (M=35.30, SD=20.36) and minority students (M=27.13, SD=19.92), Cohen’s d=0.41. In the intervention cohort, by contrast, there was a difference of 5.03 ECTS (Credits majority M=41.43, SD=18.88; Credits minority M=36.40, SD=21.21), Cohen’s d=0.25, a gap reduction of 38.43%. After 2 years, in the control cohorts, there was a difference of 3.74 ECTS between the majority (M=47.84) and minority students (M=44.10). In the intervention cohort, the majority students underperformed slightly in comparison with the minority students (by 0.27 ECTS) (Credits majority M=48.62; Credits minority M=48.88). This was a change of 107.2% (Fig. 2a).

The retention in the control cohorts, after 1 year, differed by 16.1 percentage points (Retention majority students =60.1%; Retention minority students =44.0%). In the intervention cohort, by contrast, the difference was 10.0 percentage points (Retention majority students =74.1%; Retention minority students =64.1%), a reduction of the ethnicity gap by 37.88%. After 2 years, in the control cohorts, retention rates differed by 16.4 percentage points (Retention majority students =55.2%; Retention minority students =38.8%). After 2 years, retention in the intervention cohort differed by 10.7 percentage points (Retention majority students =69.5%; Retention minority students =58.8%), a reduction of 34.75% (Fig. 2b). Overall, the difference regarding gender and ethnicity seems to diminish significantly and disappear altogether with respect to gender after the first academic year (Fig. 3).

Figure 3 Retention rate after the first academic year by gender, ethnicity and cohort. (a) While the three pre-intervention control cohorts show a consistent gender gap, this gap closes almost completely in the intervention cohort, despite the participation of all students in the intervention cohort. (b) While the ethnic minority gap seems to widen, rather than close, in the control cohorts, in the intervention cohort the ethnicity gap closes significantly. (c) The combination of gender and ethnicity shows that both gaps diminish in the intervention cohort, while the largest gain in retention is evident in the male ethnic minorities. Full size image

The goal-setting intervention therefore appears to have closed the ethnicity gap by approximately 38% in both retention and number of credits earned after 1 year. The ethnicity gap took somewhat longer to close than the gender gap, taking 2 years instead of 1 to close almost completely, by 93%.

Additional analyses

There is often a gap between important goals that people have set and their actual goal attainment (Webb and Sheeran, 2007). A significant body of research has shown that the relationship between goal setting and performance is mediated by factors such as attention to goal-relevant activities, persistence and the discovery of task strategies to facilitate goal achievement (Zimmerman and Schunk, 2001). Locke and Kristof (1996) showed, for example, that students who achieved higher grades tended to use well-specified study methods and often completed all their assigned work. These students seem to have established the more specific achievement goals that typically leading to better performance than vague or general goals, such as try to “do your best” (Locke and Latham, 2002).

The current goal-setting intervention was aimed at getting students to reflect on their general, higher-order life goals, prioritize them, form implementation intentions and monitor goal attainment. Presumably, self-regulation becomes easier when the specifics of course work were viewed by students in the context of such globally important, broader life goals. Indeed, the study of Morisano et al. (2010) showed that it was participation in the goal-setting programme, per se, rather than the number of academic goals specified, that was important in relation to academic achievement. We also attempted to determine what more specific factors might have produced these changes in number of credits attained and rate of retention.

University rules governing our participant students allowed them to skip regular exams and wait for later, programmed exam re-sits, instead. In the first year, students can take a maximum of 12 regular exams, but are allowed to defer these until later, if necessary. These exam-related rules allow for flexibility with regards to sudden, unexpected life events, but also enable counter-productive avoidance behavior and procrastination. Since diploma completion is an important goal for most students, they must stay on track, instead of procrastinating (for example, waiting for the re-sits instead of taking part in regular exams; cf. Bayer et al., 2010). Thus, we hypothesized that any decrease in the number of re-sits taken by the students after completing the intervention might be a marker for increased commitment to achievement and career (reflected in a decrease in procrastination). We also hypothesized that struggling students, in particular—males and ethnic minorities—would be better able to prioritize their goals, after completing the intervention, and would therefore be inclined to participate more regularly in the study programme, avoiding re-sits or other means of postponing exams.2

Univariate ANOVA indeed revealed that in the control cohorts there was a significant gender effect [M male =10.32 (SD=2.79), M female =10.84 (SD=2.30), Cohen’s d=0.20, F(1, 1740)=11.99, P=0.001] and a significant ethnicity effect [M majority =10.67 (SD=2.45), M minority =9.74 (SD=3.25), Cohen’s d=0.32, F(1, 1740)=41.05, P=0.000] in relation to number of exams taken. There was no significant interaction effect between gender and ethnicity [M majority male =10.49 (SD=2.62), M majority female =11.16 (SD=1.85), M minority male =9.56 (SD=3.37), M minority female =10.01 (SD=3.05), F(1,1740)=0.45, P=0.503].

In the intervention cohort, however, neither the gender effect nor the ethnicity effect remained significant: [M male =10.44 (SD=3.01), M female =10.05 (SD=4.03), Cohen’s d=0.11, F(1, 574)=1.64, P=0.201] and [M majority =10.47 (SD=3.10), M minority =9.83 (SD=4.06), Cohen’s d=0.18, F(1, 574)=3.66, P=0.056]. Similar to the pre-intervention cohorts, the gender ethnicity interaction effect in the intervention cohort was not significant [M majority male =10.52 (SD=2.77), M majority female =10.34 (SD=3.88), Cohen’s d=0.05, M minority male =10.12 (SD=3.88), M minority female =9.40 (SD=4.33), Cohen’s d=0.18, F(1574)=0.58, P=0.445].

Detailed analyses (see t-tests in Table 2) showed that in the control cohorts there were significant effects between all subgroups, except between majority males and minority females and between minority males and minority females. In the intervention cohort no significant differences remained between any of the subgroups. This may help explain, practically, why the gender and ethnicity gap closes after the intervention: the groups of students that performed worse in previous, pre-intervention cohorts now take exams at a rate equivalent to the previously higher-performing groups. This suggests that these groups are now characterized by enhanced self-regulation. As a result, their academic integration increased (Rienties et al., 2012).