Comparison of students’ pre- and post-instruction BEL survey results

Students had a statistically significant increase (p < .05) in BEL Survey index scores from the pre-instruction survey (M = 70.11, SD = 6.97) to the post-instruction survey (M = 71.72, SD = 8.80). This would seem to indicate that students possessed fewer biological evolution misconceptions following instruction as opposed to prior to instruction - a result to be expected if students’ misconceptions were supplanted by accurate concepts during the teaching process. On closer examination, however, this was not the case. The total number of students’ misconceptions increased by 260 following instruction, from 4,812 pre-instruction misconceptions to 5,072 post-instruction misconceptions. The discrepancy between the positive change in student BEL Survey mean index scores and the increase in mean number of misconceptions from pre- to post-instruction can primarily be accounted for by a statistically significant (p < .01) reduction in the mean number of ‘undecided/never heard of it’ responses from pre- to post-instruction (3.92 to 2.15), coupled with statistically significant (p < .01) increases in the mean number of pre- and post-instruction ‘strongly agree’ responses (4.20 to 4.88) and ‘strongly disagree’ responses (4.05 to 4.80; see Table 5). Although a slight majority of those students who selected the pre-instruction ‘undecided/never heard of it’ response subsequently selected the accurate post-instruction statement concepts, thus elevating the BEL Survey mean index score, a slightly smaller number selected the statements’ misconceptions, which resulted in an increase in the total number of misconceptions from pre- to post-instruction.

Although students were obviously more confident in their responses following instruction, this new-found confidence was inversely correlated to their competency in the subject matter. This finding replicates similar results identified by multiple researchers (for example, Bishop and Anderson [1990]; Cunningham and Wescott [2009]; Wilson [2001]). This increase in confidence was predominately a female phenomenon, as the average decrease in ‘undecided/never heard of it’ responses from pre- to post-instruction BEL Survey was 0.74 per female student (n = 287) but only 0.38 per male student (n = 249). This outcome may be because female students were more indecisive in their initial pre-instruction survey statement responses, producing a mean 4.25 (n = 287, SD = 4.01) ‘undecided/never heard of it’ response rate for the 23 BEL Survey statements as opposed to male students’ mean 3.54 (n = 249, SD = 3.61) response rate, a significant difference of t(534) = 2.14, p < .05. Such a phenomenon was likewise documented in both Almquist and Cronin’s ([1988]) and Cunningham and Wescott’s ([2009]) studies.

Additional evidence indicating an increase in student confidence was found in the comparison of students’ biological evolution knowledge self-rating scoring means pre- and post-instruction. Students rated themselves to be more knowledgeable about biological evolution post-instruction as opposed to pre-instruction. Again, however, the confidence gained did not correlate to increased competency in subject matter.

Researchers have observed that students are able to recognize the scientifically acceptable answer when a statement is phrased correctly, such as BEL Survey statement 2, for example, ‘The scientific methods used to determine the age of fossils and the earth are reliable’. However, when a statement is put forth that includes a common misconception, such as BEL Survey statement 1, ‘A scientific theory that explains a natural phenomenon can be classified as a “best guess” or “hunch”’, students tend to agree with the misconception (Almquist and Cronin [1988]; Cunningham and Wescott [2009]). The present study did find a similar trend in students’ responses with correctly phrased statements (n = 10) accounting for a mean post-instruction student misconception rate of 198.30 (SD = 52.10) out of a possible 536 student responses, whereas statements that included a common misconception (n = 13) produced a mean student misconception rate of 237.62 (SD = 84.93). To Cunningham and Wescott ([2009]), such a trend suggested that, ‘while our students may have heard the scientifically accurate definition of terms such as theory, fitness, and natural selection, they do not truly understand them’ (p. 514). These researchers strongly concur.

Student variables

Previous studies have shown that student misconceptions about science can differ significantly based on multiple variables including geographical region, religious background, generation, gender, and age (Almquist and Cronin [1988]; Losh et al.[2003]; Morrison and Lederman [2003]; Palmer [1999]). This study found no significant difference between the mean difference in the number of students’ pre- or post-instruction biological evolution misconceptions when related to students’ gender, ethnicity, grade level, biological evolution knowledge self-rating, or to the students’ public high schools’ urban-centric locations or ADM. With these variables minimized as contributing factors to student acquisition of biological evolution misconceptions, the focus then turns to the role of the teacher.

Teacher variables

Even though the difference proved to be outside the realm of significant (p = .08), it is interesting to note that students of male teachers had a 232.8% increase in the mean difference in the number of misconceptions from pre- to post-instruction over that of students of female teachers. This result could simply have occurred because female teachers in this study (n = 17) appeared to be more knowledgeable of biological evolution, with a mean BEL Survey index score of 94.40 (SD = 13.69) compared to 89.2 (SD = 14.6) in men (n = 18), and mean misconception rate of 4.29 (SD = 3.87) compared with 5.83 (SD = 3.78) in men. This explanation appears to be valid based on the results obtained when teachers’ BEL Survey index scores were ranked regardless of gender, divided into either a HISG (n = 18, M = 103.11, SD = 5.72) or a LISG (n = 17, M = 79.64, SD = 9.74), and then compared to the mean difference in students’ pre- and post-instruction BEL Survey index scores and number of misconceptions. Students whose teachers’ BEL Survey index scores fell in the HISG generated a BEL Survey mean index score increase of 2.58 (n = 290, SD = 9.25) from pre- to post-instruction and a 0.20 mean increase in number of misconceptions (SD = 3.48) whereas those students whose teachers’ BEL Survey index scores were in the LISG produced a BEL Survey mean index score increase of only 0.47 (n = 246, SD = 8.40) coupled with a 0.82 mean increase (SD = 3.23) misconceptions. Thus, while teachers’ gender may play a role in students’ acquisition of biological evolution-related misconceptions, a more important factor appears to be the biological evolution knowledge possessed by the teachers themselves. Certainly additional research is warranted in this area.

Whereas significant differences (p < .05) were discovered in the mean difference in number of students’ pre- and post-instruction misconceptions between students whose teachers possessed either bachelor’s or master’s degrees and students whose teachers possessed doctorate degrees, one must proceed with caution. Only 43 of the 536 student participants were students of teachers possessing doctorate degrees (n = 3), representing only 8.0% of the student population, whereas 169 (31.5%) were students of teachers possessing terminal master’s degrees (n = 10), and 324 (60.5%) were students of teachers possessing terminal bachelor’s degrees (n = 22). No doubt, larger sample sizes of teachers possessing doctorates, along with their students, are required to verify the results of this study. It is interesting to note, however, students (n = 324) of teachers possessing terminal bachelor’s degrees had a mean increase of 0.27 (SD = 3.36) misconceptions following instruction as compared to a mean increase of 0.45 (SD = 3.28) misconceptions for students (n = 169) of teachers possessing terminal master’s degrees. Results of this study indicate that such a difference in students’ mean number of misconceptions from pre- to post-instruction may be more closely tied to the teacher’s bachelor’s degree field than to terminal degree level as previous research has revealed that teachers’ understanding of content is nearly directly correlated with their education (Hoy et al.[2006]; Pajares [1992]). Students of teachers possessing science education, non-biology science, and biology bachelor’s degrees had mean misconception increases from pre- to post-instruction of 0.08 (n = 125, SD = 3.04), 0.11 (n = 122, SD = 3.46), and 0.57 (n = 195, SD = 3.34) respectively, whereas students of teachers possessing non-science bachelor’s degrees had a mean pre- to post-instruction increase of 1.50 (n = 180, SD = 3.71) misconceptions. Significant differences (p < .05) revealed between the mean difference in numbers of pre- and post-instruction misconceptions between students of teachers possessing either science education or non-biology science bachelor’s degrees and students of teachers possessing non-science bachelor’s degrees indicate that students’ numbers of biological evolution-related misconceptions are more likely to increase from pre- to post-instruction if they are taught by teachers lacking science-related bachelor’s degrees. Approximately 54.5% (n = 12) of those teachers with terminal bachelor’s degrees (n = 22) held either a science education or non-biology science degree compared to only 30.0% (n = 3) of those teachers possessing terminal master’s degrees (n = 10), and 0.0% of those teachers holding doctorate degrees (n = 3), while their students produced mean pre- to post-instruction misconception number increases of 0.27 (SD = 3.36), 0.45 (SD = 3.28), and 2.21 (SD = 3.39) respectively.

No significant differences were discovered in the mean difference between students’ numbers of pre- and post-instruction misconceptions when related to their teachers’ years of teaching experience, teachers’ rating of emphasis placed on biological evolution in their college courses, or teachers’ self-rating of biological evolution knowledge. However, statistically significant differences (p < .01) in the mean difference between students’ numbers of pre- and post-instruction misconceptions were related to the number of hours teachers dedicated to teaching biological evolution concepts in the Biology I course, with 6 to 10 hours of teacher instruction identified as the optimum duration. Although 6 to 10 hours of evolution instruction did not appear to reduce the number of misconceptions students brought into the classroom, this duration did seem to inhibit their development as compared to shorter or longer durations of instruction where the number of students’ misconceptions increased from their initial levels by levels higher than the 0.01 mean increase afforded by the 6 to 10 hours of instruction. Interestingly, one teacher in the study indicated dedicating no hours to the teaching of evolution in the Biology I course yet produced the most favorable student results. This teacher’s students (n = 17) had a mean decrease of 1.0 (SD = 2.98) misconceptions from pre- to post-instruction BEL Survey while presumably lacking any teacher instruction. Of course, the accuracy and quality of the evolution instruction is no doubt more important than the quantity of time spent teaching evolutionary concepts in the classroom.

BEL survey statement analysis

Especially enlightening were those results obtained when the 35 teachers’ BEL Survey index scores were ranked from highest to lowest, subsequently divided into two groups - the HISG and the LISG - and the mean change in both groups’ students’ BEL Survey index scores and number of misconceptions from pre- to post-instruction were analyzed. Results revealed that from pre- to post-instruction, those students of teachers classified in the HISG produced a significantly higher (p < .01) BEL Survey mean index score coupled with a significantly lower (p < .05) mean number of misconceptions than did those students whose teachers were classified in the LISG (keeping in mind that the mean number of misconceptions increased for both groups of students from pre- to post-instruction). These results indicate that students of teachers who possess a relatively better knowledge of biological evolution have an increased opportunity to learn and retain biological evolution-related concepts. While the data do not indicate a concurrent reduction in evolution misconceptions, these students have an opportunity to minimize the number of new misconceptions acquired during the course of instruction as opposed to those students taught by teachers with a relatively poorer knowledge of biological evolution concepts. Studies repeatedly show the positive impact effective teachers can have on student achievement. For example, both Sanders and Horn ([1994]) and Marzano ([2003]) revealed a 39.0 percentage point difference in student achievement gains between students with most effective and least effective teachers (as cited in Miller [2003], p. 2).

When each of the 23 BEL Survey statements was independently analyzed to determine which teachers possessed the associated misconception and which did not, followed by analysis of the mean change in their students’ numbers of misconceptions from pre-instruction to post-instruction, the data revealed conflicting results (Table 9). Analysis revealed ten BEL Survey statements in which the mean positive change in the number of misconceptions held by students whose teachers adhered to the statement misconception were greater than the mean positive change in the number of misconceptions held by students of teachers who did not possess the statement misconception. There were likewise ten survey statements in which the mean positive change in the number of misconceptions held by students whose teachers possessed the statement misconceptions were less than the mean positive change in the number of misconceptions held by students of teachers who lacked the misconception. Only one of the 10 survey statements that were revealed as possible contenders for the transmission of the statement misconception from teacher to student did so at a statistically significant level. This statement, number 20 (‘There exists a large amount of evidence supporting the theory of evolution’), produced a 0.22 (SD = 0.59) increase in the mean number of misconceptions from pre- to post-instruction in those students whose teachers possessed the misconception as opposed to a 0.01 (SD = 0.59) decrease in students whose teachers lacked the misconception. At the p = .0001 level of significance, there exists a high probability that this particular misconception was passed from teacher to student. Were other misconceptions passed from teacher to student? Most likely, as several other BEL Survey statements teetered on the brink of statistical significance (Table 9) but only statement 20 crossed the line at the p < .05 level of significance.

Even though analysis identified only one BEL Survey statement misconception with a high probability of being transmitted from teachers to their students, the study did reveal an important relationship between teachers’ levels of misconceptions and student achievement which may provide additional evidence of misconception transmission from teachers to students. Results suggest an inversely correlated relationship between the number of teachers’ misconceptions and students’ post-instruction BEL Survey index scores, that is, as the number of teachers’ misconceptions increased, students’ BEL Survey post-instruction index scores decreased and, as the number of teachers’ misconceptions decreased, students’ BEL Survey post-instruction index scores increased. Transmission of misconceptions from these teachers to their students cannot be ruled out as a causative agent although several variables may come into play in the decrease of students’ post-instruction BEL Survey index scores following instruction by teachers with high levels of misconceptions as evidenced by the fact that Pearson’s r2 indicated that only 2.9% of students’ post-instruction BEL Survey index scores were predicted by the number of teachers’ misconceptions.

Limitations of study

Several possible limitations were evident in this study. For example, all teacher participants volunteered for the study and therefore are probably not a truly random sample of all Oklahoma public high school Biology I teachers. Similarly, during the duration of the study some students may have been exposed to biological evolution misconceptions in non-biology courses or in other contexts. In addition, for some variables tested, small teacher and/or student sample sizes may have produced results that were not representative of the population as a whole. In light of these limitations, a completely causative link between students’ acquisition of biological evolution misconceptions and the variables defined within the study is not assigned. Nevertheless, evidence suggests that the data reported here are reliable and representative, and the results are consistent with those reported by previous researchers (for example, Almquist and Cronin [1988]; Bishop and Anderson [1990]; Cunningham and Wescott [2009]; Wilson [2001]).