A reader, Charlene Williams, who holds a Ph.D. in clinical psychology, sent the following comment in response to this post about the vocabulary used on one of the Common Core tests:

This speaks to one of the essential issues in the current high stakes testing debacle. Why the Pearson, PARCC, and Smarter Balanced testing is unscientific and unethical. I am a psychologist, faculty at UCLA, and a mother in California. I hadn’t heard about these concerns with the current high stakes testing, until after I became very concerned with the developmental level of the SB practice items when helping my daughter (dutifully prepare for the tests).

The 6th grade ELA practice performance task for the Smarter Balance was completely inappropriate for 11-12 year olds, requiring them to toggle between several screens (on small Ipad screens), and choose multiple pieces of evidence to evaluate, select, paraphrase, compare and contrast, as well as write a multiparagraph essay. Never mind that while practicing, toggling back to the articles caused the students’ written work on the essay to be erased (lost).



Why the current high stakes testing is unscientific:

1) There is no proven Construct Validity (does your test measure what you think it measures)

2) Cut scores are determined by an unknown (arbitrary) process- labeling children as proficient, or failing appears to not be based on any scientific process. It is not scientific to arbitrarily decide what levels of your test scores actually mean in the real world. Scientific measurement requires cross-validation with external measures that provide evidence for your claims (like grades, or independent in-depth measures of children’s educational achievement in a a smaller sample with highly experienced evaluators).

3) Computer adaptive tests- there have been many concerns raised about how item difficulty has been decided. Children continue to progress on these tests if they continue to get a certain number the most recent answers correct. Educational measurement specialists (true academically trained professionals) and parents and children have observed that very often items following very difficult questions are significantly easier. This raises concerns that children’s scores are artificially deflated by unscientifically determined item difficulty determinations.

4) Inter-rater reliability- No checks exist to independently determine whether the scoring administered by these testing companies has truly reliable and valid measurements of children’s answers (see Todd Farley http://www.bkconnection.com/static/Making_the_Grades_EXCERPT.pdf )

Most importantly, the Pearson, PARCC, and Smarter Balanced testing is unscientific because they violate the basic rule of science. The assessments are not verifiable, because they are not permitted to be subject to independent scientific evaluation. Their validity cannot be proven nor disproven. Under the guise of “test security” companies use copyright laws so extreme they prevent true scientific evaluation of the validity of these tests, by scientists with expertise in the fields of Education, Psychology, and related fields.

So I am deeply concerned that the profit-driven testing business is using unscientific (and expensive) testing which is portrayed to the public as if it’s truth, with high stakes ramifications on children, teachers, and our public education system. As stakeholders and parents, we need to demand accountability, real science, and an ethical separation between profit-driven educational businesses and the true scientifically-based education and measurement. For the sake of our children, our teachers, and our educational system which is truly one of the foundations of our democratic country.