When the Hawaii Department of Education released the details of its new teacher evaluation system three years ago, veteran teacher Mireille Ellsworth made a radical decision: She would simply refuse to do part of it.

Like many teachers in the state, Ellsworth felt that linking teacher pay — even partially — to student test scores was unfair. But there were other portions of the complex and multi-tiered system that she objected to as well, including the use of Student Learning Objectives as a measure of teacher success.

“I could tell it was something that could be easily manipulated by any teacher,” Ellsworth said. “Essentially it would be a dog and pony show.”

Mireille Ellsworth

This year — the first that the state’s Educator Effectiveness System impacts teacher pay — Ellsworth received a subpar rating because of her incomplete evaluation. She appealed the rating. And she won.

The DOE says Ellsworth won her appeal because of a procedural error in how her school handled the review, not because of her detailed objections to the system itself. But Ellsworth is still hoping her experience makes a larger statement to teachers.

“It really was a personal action against the system,” Ellsworth said. “It was intentional, and I really hoped it would inspire others to stand up not just for this, but for anything in education they think is wrong.”

Ellsworth’s appeal is just the latest pushback against an evaluation system that has been scaled back several times in the last three years, but remains contentious among local educators.

In fact, the new leaders of the teachers union campaigned on a goal of getting rid of the EES entirely.

Others say attacking the EES, which the DOE spent years developing and millions of dollars implementing, could have repercussions for teachers in the future.

Ahead of National Movement

Teacher evaluations — particularly ones that link pay to student test scores — have long been a thorny national issue.

Despite union concerns, a growing number of states have moved to implement high stakes teacher evaluations, in part because of federal incentives under grant programs like Race to the Top.

Studies have shown that while myriad factors impact student achievement — from poverty and parental involvement to class size and even nutrition — access to an outstanding teacher might be one of the biggest determinants.

“Currently, there is Hawaii BOE policy that mandates we have an evaluation system that has student growth and achievement as well as teacher practice.” — Deputy Superintendent Stephen Schatz

The U.S. Department of Education’s push for better evaluations also stems from a 2009 report by The New Teacher Project called “The Widget Effect,” which argued that in most districts teacher performance was not measured or recorded in any meaningful way.

In the 12 districts surveyed for the report, less than 5 percent of teachers were given unsatisfactory ratings, and few if any tenured teachers were dismissed for performance-related issues.

“What that report did was quantify what we knew anecdotally, which was that teacher evaluations were pretty broken,” said Sandi Jacobs, policy director at the National Council on Teacher Quality. “They were very checklisty, and not very meaningful and not doing a lot to identify if a teacher was really high performing or low performing.”

Since the report came out there’s been a huge national shift in teacher evaluations, Jacobs said, with nearly all states now including student achievement.

Most states are still pretty early in the process of implementing these new evaluations, however.

Hawaii is one of a very few states at the point where “the rubber is starting to meet the road for positive or negative consequences,” Jacobs said.

Union Was ‘at the Table’

Hawaii started working on the EES in 2010 as part of its Race to the Top grant application.

The system was initially piloted in 81 schools from 2011 to 2013. Although complaints abounded from teachers about a lack of information on the evaluation plans, the DOE conducted numerous stakeholder meetings with teachers and administrators to garner input into the final system.

“Teachers had the opportunity to be at the table,” said former Hawaii State Teachers Association Vice President Joan Lewis. “We had less input on construction of evaluation, but more input on the protection of teachers.”

The EES is based on two categories: teacher practice, and student learning and growth. Within those categories, teachers are judged using classroom observations or portfolios, student surveys, test scores, and Student Learning Objectives or SLOs.

Department of Education / screenshot

“At its core, the Educator Effectiveness System is about professional growth,” Deputy Superintendent Stephen Schatz explains in a video for teachers on the DOE site. “It’s a foundation from which teachers collaborate, reflect upon data and apply learning to enhance instructional practice.”

Based on the various components in the evaluation system, teachers can receive one of four ratings: highly effective, effective, marginal, and uneffective.

One of the teacher protections that HSTA negotiated in its contract with the DOE was the right to an expedited appeal for tenured teachers who received a marginal rating.

Hawaii is one of a very few states at the point where “the rubber is starting to meet the road for positive or negative consequences.” — Sandi Jacobs, National Council on Teacher Quality

The DOE rolled out the EES to all schools in 2013-14, and then made 18 revisions aimed at simplifying the process before 2014-15, when the evaluations were linked to teacher pay.

Earlier this year the DOE also agreed to conduct the reviews every other year for teachers who received an effective or highly effective rating — unless their supervisor flags them for review because of problems such as repeated parent complaints.

“EES is a comprehensive evaluation system that sets clear expectations for effective teaching, provides educators with quality feedback and support to improve their effectiveness with students, and informs professional development,” DOE spokeswoman Donalyn Dela Cruz said in an email.

Despite efforts to streamline the process, it remains complex enough to require a 54-page handbook for participants and evaluators.

An Act of Rebellion

Ellsworth, who teaches English and drama at Waiakea High School in Hilo, has a slew of objections regarding the EES. The 18-year teacher’s biggest beef though is with the Student Learning Objectives or SLOs, which she refused to complete two years in a row.

For the SLOs, teachers are asked to predict the growth or achievement of each student — something they can then come back and revise mid-semester. Ellsworth felt it was a student privacy violation for this student data to go into her personnel file, and said the data could easily be manipulated by teachers.

“It’s just an exercise in trying to justify your existence and pass it no matter what,” Ellsworth said.

She had philosophical objections to the SLOs as well.

“If a teacher has low expectations for a student, research has shown that student will perform at a lower rate,” Ellsworth said. “For me to put on paper and then in my professional portfolio online that I expect anything short of success is completely wrong and is against everything I’ve been taught.”

“Do you really want to keep investing that level of resources in a system that gives you the same results that previous ones have done for a fraction of the cost?” — Joan Lewis, former teachers union vice president

It is, she said, like committing “educator malpractice.”

Jacobs, from the National Council on Teacher Quality, disagrees.

“The idea that your mission and accountability target need to be the same thing is kind of silly,” Jacobs said. “I don’t think anyone is saying if your SLO target is 80 percent that you are writing the kids off.”

Ellsworth, who said she sent her administrators articles throughout the year supporting her objections to the SLOs, received a marginal rating at the end of the 2014-15 school year. The lack of SLO data was the only reason she was rated as less than effective, her principal testified at the expedited hearing last month.

Her appeal was heard by two representatives from the HSTA and two from the DOE. During the meeting, Ellsworth presented nine objections to the system, including that the EES is failing to meet its mission of guiding her in her practice, that it causes “inequitable loss of instructional time,” that no principal-directed professional development plan was implemented for her as required, and that SLOs are neither valid nor reliable and require “psychic abilities” to complete.

As evidence to support her claims, she points to a 2013 report by the U.S. Department of Education’s National Center for Education Statistics that suggests districts may want to hold off on making SLOs a part of teacher evaluations, in part because they are difficult to make valid and reliable.

“They are by definition customized to individual teachers and based on the professional judgments of teachers and principals,” the report stated. “Making SLOs an important component of high-stakes evaluation could undermine their validity, because it means that teachers are in essence grading themselves.”

This year 13 teachers went through an expedited appeals process after receiving a marginal rating, Dela Cruz said.

Of those 13, 10 were changed to “effective” because of “procedural errors.” None of the appeals set any precedent for future teachers, Dela Cruz said.

It will be a few more months before the 2014-15 results are finalized and released, but if last year was an indicator, most teachers will pass with flying colors. In 2013-14, nearly 97 percent of teachers in the state were rated as effective or highly effective — something Ellsworth and Lewis point to as a possible sign that the system is not worth the amount of time and money invested into it.

“I do think there is a legitimate question to be asked of the state, given so many teachers are rated effective or better,” Lewis said. “Do you really want to keep investing that level of resources in a system that gives you the same results that previous ones have done for a fraction of the cost?”

But for teachers to try and walk away from the EES now, Lewis said, would be a mistake.

What’s Next?

The new HSTA leaders have been vocal in their objections to EES, and listed eliminating the EES as one of their campaign goals.

DOE leaders meanwhile, emphasize their commitment to the system.

“Currently, there is Hawaii BOE policy that mandates we have an evaluation system that has student growth and achievement as well as teacher practice,” Schatz said in an email. “We also have a collectively bargained agreement with the teachers union. Finally, we do have a commitment to (the U.S. DOE) that we will continue to implement our system. We are committed to doing this well.”

Without being renegotiated into the next teachers contract, the EES will sunset after the 2016-17 school year, Lewis said.

That’s why she says that trying to eliminate the EES before then could be harmful to teachers if they lose their seat at the table for deciding what replaces the EES.

“We are in a position to say we want a say in designing our evaluations,” Lewis said. “That’s not something employees have the opportunity to do.”

The biggest argument against the EES in coming years, could be the high percentage of teachers who pass it with flying colors. In any teacher evaluation system, the percentage of teachers ranked as in need of improvement should be in the double digits, said Jacobs of the National Council on Teacher Quality.

Last year in Hawaii, fewer than 4 percent of teachers were deemed marginal or ineffective.

“The fact we have a 90-something percent effective rating in Hawaii is meaningless and is giving a false sense of security,” Ellsworth said.