While driving, have you ever attempted keeping your vehicle in your lane by looking exclusively through the rear-view mirror? I hope not. If you do, you are probably destined for a collision. Rear-view mirrors are great for seeing objects behind you, and for seeing where you have been. They are not useful for seeing where you want to go.

Managing a company by means of the monthly report is like trying to drive a car by watching the yellow line in the rear-view mirror. ~Myron Tribus

Donald Wheeler, in his book Understanding Variation: The Key to Managing Chaos, references this quote from Myron Tribus comparing the absurdity of managing using statistics from the past. Attempting to manage or lead a school by numerical goals based on the previous year’s test results is like exclusively using your rear-view mirror to drive. Yet, this is a commonly accepted practice in industry as well as education. During the summer months, as school administrators plan for the upcoming school year, a comprehensive analysis of the previous year’s data is performed. A needs assessment is completed. From the previous year’s data and the needs analysis, a campus improvement plan is created, complete with goals for the upcoming school year. Teachers analyze their own data from the previous school year and make plans for the future. This is so common, I do not know if we have ever questioned if this was the best way to improve. And, I admit, as a teacher and administrator, I did the same thing.

While in grad school, I remember learning how to read a school’s annual report from the Texas Education Agency. These reports would detail a school’s STAAR (state standardized test) results, as well as other important school related data. At first, when attempting to create a needs analysis and plan for improvement, I questioned whether I should consider test data from the previous school year for the next school year. After all, the students are different. Could I make plans for the current students based on the previous year? Should I look at the current students’ previous data? But, then, I would be looking at a different test. It was confusing. I thought this was a very imperfect way to make improvements. Here are some issues with the way we currently use data.

Using test data from the previous year describes what happened last year regarding students from last year. The data tells us nothing about this year. We need more information. Data from the previous year does nothing to predict improvement of future outcomes. If anything, from last year’s data, if the system or processes are stable and do not fundamentally change from the previous year, we should expect similar results.

Enumerative vs Analytic Studies

This past year I discovered enumerative and analytic studies. In both Out of the Crisis and The New Economics, W. Edwards Deming described both types of studies. Knowing and understanding the difference between enumerative and analytic studies helped me understand why I had trouble with improvement efforts based on data from the past. John Hunter from his post on the Deming Institute’s blog provides an excellent explanation of enumerative and analytic studies. Hunter describes an enumerative study as “focused on judgment of results. These studies are meant to enumerate (explain, evaluate, describe) the condition that exists with the existing population.” This would describe standardized testing data from the previous school year. Deming used the example of a census.

In contrast to an enumerative study, Hunter explains “an analytic study is focused on improvement of the process which created the results and which will continue creating results in the future.” The focus on an analytic study is the process and what happens when we make some change to a process. In short, enumerative studies focus on results, while analytic studies focus on what produces results, the process.

I encourage you to read the short blog post from John Hunter about enumerative and analytic studies. Also, the post contains a short video of Mike Tveite explaining how Deming referenced enumerative and analytic studies and their purposes. First Deming said, “The purpose of any statistical study is to provide a rational basis for taking action. The distinction between types of studies revolves around where the action will be taken.” Tveite explained that in an enumerative study, the action is taken on what you are taking data from. In an analytic study, action is taken on the process.

My intention is not to disparage enumerative studies. School data from last year is important. They tell us what happened. Enumerative studies do nothing to improve the current process, which is an analytic problem. We need analytic studies where action is taken on the process producing the results. This is process improvement. If we want to develop meaningful campus improvement plans, then we should move from setting goals based on enumerative studies to process improvement using analytic studies.

How does one start an analytic study? You may not be aware of it, but you already have experience with them. It is pretty simple. Here are some examples:

While following a favorite recipe, you might try a different ingredient or change the process of preparing the food for a better texture or taste. While driving to school, you try a different route to see if you arrive earlier. Or, because of traffic, you leave at a different time to see if there is a difference in commute time. While mowing the yard, you try a different path of cutting the grass to see if you can increase efficiency. In your class, you test different ways of distributing materials to find the most efficient and least disruptive process.



Plan-Do-Study-Act

By changing our practice from setting goals using static enumerative statistics to dynamic analytic studies using Plan-Do-Study-Act (PDSA) cycles, we can make improvements and even innovate. Here’s how a simple PDSA works:

Plan – If we make this change, we believe this will happen… Do – Make the change Study – Did the change improve results? Why did the change make a difference? If the change didn’t make a difference, why? Act – Do we incorporate this change into our process? Do we continue as before with no change in the process? Do we try something else? Start cycle again with the Plan stage, continually improving the process.



I just described a very simple and incomplete explanation of PDSA. The book Improving Learning: A How-To Guide for School Improvement by Michael King and Jane Kovacs provides a practical, in-depth guide to improving schools through the PDSA process. In my opinion, every school should buy a copy and use it to improve their systems and processes. David Langford is also a leader in school improvement using Deming’s System of Profound Knowledge. He provides seminars leading participants through the PDSA cycle. His book Tool-Time is an easy to use and valuable resource for learning and improving processes through PDSA cycles. It is a reference I have used repeatedly, and in my opinion, schools should have multiple copies. Tool-Time and other information about David Langford’s services can be found on his website http://www.langfordlearning.com/.

Improvement of Process is the Difference

The use of analytic statistical studies to improve an organization is one characteristic that separates Deming’s System of Profound Knowledge from the traditional, widely-accepted style of management (dependence on enumerative studies). Deming continually shed light on the process of production, rather than the end results. This helps me understand why he asked the question “by what method” when most managers are focused on the bottom line. For educators, it is important to know whether the statistics we are viewing explain the outcomes of where we have been or pertain to where we are going (process improvement). Managing by outcome is rear-view mirror driving, while managing by process improvement is forward-looking. This understanding is a huge step toward developing profound knowledge.

Remember, if you want to take your school or classroom to the next level, you can’t improve by using rear-view mirror thinking. Follow Dr. Demings 5th Point of his 14 Points for Management: Improve constantly and forever the system of production and service to improve quality and productivity.

Share this:

Tweet





Print



Like this: Like Loading...