I just watched John Oliver eviscerate standardized testing on “Last Week Tonight.” Everyone should watch it. As usual for the show, it’s spot-on and hilarious. It also got me thinking. There seems to be a fundamental flaw in how we measure effectiveness. Let’s take George W. Bush’s No Child Left Behind act that really pushed standardized testing over the edge.
One of the initial motivators for NCLB was the overall poor performance of US students on the PISA tests. These are international test that try to gauge overall performance on basic competencies like math and reading. US students were ranked near the bottom of those tests when George W. Bush announced his No Child Left Behind program. They are still at the bottom.
NCLB mandated increased testing and tied it to school funding based on student performance. This meant that schools had to stop teaching a broad curriculum and focus on preparing students for the barrage of tests just to survive. Ok, now this seems a plausible solution . . . ok, it’s not even superficially a plausible solution.
The NCLB approach to standardized testing relies on the tests being accurate measures of an individual student’s progress. The increasing performance disparity between students with different cultural or economic backgrounds certainly questions the accuracy of those tests but so does common sense.
Standardized tests are poor measures of individual performance precisely because they are standardized. In any population of students, there will be varying comprehension of topics. Standardized test don’t measure shades of gray so they will never tell you if a particular student understood 90% of a problem or 10%. They don't measure partial understanding. They also don't measure if the student slept well or ate breakfast or got in a fight on the bus to school. All these things have an effect on performance.
Even for fundamental concepts that should be a sort of “you get it or you don’t” situation the test can’t be sure that the student missed a question because they didn’t understand the concept or because they didn’t understand the question. Imagine a student whose first language is not English attempting a simple math problem phrased in English sentences. Did the student miss it because they didn’t understand the math or because they didn’t understand the grammar.
But I think all of this misses the fundamental flaw. The tests aren’t measuring competence. They are measuring test performance. Is that the goal of our education system? If so, it is failing. But I don’t want test performance to be the goal of our education system.
Shouldn’t we be measuring real world performance? Shouldn’t we prepare students for jobs or for college? Measures for success in those areas already exist. Measure how many students graduate and get jobs. Measure how many students go to college and complete a degree. Then measure how many of those students get jobs. Those are the real goals, aren’t they?
Of course we need measurements along the way to find areas where we need to focus or improve our approach but don’t tie those intermediate measurements to funding. Use those to get better not to punish schools and teachers.
Let me try an analogy. If you make a pizza and it doesn’t taste good, you can’t just adjust the taste the next time around. The taste is result of a complex procedure involving the quality of the ingredients, the process of combining them, cooking procedure, and personal preferences sprinkled throughout. What you would do is break it into the necessary steps and ingredients. None of these intermediary steps will have the taste of the final pizza at that moment, but if you judge each part individually and improve where appropriate, you will likely improve the taste of your final pizza. The flour used to make the crust in no way tastes like the final pizza but the choice of flour is essential to the final outcome.
Education is far more complex than making a pizza but I think the principle is the same. If you monitor the process throughout and adjust the individual components, you might get the results we all want, happy and successful members of society . . . and better pizza.