July 16, 2006

The politics of composite scales

The logjam is gone now, at least for chapter 2, thanks to a discussion with a classroom teacher (my wife). Talking the ideas over last night made me realize I had been dancing around the central dilemma of relying on tests: The same qualities that satisfy technical requirements for psychometrics distance us from knowing concretely what children do. As someone with statistical training, I know all the advantages of test construction that leads to a composite scale that ranks individuals in a consistent fashion. (Here, it doesn't matter particularly whether we are ranking individuals against each other or against a particular standard.)

Yet such a composite scale cannot be the basis for a transparent accountability system. As far as I am aware, no test currently used by any state—nor NAEP—can tell us how many 9-year-olds know their multiplication tables. We can quibble all we want about whether such a statistic captures what we want of children's knowledge of math—thus, the advantage of a composite scale—yet turning achievement into a composite scale creates a distance between what children do and what we know about their skills. Imagine for a second if we did have such knowledge: we'd probably debate whether knowing multiplication tables is a reasonable goal for 9-year-olds, where we should want automaticity in arithmetic, whether multiplication tables say anything about math as a more complex subject in general, etc. That debate would be raucous, splintered, and absolutely appropriate for a democracy. That we don't have such a debate about expectations impoverishes us and our schools.

Focusing on different aspects of testing—the sociology of knowledge (e.g., Noel Wilson's critique of psychometrics), how test construction provides an opening for test-prep, the political legacy of IQ tests, etc.—is important, and I will address several in the chapter, but I needed someone to push me on the central issues. Thanks, Elizabeth!

Listen to this article
Posted in Accountability Frankenstein on July 16, 2006 8:42 AM |