June 29, 2006

Testing as technology

I've spent my book-time today pondering the nature of testing as technology. For the first time, I'm using concept-mapping software (CMAP), because, well, I haven't been able to get my mind to think about this rigorously, and anything's worth a shot.

So it's been fruitful. One thought in my head before today, which I knew was incomplete, was the technical focus on consistency, consistency among different levels of objects (content standards and item specification, item specification and items, items and total tests) as well as the type of consistency we think of as the technical term reliability. Item response theory (which I only grasp in a general way, having had no practice in it at all) is a tool in service to this consistency.

Trying to wrap my head around this made me think of the obvious criticism of this focus on consistency—the way our real-life skills are not consistent, despite the conformity of tests to this consistency standard. But that doesn't really touch on the core tensions between technologies (such as testing) and democratic politics (no matter what your political theory). Consistency is more a matter of minimum quality, a key concept in the same way that optimization is a driving force in engineering. It doesn't really tell you anything about the politics of technocracy.

So there are two thoughts that are running through my head this evening, after this exercise:

  • The control and oversight in testing has some interesting professional characteristics—internal accountability (within an organization) is critical, the standards are technical, market demands shape behavior (even if the buyers are generally public agencies), and the public legitimacy of testing is crucial to the survivability of the whole enterprise.
  • I need to discuss political-science notions of the iron triangle of industry, Congress, and regulation.

I have nothing more useful or synthetic right now, other than just wanting to jot these down to ponder overnight. Tomorrow is my son's last morning in the summer chess camp (more about chess and history after that's done), so I'll spend the morning in some place trying to ponder/write, pick him up, and then head to campus for a few errands. But this is enough to ponder.

Listen to this article
Posted in Accountability Frankenstein on June 29, 2006 10:18 PM |