November 21, 2006

The last chapter problem

In Improving Poor People (1997), Michael Katz wrote:

Historians and other social scientists who offer interpretive accounts of social issues always face a "last chapter" problem. Readers expect them to extract clear lessons from history, offer unambiguous recommendations, and foresee the future. My standard response—my role is to analyze and explain the problem; I have no special expertise in devising solutions—although honest, rarely satisfies. When historians tack on a set of conclusions, more often than not they appear utopian, banal, not very different from what others have suggested, marginally related to the analysis that precedes them and far less subtle. The reason, of course, is that no set of recommendations flows directly from any historical analysis. Understanding the origins and dimensions of a social issue can lead in very different policy directions. (p. 7)

As someone (Groucho Marx?) once said, I resemble those remarks! I've spent 80% (or more) of my current book-in-progress analyzing high-stakes accountability from different perspectives (often historically rooted), and I'm now in the last chapter. Do I repeat what Katz said and wash my hands of any specific recommendations? I can't. I'm too deeply into this and, what's more important, a significant part of my argument is that test experts have no business trying to decide what a democratic process should craft.  To say that I demur because I am not an expert would be hypocritical! So I will take a citizen's and not an historian's right to make recommendations, however rooted they are in my sense of humanity's quirks and the institutional and political legacies we have inherited.

However, Katz's warnings about utopianism, banality, and the disconnect from the rest of the book are well-warranted. I have no magic charms against banality, but I can take a few steps against the others. After returning home bleary-eyed after 10 pm last night, I told my spouse I had just spent a few hours skimming over the chapters already drafted so I could be consistent.  She nodded, "Readers might have a few concerns if you're essentially making a new and completely different argument in the last chapter." 

And to make sure that I don't step towards utopianism, I will describe three utopian accountability mechanisms that will not appear in the book. Correction: One does appear in the book, largely to explain why it wouldn't work. (Why these are utopian is left as an exercise for the reader.)

  • A recursive system based entirely on formative assessment: teachers analyze student data formatively, then principals analyze teachers formatively using how teachers use data formatively, and those over principals analyze principals formatively using... you get the picture.
  • A high-tech way of finding out what students are working on: sample the written work of five students in each grade daily.  Have a random draw of students in the morning, get them to turn in the previous evening's homework and anything completed that day, cover up their names and the teachers' names, scan their written work, and upload it to a central server that's entirely public.
  • TeacherCam: A video camera in every classroom and in the hallways, allowing the public to see what happens anywhere in any school.

Now that I've gotten that out of my system, it's time to write about something that's workable.

Listen to this article
Posted in Accountability Frankenstein on November 21, 2006 8:48 AM |