November 19, 2006

Carrots and sticks in education

I'm now more convinced than ever that discussions of consequences in K-12 accountability systems happen in bunkers, without enough understanding of how things work together (or don't).

But first, a digression about literature searches. I'm in the middle of writing chapter 4 of Accountability Frankenstein, a chapter tentatively titled "Consequential Thinking," about the consequence systems that make accountability high-stakes. I have my own historian's spin on this, but the general arguments I've read in various places are heavy on management jargon, moderate on cognitive-psych jargon (extrinsic vs. intrinsic motivations), and very thin on research in fields that should take this on (industrial-organizational psychology and industrial-relations research). There's something about this impression that struck me as wrong, so tried to find anything I could in "I-O" psychology to bring to bear. (Odden and Kelley's 2002 book discusses theories of motivation, drawn from I-O and other realms, but they're an exception, I'm not sure how many policymakers have that book, and I have a few concerns about that chapter that I won't get into.)

Consider, for example, goal-setting theory, which posits that establishing goals is crucial for motivating most people on their jobs. But how do I search for that in the literature on teacher pay or school probation? Goal setting is a generic enough term (in contrast to expectancy theory--see one MBA-ish website for a brief explanation, and then please me a better URL in comments!), and looking for that was frustrating. 

But there's a marvelous trick to use: citation indices. The most well known is ISI's Web of Science, which includes the Science Citation Index and the Social Science Citation Index. There's also a citation tracker in Google Scholar. The most common use of citation indices some years ago was to find out who cites whom, in terms of influence on a field of study. (Junior faculty going up for tenure generally do this in the year before going up.) But there's another use: looking for literature that cites a germinal article. The most recent "classic" article on goal-setting theory is Locke and Latham's "Building a Practically Useful Theory of Goal Setting and Task Motivation" (American Psychologist, 2002). Is there anything with teacher pay, merit pay, performance pay, or similar terms, which cite this article?  Yes.  Marsden and Belfield's Pay for performance where output is hard to measure: the case of performance pay for school teachers (a book chapter published this year) cites the Locke and Latham in the literature review.

The fact that few in the area cite this work (or a 1990 book by the same authors) is troubling.  There's the Odden & Kelley book and there's also a 1998 Heneman article on Charlotte's school-based performance pay system. Heneman's use of the 1990 book is especially problematic:

Moreover, borrowing from goal-setting theory (Locke & Latham, 1990), characteristics of the student achievement goals may help shape expectancy perceptions in terms of intensity, focus, and persistence. In particular, goals that are perceived as meaningful, clear, specific, and challenging will foster high expectancy perceptions by teachers. (p. 45)

Unfortunately, there's also a body of literature contrasting goal-setting for simple tasks (i.e., can unionized truck drivers get higher loads on their trucks, to save a company time?) and the more complex tasks involved in teaching. My understanding is that while goal-setting is important, its effect depends dramatically on context. Distant goals are less useful than proximate ones, and goals involving complex skills may be better to combine with specific goals targetting acquisition of strategies to accomplish the distant, slightly vague goals in a complex area.

My conclusion: the performance-pay advocates are even more in isolated bunkers than I expected, with folks not looking closely at relevant literature bases elsewhere. And that doesn't even touch the question of whether anybody discusses carrots and sticks in combination.  But that's for another entry...

Listen to this article
Posted in Accountability Frankenstein on November 19, 2006 2:07 PM |