September 29, 2008

On Wendy Kopp, TFA, and Linda Darling-Hammond

Back in June, I wrote a long entry on Teach for America and Linda-Darling Hammond's critique of the Kopp organization and model. I had been puzzled at the claim by Kevin Carey and others that Darling-Hammond simply hated TFA with the type of bile that is usually attributed to Karl Rove, Bill Belichek, and others with a take-no-prisoners approach to civic life and sports.

I don't recall who e-mailed me and pointed me to the 1994 Kappan article on TFA by Darling-Hammond, and the description of its aftermath in Kopp's book. But I went back and read the article carefully, then the relevant passage by Kopp. And I will freely and openly admit that I was wrong: I now know why some describe LDH as having a visceral opposition to TFA. I think the description is wrong, but it's understandable enough, since a vivid conflict often is frozen in people's memory as an enduring symbol of a relationship. I'm sure Frank Zappa and Tipper Gore quickly got tired of being asked what they thought of the other's latest initiative, life events, whatever. But because they clashed over the labeling of popular music, that became etched in people's memories. (Well, the memories of some of us.)

This summer we've had another Kappan issue focusing on Teach for America, with both Darling-Hammond and Kopp contributing. TFA has been around long-enough, with enough scars and criticisms of it, that I can make some long-term observations. I suppose that it is the unique prerogative of an historian to live long enough that he or she can proclaim that, no, I wasn't ignoring things; I was just waiting for the dust to settle.


So let me start with some general observations about worldviews: in the late 1980s and early 1990s, Darling-Hammond and Kopp worked from two very different views of teaching and teachers. (I'll do my best to present their perspectives from the best vantage point.) For Darling-Hammond, teaching is inherently a complex occupation, with the best teaching full of nuanced judgments that require deep and complex knowledge. The consequence of this perspective would be requirements for teachers to have a good deal of content knowledge, a good deal of pedagogical knowledge, and a great deal of what has come to be known as pedagogical content knowledge (or a repertoire of how to teach specific subjects).

In contrast, Kopp worked from the assumption that the greatest gap in poor districts is an insufficient supply of young, enthusiastic teachers with a minimum threshold of intellectual authority. The consequence of this perspective would be her initial recruiting model for Teach for America: the "best and brightest" new graduates from the liberal arts. Later, she acknowledged that teachers do need some basic pedagogical skills, and TFA's greatest public challenge over the past two decades has been getting its recruits up to speed fast enough to survive their classrooms (and let their students survive, too). 

One irony of these perspectives is that each operates in her professional life with the way the other views teachers. Darling-Hammond's professional life at Teachers College and then Columbia is full of the type of intellectual authority (refereed publications, confirmed recognition of her colleagues, a connection to a top-notch facility) that Kopp asserts is necessary for teachers. For Kopp, her work as a social entrepreneur is absolutely full of the type of occupational complexity that Darling-Hammond claims is the life of a great teacher. Of course, being a professor and the leader of a non-profit is not the same thing at all as being a teacher. But I am a bit surprised that neither of them has said, "Well, in some ways I have the qualities that my opponent thinks is necessary for teachers. Let me explain why that is the wrong perspective on teaching, from my role that is removed from the K-12 world."

Darling-Hammond has been consistently skeptical of TFA's activities, but she has moved from her early 1990s writings that portrayed TFA simply as an almost fraudulent organization (as in the 1994 article) to a more careful focus on the new-teacher issues (in her research on student outcomes). The 1994 article relies on a considerable amount of anecdotal evidence to portray TFA as sloppy and possibly quite dangerous to schoolchildren. Darling-Hammond has continued to be skeptical, but since I have started reading this stuff (from about 9 years ago or so), I don't recall anything in print where Darling-Hammond has veered away from a fairly strict focus on, "Okay, let's look at what's happened from the data..." You may or may not agree with her conclusions, but is there anyone who likes TFA's work who thinks that's a bad focus? (As I've noted elsewhere, I am not sure that is the only potential value in TFA, but it's a legitimate question.)

As far as I can tell, Kopp's focus for the last 15 years has been on organizational growth, shifting much of her efforts to shoring up organizational operations, especially fundraising, recruitment, and connections with districts. Several times, TFA has reworked their programs for supporting recruits after placement, but my sense is that's still in flux, while the other pieces are more stable.  

To be fair to Darling-Hammond, I think she had some evidence to support the claim that TFA was an organizational mess in the early years, and Kopp has pretty much admitted as much. Looking at what was available in print in 1993 and 1994, TFA's reputation for shoddy work really was fair game. The recent audit of TFA's use of federal funds may raise those questions again, but the point is that it wasn't an obviously wrong concern. The occasional problems with TFA's organizational reputation is inconsistent with Kopp's enterpreneural reputation as a go-getter and someone who cares about poor children. I'm not surprised it's that image clash that raises the hackles of TFA supporters; Kopp's brand is as a social entrepreneur, not someone who runs alternative certification programs. 

I think Kopp may have fed a bit of Schadenfreude here, because she helped propagate the myth that TFA tells us anything about teaching in general, that TFA is a model for the New Teaching. Instead, if she had focused on less millennial and more defensible claims, that stopgaps are ethically defensible and bolster the public system, she probably would have found a more ready audience among those who should recognize the value in finding new ways of bolstering public support for the public sector.  That's a missed opportunity, I think.

The organizational woes of TFA should tell us something, but it hasn't been discussed much among the social entrepreneurial crowd or the critics of TFA.  Let's suppose for the moment that TFA's fans are absolutely correct, that TFA really is a new model both for recruiting new teachers and also for generating social entrepreneurs. Given what we know about TFA's organizational history, that means that one of the most successful social entrepreneur organizations required almost a decade for this Great Hope to become a competent organization. We should be skeptical that any similar Great Hope could become competent in a shorter period of time.

Listen to this article Listen to this article
Posted in Education policy at 10:35 PM (Permalink) |

September 27, 2008

Both Fish and Bérubé are wrong

Some years ago, I ran across someone who was so firmly convinced that schools were heterosexist, he thought that K-12 teachers should be forbidden from mentioning anything about their private lives lest they reinforce heteronormative assumptions. I asked, "Okay, so that means you can't have a picture of your spouse or children on your desk?" "Of course not!" was the reply. That took my breath away, and I was thinking of asking whether we should just give up this parental childrearing idea entirely and have state-run creches. But I thought better of my time and his and just shook my head and walked away.

That type of foolishness has its parallel in higher education with the biennial arguments about Bumper Stickers and Buttons. Along with the foolishness this week in Illinois whereby faculty and staff were told they could not have political bumper stickers on cars they parked on campus (All faculty must leave their classes right now and scrape the "Harry Potter for President" stickers off their cars, or so I imagined), I received an e-mail from a colleague asking about candidate buttons worn on campus. I explained the usual distinction between public and private resources—you can't use public property to support candidates, but I assume faculty buy their own clothes, so they're festooning personal property—and the distinction between sense and propriety. Not everything that is unwise is unprofessional: you're not going to impress your students if you wear a huge McCain or Obama button, but telling a faculty member not to wear campaign buttons is a violation of a faculty member's rights. Yes, faculty and students have rights to do foolish things as well as brilliant things.

And, yes, I included both faculty and students in that statement. When he was on campus Tuesday, Michael Bérubé said that students do not have academic freedom and that he agrees with Stanley Fish's argument that academic freedom is a guild concept. Because I agree with Bérubé on a great deal in terms of academic politics, in some ways it is a relief to find something on which we disagree; otherwise, I'd worry that I was a figment of his imagination. (Please don't explain in comments that he could surely imagine someone with whom he disagrees and thus I am still a figment of his imagination. I know that argument, it ignores the ineffability of English professors, and I'm just holding onto this thin reed of intellectual autonomy as is, so will you stop with the Jesuitical reasoning already?)

More seriously, Fish's argument is an understandable but narrow view of academic freedom, and despite what he thinks, it is weak ground on which to make the case for academic freedom.


Fish asks, Is academic freedom a philosophical concept tied to larger concepts of individual dignity and autonomy, or is it a guild concept developed in an effort to insulate the enterprise from the threat of a hostile takeover? That's a great start, a combination of a false dichotomy and straw-man argument. Apart from the fact that there are arguments in favor of academic freedom that are not rooted in either a priori concepts of intellectual freedom or guild protections, though, using the term guild is not very specific. This is fairly typical of Fish's ex cathedra pronouncements of Academic Truth, full of elisions that make me want to tear my hair out.

Fortunately for my sanity, if nothing else, Michael Bérubé put flesh on Fish's frisson in his talk Tuesday. He argued that Fish's guild concept was rooted in the academic's search for truth, whose path is unpredictable. Because of that unpredictability, faculty could not be restricted in the direction their inquiries took. Faculty are confirmed in their expertise, so they get this freedom. Students are not, so they don't have academic freedom.

This sounds like a clean distinction until you poke below the surface. Do I have academic freedom because I engage in research but my colleagues who are just instructors do not have academic freedom because they don't publish? Wait: maybe we let teachers have academic freedom because you never know where class may go in a field like mine. So do instructors have academic freedom in the humanities but not in calculus, because intro calc is well defined? Or suppose you tie it to the stability of the job because you don't want some full-time faculty to be excluded or have there be arguments about which field has academic freedom. Then you have the question of whether full-time faculty have academic freedom but adjuncts don't. What about graduate students, who are learning but also teach and engage in research? Ah, but they're not yet confirmed experts. But in some fields doctoral students commonly publish before their dissertation, while in other departments new assistant professors sometimes are hired as ABDs without publications. So does the ABD and unpublished assistant professor have academic freedom at a university where the published advanced doctoral student doesn't? Or suppose you have a doctoral student at a university who also teaches and has tenure at a nearby community college. Does she have academic freedom or not? According to the guild concept, she might have it when at work at the community college (where she has tenure), but not at the university, even though her work at a university may contribute more to the body of knowledge in her field. If your brain is about to explode from these problems, follow my advice: don't root academic freedom in a guild concept.

The other problem with the guild notion of academic freedom is its political viability: today, not only is it dangerous to imply that faculty should have academic freedom while you don't because we're special, it fails a basic reality check. A high enough proportion of the general population has a college education that we just aren't that special. Maybe only one percent of the American population has a Ph.D., but we've done a pretty darn good job of educating our neighbors so that they can think for themselves. That's a good thing, on the whole. Maybe you're not a trained scientist, but some of you participate in the annual Christmas bird count, or you're an amateur astronomer, or you know Lilium columbianum when you see it. For me to claim that only I have the academic freedom to be protected when I talk about those things while you don't is guilding the lily (the Tiger lily, if you're curious, though I can't guarantee I could spot it in a field). When defenders of academic freedom use arguments that are as fallacious as they are pretentious, they are not helping defend the professoriate from political interference.

A far better route is to take part of Bérubé's commentary on Fish—that academic freedom is rooted in the job we do—and expand the way we look at the job of faculty and universities. Maybe Stanley Fish thinks the academic is interested in an abstract, decontextualized search for truth (see Steven Kellman's Chronicle column for a nice response to that claim), but many of the historical academic freedom controversies are rooted firmly in politics. I suspect that for those whose academic freedom was violated thanks to the economics of the dairy economy or the politics of the Cold War, Fish's defense of them as only in search of the (defenestrated, lifeless) truth would be cold comfort. We may academicize the world because that's the modus operandi of analysis, but we can be motivated by the same passions as our neighbors.

The search for truth isn't as ascetic as Fish would hope. It is emotional, personal, and often a matter of sensitive politics. As higher education has evolved in the U.S. and elsewhere, college and university faculty look for truth and are general social critics. The rhetoric and reality of academic freedom is a political construct, tied to our institutional role as social whistleblower. Sometimes that's "social" in an ascetic-truth sense, and sometimes it's social in a very political sense. To divorce faculty from the development of political rights in American history is to ignore the real history of academic freedom controversies and the growing recognition of general free-speech rights. Of course, Stanley Fish doesn't believe in free speech, either. But I do, I bet you do, and that means that we can and should talk about academic freedom in a political context.

To make that case means that we have to acknowledge that students have academic freedom in an institutional context (i.e., when they're at a public university). If we tell students that they have no academic freedom, we're inviting them to care less about the academic freedom of faculty once they leave us. If we invite them into the sphere of protection we'd like enlarged, they'll be far more likely to support academic freedom as older adults. So for all sorts of selfish and historical reasons, I hereby proclaim that college students have academic freedom, and it's a good thing, too.

Listen to this article Listen to this article
Posted in Academic freedom at 7:32 PM (Permalink) |

September 26, 2008

In formidonis

A week ago or so, my daughter's composition class teacher assigned an essay that was clearly linked to the history class: not an essay, but Jonathan Edwards's Sinners in the Hands of an Angry God (1741), the quintessential First Great Awakening sermon. I am not religious, certainly not Christian in any of its flavors, yet I think I recognize the emotion Edwards was trying to awaken in his listeners: terror.

Terror was the emotion that would drive listeners to convert, to join Edwards as a parishioner.

Terror is the term that we apply to those who would settle their grievances by wreaking violent havoc on the world.

Terror is the emotion that President Bush exploited to justify an ill-planned invasion and occupation of Iraq.

Terror is now what millions of Americans face in uncertain economic conditions.

Apart from questions that HR departments and unions have fielded about pension plans, what are the consequences for schools?


The major historical work on the Great Depression and education is Tyack, Hansot, and Lowe's Public Schools in Hard Times (1984), and it easily filled the historiographical space of "what we think we know about the subject but haven't yet seen written down in one place." Schools were hit fairly hard in the Depression, as were all public services—teachers had their share of payless paydays in cities that had insufficient revenue to meet payroll. That's on the downside. But the Depression was also the era when the majority of teenagers began attending high school for more than a collective smudge of time. The Depression did not entirely cause the switch in teen behavior: for several decades, children had been increasingly excluded from full-time work either because of the growth of child labor laws, employers' decisions that they didn't want underage workers, or in the 30s, the Agricultural Adjustment Administration, which encouraged Southern landowners to kick tenant and sharecropping farmers off the land. (The growth in mechanization of farming in the postwar era, together with a shift in cotton growing from the South to California, essentially finished the job as far as the 20th century trends were concerned.)

With few alternatives, high schools became the institution that absorbed millions of teenagers who would probably not have attended school at the same age in the 1920s. So, as the late Edward Krug has noted, though the federal government had New Deal programs that looked like they could be the institutions of youth (the NYA, WPA, CCC), public schools easily rebuffed that in the late 1930s and became an institutional reservoir for teens. The extent to which that's good is an open question: I would love to see a debate on that point between Claudia Goldin and Jeffrey Mirel.

The larger point, if there is one: don't assume that the educational consequences of an economic downturn are easily predictable. Some are: budget cuts are likely to be severe in many places. But the twists and turns of schools can surprise us. I'm not going to go all pollyanna and suggest that retrenchment will "enforce efficiency." That's happy talk b.s. (before scrutiny). But there are going to be surprising consequences from our current crisis. I'm not sure what they're going to be: when I received my history Ph.D., I got a rear-view mirror, not a crystal ball. But you don't need a history Ph.D. to know that something interesting in education will happen in the next year that would not have happened without the financial crisis.

Listen to this article Listen to this article
Posted in Education policy at 12:57 PM (Permalink) |

September 25, 2008

Epistle to the late night

Dear evening,

I very much enjoy your company at times such as this evening, when I love to unwind at the end of a day and contemplate ideas that I usually do not have time to.

But that's not this evening. You may notice me at 1 am with a stack of student papers, not relaxing, not finishing a glass of wine and a book, and wonder at my behavior. Certainly, if I must rise a little over 4 hours from now, what in the WORLD am I doing now?

It's called reading student work. It's long past the hour at which I think my students would want my reading their work. Nonetheless, if I do not return their papers tomorrow, two disasters will befall the world. First, they will go more than a week without feedback. Second, I will then have two sets of papers to read, since they are turning in another batch tomorrow morning at 8 am sharp. In both cases, I will be both the victim and the perpetrator of the crime. I cannot complain about the quality of papers when I do not give feedback, and I cannot complain at the workload when I assign papers and then fail to read them promptly.

And now, if you will excuse me for my brief rant and then disappearance, I must get back to the papers.

Listen to this article Listen to this article
Posted in The academic life at 1:10 AM (Permalink) |

September 23, 2008

Michael Bérubé Tuesday

I'm too exhausted right now to think too much about Bérubé's visit today. I know that his synapses work at least three times as quickly as mine do, he and I disagree about the definition of cultural studies (and at and on this point I think he's right), and we also disagree about the underlying rationale for academic freedom (and I think I'm right about that). But all of that is for another day, I think (or rather that I will think on another day). You can listen to his interview on the student radio station, at least until I have my brain back in working order.

Until then, I will just leave you all pondering the fact that I held up a sign at the airport while waiting for him that had "DANGER" written on it, and no TSA officer ever came up to me to ask what the heck was going on. Well, you don't have to ponder it at all, if you don't want to. It's not Bérubé's fault, if you're wondering.

Listen to this article Listen to this article
Posted in The academic life at 11:21 PM (Permalink) |

Critical thinking and cultural work

I have another hour or so of work to do before bed, out of a combination of weekend-long computer woes, an uncooperative body, scheduling near-misses, and a delayed plane. But as a result of Michael Bérubé's visit this week, I've been thinking about What's Liberal about the Liberal Arts? and his discussion of his classes. I know that his explicit intention is to show how a liberal professor can teach (and usually does teach) literature without using it as an excuse to propagandize, because the other issues swamp anything that might stem from a professor-as-policy-liberal as opposed to a professor-as-procedural-and-intellectual-liberal.

But there are a bunch of other things in there, and one of them is how classroom discussion is cultural work. A seminar discussion about The Rise of Silas Lapham involves a great deal of give and take between students and faculty and among students. I get the sense from reading Bérubé that he works very hard to engage students and push them to think, about the book and related ideas about literature and humanity.  And students work hard as well. I suppose somebody might say that they're engaged in critical thinking, but that's wrong on several levels. At one level, it's wrong from the perspective of cognitive psychologists who have tried but failed to identify the modules that are connected to this mysterious entity. That doesn't mean that there is no such thing as critical thinking but that it may not be what we think it is, or we have to look at it differently. 

So, back to the students who are struggling to grasp what a Penn State English professor is saying. He's pushing them to examine the implications of Silas's ethical choices, force them (the students, readers) to decide what's right and wrong, to make connections. And they begin to (or so MB describes, and I have no reason to doubt his account). It's not a brilliant eureka moment that stems from cognitive growth, or at least not in any coherent sense that my friends the cognitivists can point to. But there is something going on, in the classroom space that has discussion, open questions, leading questions, pushy questions, pushback, and occasionally silence. Hundreds of thousands of students go through that process each semester; they may not go through it with Silas, and their epiphanies may not be original except to them and their classmates, but in the type of classroom that I hope all of us experience at least once, they do a type of work that can only happen in or with groups: cultural work.

Yeah, yeah, Peter McLaren wrote that a few decades ago, I know: the classroom is a performance space. But I mean something a bit deeper and more problematic: some of the best opportunities for cultural work is in a functional, engaging classroom. For a whole variety of reasons I won't go into detail about, beyond cognitive psychology, I am very skeptical of broad generalized claims about critical thinking when posed as a cognitive-psychology question. Usually, that turns the college curriculum into a sort of faculty-psychology jungle gym, much as the 1824 Yale Report claimed in its defense of a curriculum. But there is stuff going on in a good liberal-arts classroom, and that's inherently hard to capture because cultural work can simultaneously be local and universal, even at the mundane level of the individual, personalized classroom discussions that are going on about Moby Dick this fall, not at one university but at hundreds. To put it in a concrete sense, there are probably hundreds of students in different high schools, colleges, and universities who are talking this week about the fact that "The Cassock" (chapter 95, I think) is about the disposal/use of the whale's penis and foreskin, either giggling or being taken aback at it. Widepsread, but very personal and local. 

I think this cultural work is what distinguishes a liberal-arts college from lots of other educational experiences. I think it is why the Amethyst Initiative signatories are disproportionately from liberal-arts colleges: Despite the research suggesting how the 21 age threshold for alcohol saves lives, and in addition to the legal/political liability issues, liberal-arts college presidents are less devoted to a certain definition of "cognitive thinking" than to a common sense that college is for discursive, social learning. 

I have still been unable to find a work by an anthropologist of education who studies the type of cultural work that happens in college seminars. So maybe instead of hoping that an anthropologist of education takes this up, I'll issue a challenge instead to cognitive psychologists: surely you can do better than my social-science history-ish writing in capturing the cultural work that happens inside seminar classes, of finding more specific and narrow stuff rather than the global claims of "critical thinking" might suggest.

Listen to this article Listen to this article
Posted in Higher education at 12:35 AM (Permalink) |

September 21, 2008

Michael Bérubé in Tampa Tuesday!

I'm exhausted and probably should head to bed soon, but Professor of Dangeral Studies Michael Bérubé is the honored guest of the United Faculty of Florida USF Chapter, who is bringing him to the University of South Florida to talk Tuesday, 5:15 pm (CHEM 100), about academic politics. I'm happy. I will also be running completely on adrenaline tomorrow, since my short-term reserves appear to be 100% depleted... but I'm happy.

Listen to this article Listen to this article
Posted in The academic life at 9:22 PM (Permalink) |

September 19, 2008

I want to be able to lead a double life

At the end of the week and looking forward (not!) to a rush-hour drive to a hotel for a weekend meeting, I realize that I really want to have the time to lead a double life, because I have double the obligations of what will fit into 24 hours a day. I feel rather productive with the time I've spent this week, and I don't really want a double life (secrecy's just not my thing, my potential superpower is falling asleep at 10 pm, and I don't look nearly as good in spandex as would be necessary), but I just want the time to do stuff, do more stuff, read all the stuff I haven't yet gotten to, spend time with family and friends, and have a little more time to myself.

Listen to this article Listen to this article
Posted in The academic life at 4:52 PM (Permalink) |

September 16, 2008

Tuesday bits, September 16

Right now, at about 10:30 in the morning, I have a "this too shall pass" attitude towards anything that isn't working right now. Why?

  • Most of class this morning was spent delving into the grandiose education plans of Thomas Jefferson, Benjamin Rush, and Noah Webster. The students read closely, compared their notes to their classmates', compared the views of the three, and left with a puzzle I framed for them. I don't know how hard I worked, but I know they did. Yes, I'm a happy teacher right now.
  • I haven't finished grading their papers from last week. I like to get that done over the weekend, because now I have that hanging over my head before the next batch comes flying into my life on Thursday.
  • Siva Vaidhyanathan has it right in a Chronicle column I want to make mandatory reading for the whole world: Talk of a "digital generation" or people who are "born digital" willfully ignores the vast range of skills, knowledge, and experience of many segments of society. It ignores the needs and perspectives of those young people who are not socially or financially privileged. It presumes a level playing field and equal access to time, knowledge, skills, and technologies. The ethnic, national, gender, and class biases of any sort of generation talk are troubling. And they could not be more obvious than when discussing assumptions about digital media.
  • I now have another historical concept I need translated into statistics. (The last one led me into some exploration of permutation tests.)
  • One doctoral student is now involved in several multi-year, multimillion-dollar grants (one of which could serve as a foundation for a dissertation), and the other is doing great work helping with EPAA. I am SO happy with their work and very lucky to have them as students.
  • My son's cast will probably come off this afternoon.
  • Michael Bérubé is coming to USF's campus next week and giving a public lecture Tuesday (5:15, on the Tampa campus, CHEM 100).
  • My next belt test is a week from Friday.
  • My blood is now flowing with caffeine.

Listen to this article Listen to this article
Posted in The academic life at 10:26 AM (Permalink) |

September 15, 2008

Monday bits, Sept 15 version

Another potpourri entry for Monday morning.

  • Plane flights that land at 10:40 pm lead to grogginess at 5:30 the next morning. Evidence: I think this is a profound observation right now.
  • The count in my EPAA in-box this morning: Spam 42, Real Authors 1. And that's after the spam filter did its work.
  • Some bar/bat mitzvahs have very difficult Torah portions. My nephew's was Ki Tetze (Deuteronomy 21:10-25:19), which begins, translated roughly, "If your army has taken a beautiful woman captive in war, and you want to rape her, you first need to wait a month for her to mourn her dead parents."
  • No meetings today!!!!

Listen to this article Listen to this article
Posted in The academic life at 9:06 AM (Permalink) |

September 12, 2008

Shared responsibilities III: The next ESEA

Over the summer, Charles Barone challenged me to put up or shut up on NCLB/ESEA. I immediately said that was fair; Accountability Frankenstein had a last chapter that was general, not specific to federal law. I'm stuck in an airport lounge waiting for a late flight, so I have an occasion to write this now. Because I'm on battery power, I'm going to focus on the test-based accountability provisions rather than other items such as the high-quality teaching provisions. Let me identify what I find valuable in No Child Left Behind:

  • Disaggregation of data
  • Public reporting
I think most people who don't have their egos invested in NCLB recognize that its Rube Goldberg proficiency definition has no serious intellectual merit and has been a practical nightmare. Yet there is the policy dynamic that observers in the peanut gallery like me can recognize, which is the practice of states in gaming any system, and the way that such gaming undermines the credibility of states with those inside the Beltway. So there's a solid justification in a continued regulatory regime if it is sane and recognizable as such by most parents and teachers (i.e., the connotation of "loving hardass" that I meant in a prior post and that some readers have recognized). I'll have to write another entry on why I think David Figlio is wrong and why teachers are not magisters economici, but incentives just don't appear to be doing that much. An appropriate regulatory regime has to make it easier to be a good educator than a bad educator, make it easier for states to support good instruction than to game the system, and be reasonably flexible when the specific regulatory mechanisms clearly need adjusting.

So where do we go from here? I don't think trying to tinker with the proficiency formula makes sense: none of the alternatives look like they'll be that much more rational. What needs more focus is what happens when the data suggest that things are going wrong in a school or system. On that, I think the research community is clear: no one has a damned clue what to do. There are a few turnaround miracles, but these are outliers, and billions of dollars are now being spent on turnaround intervention with scant research support. To be honest, I don't care what screening mechanism is used as long as (a) the screening mechanism is used in that way and in that way only: to screen for further investigation/intervention; (b) the screening mechanism has a reasonable shot of identifying a set of schools that a state really does have the capacity to help change things -- if 0 schools are identified, that's a problem, but it's also a problem if 75% of schools are identified for a "go shoot the principal today" intervention; (c) we put more effort and money into changing instruction than in weighing or putting lipstick on the pig. Never mind that I'm vegetarian; this is a metaphor, folks.

So, to the mechanisms:

  • A "you pick your own damned tool" approach to assessment: States are required to assess students in at least core academic content areas in a rigorous, research-supported manner and use those assessments as screening mechanisms for intervention in schools or districts. Those assessments must be disaggregated publicly, disaggregation must figure somehow into the screening decisions, and state plans must meet a basic sniff test on results: if fewer than 5-10% of schools are identified as needing further investigation, or more than 50%, there's something obviously wrong with the state plan, and it has to be changed. The feds don't mandate whether proficiency or scale scores are used; as far as the feds are concerned, it's a state decision whether to use growth. But a state plan HAS to disaggregate data, that disaggregation HAS to count, and the results HAVE to meet the basic sniff test.
  • A separate filter on top of the basic one to identify serious inequalities in education. I've suggested using the grand-jury process as a way for even the wealthiest suburban district to be held to account if they're screwing around with racial/ethnic minorities, English language learners, or students with disabilities. I suspect that there are others, but I think a bottom line here is the following: independence of makeup, independent investigatory powers (as far as I'm aware, in all states grand juries have subpoena power), and public reporting.
  • Each state has to have a follow-up process when a school is screened into investigation either by the basic tool noted above or through the separate filter on inequality. That follow-up process must address both curriculum content and instructional techniques and have a statewide technical support process. At the same time, the federal government needs to engage in a large set of research to figure out what works in intervention. We have no clue, dear reader, and most "turnaround consultants" are the educational equivalents of snake-oil peddlers. That shames all of us.
The gist here is that we stop worrying about perfecting testing and statistical mechanisms as long as they are viewed properly as screening devices. Despite the reasoned criticisms of threshold criteria (e.g., proficiency), the problem is not that they exist but that these mostly jerry-built devices are relied upon for the types of judgments that make many of us wince and that the results fail the common-sense sniff test. As long as the federal government tries to legislate a Rube Goldberg mechanism, it will have little legitimacy, and states will continue to be able to wiggle away from responsibilities when they're not doing stupid things to schools. (Yes, both can happen at the same time.) Much wiser is to shift responsibility onto states for making the types of political decisions that this involves, as long as the results look and smell reasonable.

Doing so will also allow the federal government to focus on what it's largely ignored for years: no one knows how to improve all schools in trouble (and here I mean the organizational remedies -- there's plenty of research on good instruction). Instead of pretending that we do and enforcing remedies with little basis in research, maybe we should leave that as an open, practical question and... uh... do some research?

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 11:59 AM (Permalink) |

September 10, 2008

Mostly bright spots

Miscellaneous thoughts after being on campus for about nine hours:

  • It's wonderful having a graduate assistant help with preparing manuscripts. I am now several weeks ahead on getting the next article to publication for Education Policy Analysis Archives, so I can focus the next few Mondays on review-work catchup.
  • I think I'm recovering from the cold that struck a few days ago. I need to stay away for a few more days from a friend/coworker who Does Not Need An Infectious Disease.
  • Obama's education speech in Ohio has irritated Checker Finn, pleased DFERs and AFT President Randi Weingarten, and will probably be the last substantive thing said on education in the campaign.
  • The online class has begun to slip into a regular routine each week. This is good.
  • I tried using Poll Everywhere in my undergrad class yesterday morning. It worked, reasonably, for the few things I wanted it to do. Don't worry, Margaret: I haven't slipped over COMPLETELY to the dark side. Yet.
  • What did faculty joke about on transitions to administration before Star Wars and "the Dark Side"?
  • New Orleans didn't need Ike, true. That doesn't mean Texas needs it any more.
  • My nephew's bar mitzvah is this Saturday. My daughter may try to visit a college while we're there. I'm not sure my poor brain can handle all that activity on a weekend.
  • I think I can boil down the change-ESEA list that's in my head. Watch out, manifestistas! Or is that manifestoistas? Manifistas? Dang, I need to learn Esperanto to come up with jargon that just trips off the tongue like "misunderestimate."
  • I still like Falstaff as a character.

Listen to this article Listen to this article
Posted in The academic life at 4:28 PM (Permalink) |

September 9, 2008

Cold permutations

First, to provide a minor update on this morning's news items:

  • Semi-success on the reserving-time front. I had a lunch meeting and then a 3 pm meeting, and the time in between was too short to do much, so I exchanged one parking sticker for another. Whee. At least my wonderful grad student assisting with the journal did a monster job helping on a long MS, giving my head-cold-affected mind a much easier job going through the next article. I WILL climb on top of this mountain of work. Just not today.
  • It's a semi-full-blown cold now. Proof: I should be asleep, and I'm exhausted, but I can't sleep.


I've been trying to wrap my mind around permutation tests and exchangeability for about a week, and I figure that my typical head-cold mentality may be the best shot I can take at it both in terms of the orthogonal way I think at way-too-late-on-a-head-cold evening and also the fact that once I'm up this late and in this state, no student or MS author wants me to be making decisions right now. (For the record, I'm on antihistamines. I know, I know: Never take Benadryl and grade. No. That's not funny, not even in my state of mind.)

A few weeks ago, I was pondering the NYC achievement gap controversy, a debate over the summer that among other things spawned a Teachers College Record commentary by Jennifer Jennings and me (available just to subscribers for now, but to the world in a few weeks). And while the limits on TCR commentaries and op-eds require a fairly narrow argument, I kept thinking about trends and time series data as I looked at the New York City Department of Education's claims. I kept thinking to myself, There has to be something an historian can contribute to this debate that is specific to the way historians think. I'll probably write something at length when I'm more coherent and have some time, but there was an obvious answer that came to mind: to historians, the order of events matter. An argument about causality depends on contingency which depends on a sequence. (Historians often focus on contingency rather than causality, except when we're playing the counterfactual game. The obvious answer to the question, "What caused Gore's defeat in 2000?" is "everything, or almost everything.") The sequence doesn't prove causality (or contingency), but it's necessary.

That logic is usually not applied in policy. In the case of New York City, as is typical in this type of reform publicity, someone pointed to a time series of data and claim, "Aha! See this trend? Ignore its tentative nature: it's PROOF that we're on the right track." One obvious problem with the NYC data is the reliance on threshold-passing percentages; that's the focus of the TCR commentary. But the NYC Department of Education made claims about the achievement gap more broadly, and the data is a lot messier than the folks in Tweed would state. Below are three permutations of the "z-scores" of achievement gaps (the differences in Black-White means on the 4th-grade state math tests, scaled to the population's standard deviation). One is the real time series that runs between 2002 and 2008. The other two are permutations. Before you look for the data (it's on p. 13 of the PDF file linked above), see if you can tell the differences among them, and which is the observed order:

0.74
0.79
0.73
0.67
0.72
0.67
0.71
0.79
0.67
0.72
0.67
0.71
0.74
0.73
0.79
0.72
0.71
0.74
0.73
0.67
0.67

My professional judgment as an historian is also common sense: if the order of events does not make a discernible difference, even if you ignore measurement error and standard errors, then it's hard to conclude that there's a trend. How to test that is the realm of statistics, and when I explained the issue to my colleagues Jeffrey Kromrey and John Ferron, the answer from them was clear: permutation tests. That's a general family of nonparametric tests of inference that's the formal version of the question I asked: if you jumble up the data in all the possible ways they could be permuted, and if you look at a particular measure of interest (a test statistic), where in the distribution of all permutations does the observed data set fall? In the case of the 4th grade Black-White gap on New York state math tests measured as a z-score, we have 7 points of data, which have 7! = 5040 permutations. If you choose an appropriate test statistic for each permutation and the observed time series is about 125 from either end of the distribution, that excludes the 95% or more permutations in the middle of the distribution.

No, I haven't had the time or inclination to follow up, learn how to calculate one of the possible test statistics and how to get the R statistics program to do a permutation test. There are two problems, as I've learned from my colleagues: choosing the right test statistic is a matter of art as well as science; and there may be a problem with exchangeability. As far as I understand it, exchangeability is a less constricting assumption than the standard "independent, identically-drawn" sample assumption in parametric inferential statistics. From what I understand, the practical definition of exchangeability means roughly that you could theoretically exchange all the data points without screwing up the distribution. Again, if I understand correctly, one situation that violates the assumption of exchangeability is in autocorrelated data—i.e., when one data point influences the next one (or the next few). And if there's anything that's likely to be autocorrelated, it's a time series. That's not a serious problem if you're just looking to see if a trend exists at all; for that, autocorrelation is a form of trend (though an artifactual one). But if you're trying to make causal inferences or anything more complicated when there's autocorrelation (i.e., if achievement data levels or trend slopes are different before and after a policy change), I think you have to throw permutation tests out the window.

And that's such a shame, because the concept is still right when extended beyond the question of a trend: if a policy makes a difference, then it should make a difference on which side of the policy change you're sitting. So if you're a clever person with statistics, please provide some ideas in comments for where to go with this or if, as I suspect, the best we can do with permutation tests is ruling out possible trends/autocorrelation.

Listen to this article Listen to this article
Posted in The academic life at 12:56 AM (Permalink) |

September 8, 2008

Monday bits

I didn't have time this weekend to write a lengthy, thoughtful post, or even a lengthy and thoughtless piece, so you get bits this morning.

  • Reserving Mondays: I've shut off my e-mail for now to get some editing tasks done, and I'll see if I can reserve Mondays for selfish purposes for the entire semester. Wish me luck on this one!
  • Honesty: the Palm Beach Post's editorial board approves a draft change in calculating graduation rates in Florida. Kudos to Florida's commissioner of education, Eric Smith, for pushing this. (Disclosure: I've given a few ideas to the state department of ed on options for how to handle graduation in 5, 6 years, etc.)
  • Sunday morning grading: I got out to a coffeehouse early yesterday to read my first batch of undergraduate papers. Several brought smiles to my face with great writing, provocative ideas, or both. That's a good sign for the semester.
  • Fetishized vs. nonfetishized curricula: I wonder how the history of the Core Knowledge Foundation would have been different if E.D. Hirsch had thought to frame the issue not just as accumulating tiny bits of knowledge (how Herbartian of him!) and instead had framed it as a matter of both a knowledge base in different disciplines and the heuristic frameworks of those disciplines.
  • I know I have at least a below-the-radar version of a head cold because I've had moments of earache in the last day, I had less energy over the weekend than I normally do, and I was sure last night that a mashup of Timothy Burke's guide to historical arguments and Atlas Games's Once Upon a Time would make a great introduction to historiography.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 9:04 AM (Permalink) |

September 1, 2008

Shared responsibilities for children II: The loving hardass manifesto

Back in June, I briefly noted the potential political dynamics of the dueling manifestoes associated with the Broader, Bolder Approach to Education and the Education Equality Project, apologized for overplaying that analysis, and wrote an entry to talk broadly about shared responsibilities and education as part of the state. I've promised but have not followed through on my own manifesto, and it's now long past time for that. So, without further ado...


The Loving Hardass Manifesto*

I'm going to cut the shared-responsibility issue in a way that doesn't avoid the hard problems. Essentially, wherever your work touches children's lives, you're responsible for busting your butt without ruining your health or life. Unlike the Education Equality Project manifesto, I do not think that teachers are all-powerful or all-responsible. They're very important and responsible, but not for everything. Unlike the Broader, Bolder Approach, I do not think we can avoid central questions about accountability within school by reference to the other legitimate needs of children outside of schools. Yes, children have lives outside school, but it's acceptable to focus on what happens inside schools for things schoools are responsible for. And unlike Barack Obama, I am not going to say that both statements are right. Both statements are partially right. And while I know and admire several people who have signed one or the other statement, I will not sign either one, because both are flawed.

Let me start with the Project crowd. If you're a politician or administrator and believe that everything you've done is perfect, with no regrets, and all the evidence points in your favor, I hope you brought enough to share, because whatever you're smoking, I want to try it. Using only the high-quality evidence that is in your favor (and here I mean David Figlio-quality evidence), you can make a claim that high-stakes accountability leads to modest improvement in outcomes. But that's about it.

If you're a civil-rights activist and think that the best way to improve schools is to lambaste teachers and their representatives, I have a year for you: 1968. And a book: Tyack and Cuban's Tinkering toward Utopia. I have plenty more to suggest, but I figure that's enough.

But I'm also disappointed by the Broader, Bolder Approach. Everything it says about putting education in the context of broader government programs for children is correct. And yet, if its purpose is to get us to think in a different way about accountability and NCLB, it underwhelms. There's something odd about a statement on school accountability that has precisely one paragraph suggesting vague ways to change how accountability should work within schools.

Let's think about some basic facts: most kids come to school with families they go home to at night. If the children and their teachers are lucky, their families will only have the ordinary neuroses that God or Woody Allen placed there. If the children are unlucky, they'll also deal with poverty, disability, abuse, negligence, or having Paris Hilton as a distant relative. If you're a teacher, you can gripe about the families, but it's probably best not to, for a few reasons:

Your complaining to peers will not improve the parenting of anyone.

We've heard it before, and it wasn't convincing the last time, either.

If you complain about the parents, you will be depriving your students of their internationally-recognized right to be the first to complain to a therapist about how they were brought up. Really: it's in the UN Charter, under "Psychotherapy as an Adolescent," right above the bit about iPods and PlayStations. Go look it up if you doubt me.

I just lied. You may not have caught this, but the 1959 Declaration of the Rights of the Child does not mention the right to criticize parents in therapy or the right to consumer electronics. There isn't a single mention of either Apple or Microsoft, a shameful omission which Bill Gates is working hard to remedy. But until then, children only have the recognized right to things such as health care, food, shelter, the care of parents or other responsible adults, freedom from discrimination, and education.

I don't know if you've noticed this, but as a society we're not doing so well on fulfilling these rights. 600 million Chinese citizens use cell phones, and in a country that is far wealthier, we've still got millions of children without health care. It used to be that American parents would shame their kids into eating everything at dinner by pointing out that children around the world were starving. That makes you wonder what Chinese parents tell their children to shame them. Maybe they say, "Take your vaccination and stop crying: Kids are getting sick in America!"

Since the dueling manifestoes appeared in June, I've been scratching my head. The broader, bolder approach is fine as a statement of broad social policy but it doesn't work in terms of day-to-day accountability. You are responsible for the people who are in your life. When my children have been sick, and I've taken them to their doctors, I've never once been asked, "How are they doing in math?" and then had a doctor refuse to treat my child because they're not yet evaluating double integrals. They treat the kid in front of them the best they can. My father was a pediatrician and allergist who treated both wealthy families from one side of town and working-class families from another part of town. He never complained about the families from one side or the other. He just treated them.

But that doesn't mean my father had absolute responsibility, either. He was expected to be a professional, to keep up with the literature, and to follow standards of medical practice. But there has never been a "Health Care Equality Project" whose primary activities were to take pot-shots at doctors, call them "interests who seek to preserve a failed system," and want to pay doctors by a handful of measures of the health of their patients. My father was never paid by how much his patients weighed that year, or by how many tissues they used because of colds. We already have accounting-driven health care, and I don't know of any doctors or patients who think it's a good idea.

We also don't have ridiculous fads in medicine. Well, we do, but it's generally called the X diet (for various string values of X), or "alternative medicine," for those who think that if you dilute some processed duck liver by 30 or 40 orders of magnitude, your body will react in any way other than, "I'm sorry if you paid for that sugar pill instead of your mortgage, but the best I can do right now is a placebo effect. I hope you like it." In education, we have far more fads. If we had as many fads in medicine as we do in education, people would think that wearing uniforms made you thinner.

So there is something about the dueling manifestoes that just does not seem real to me. It's not that I am immune to their appeal. I want there to be equal education. And I've already written in many places that schooling needs to be thought of in the context of all the state structures that touch kids' lives. But it's still not resonating with me. My generation of the family takes care of these issues collaboratively. My oldest brother has been a lawyer, lobbyist, and think-tank staff member on health-care policy, which takes care of one right. I teach and write about education. The rest of the immediate family's a bunch of layabouts who do nothing other than have jobs and take care of their families, but Stan and I, we're holding our own on this caring-for-children thing, and if your family isn't, don't blame us. We are the Broader, Bolder Approach. But we're both going on diets soon, so that will change.

Back to the central point about responsibility. The hard task that both manifestoes avoid is defining what we really should expect from schools. I don't know: maybe "bust your butts" isn't something people say in polite company. And it's even harder to define in practice. But since the people who signed the Education Equality Project say they're in favor of holding people accountable, here's my charge: go define what "bust your butts" means in ways that are realistic, or fold your tent. I suggest you start by talking with teachers and parents, not among yourselves. This is just one (loving hardass) reader's response, but I know you can do it, or I wouldn't insist on it.

And for the Broader, Bolder crowd, you know you can do better. As a group, you include a bunch of incredibly well-read, smart researchers. And you're right on putting schooling in a broader context. But you just fell down on the accountability part. That one short paragraph on accountability? Please reread it. Really. You think that was the best you could do? You KNOW what you'd say to a grad student who had that fluff in a dissertation. Revise and resubmit, because I know you can get this up to your usual standards.

And the rest of you in the peanut gallery? Don't think that we can rest on our laurels, either. The folks I'm criticizing at least had the energy and guts to put pen to paper. What have you done to define "bust your butts"?

And, yes, this means that I need to look back at the last chapter of Accountability Frankenstein and see if it needs to be sharper. A commenter some months ago said it was not specific to NCLB, and that's a fair enough point. I wanted the book to be about accountability in general, but if I really know my stuff, I should be able to apply it in specific situations. Want a specific list of changes that should happen with the next reauthorization of ESEA? Coming up this fall...

* While I was drafting this in bits and pieces, I pondered whether to use the term hardass, but since Bob Sutton has written the book The No Asshole Rule and Harry Frankfurt's On Bullshit won a book award, I don't think I'm going that far out on a limb. A loving hardass knows that holding people to standards can be in their best interest. So for everyone who signed one of the manifestoes and think I'm nuts here, you're wrong. And in two years, you'll thank me for this.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 9:06 PM (Permalink) |