August 5, 2010

"Overcaffeinated value-added enthusiasts" and the public

I've developed a fondness for Rick Hess's phrasing. Whether or not I agree with him, I have to smile when he describes advocates of value-added or bust as "overcaffeinated value-added enthusiasts." This is in the context of the back-and-forth over value-added measures in the District of Columbia teacher filings (see blog entries by Aaron Pallas, Hess, Pallas, and Hess, from which I drew the overcaffeinated term). What we're seeing here is the beginnings of a public dialog over the technical details of value-added measures, whether in DC or here in Florida (see today's St. Pete Times story on two audit reports over Florida measures, plus articles from Jacksonville and Miami over continuing questions).


Or, rather, we're not seeing much of a dialog, more of a he said-she said dynamic. Pallas wrote from what was publicly available (which was as simplistic as what I read about Bill Sanders' techniques in newspapers in 1990s Tennessee), Hess criticized him for insufficient due diligence for an academic blogger, and we're now into round two on who owes whom what on transparency. Florida is slightly different in the actors: most of the critics this summer have been superintendents, worried about whether problems with the underlying test scores or value-added measures will end up shaming their elementary schools (and them) with lower ratings on the state's accountability system. But it's still he said-she said with the auditors saying they found no problems and the superintendents still having reservations.

What we're missing is clear reporting on the technical issues, but don't blame the reporters. In some cases, there is poor planning by state departments of education (or the DC schools, in this summer's news), so there's nothing clear and accurate and easy to communicate. In other cases, as in those jurisdictions using Bill Sanders' techniques, you've got a proprietary model that the public isn't allowed to inspect. And then there's the simple fact that there is no single holy grail of value-added measures and inherent error issues that tend to be underplayed because standard errors and measurement error are eyeglaze-inducing even if they're important. So the reporting is a far cry from the sensawunda reporting on scientists who uncovered BL's lowball estimates of the gusher's output: oh, wow, there are pools of oil under the surface? oh, wow, you can estimate flows from the speed of particles in a fluid? Nothing like that exists on value-added or growth measures.

Some part of the situation is inevitable when a technical apparatus becomes a tool of political discussion. I don't mean the partisan politicization of statistics (though that happens) but the fact that even mildly controversial bills do not pass in many legislative bodies unless there is a certain amount of pathos in the debate, and the exaggeration of debate tends to drown out the caveats for anything. There are plenty of very careful statisticians out there who can tell you the issues with value-added or growth measures. They're not quoted in news stories, because no editor is going to let "mixed-model BLUE algorithms tend to swallow dependent-variable variance before you get to the effect measures" appear in a newspaper. So there's a mismatch between the technical issues and the level of discussion. You shouldn't need someone with the skills of Robert Krulwich to report on technical measures affecting public policy, but that's where we are.

That feeds into the dichotomous debate that is dominated by the "let the measures work" and the "it's imperfect, so toss it out" arguments. As I wrote a year ago,

The difficulty in looking coldly at messy and mediocre data generally revolves around the human tendency to prefer impressions of confidence and certainty over uncertainty, even when a rational examination and background knowledge should lead one to recognize the problems in trusting a set of data. One side of that coin is an emphasis on point estimates and firmly-drawn classification lines. The other side is to decide that one should entirely ignore messy and mediocre data because of the flaws. Neither is an appropriate response to the problem.

When the rubber meets the road, you're sometimes going to get the firmly-drawn classification lines in Florida that lead people to nitpick technical details (I wonder how many of the superintendents griping this summer have bonuses tied to school grades), and you're going to get nebulous debates when systems such as IMPACT are not accompanied by technical transparency. This just doesn't work for me, and it shouldn't for you, either.

July 30, 2010

"Pushback" week

It's almost as if Nick Anderson and Ruth Marcus worked at the same paper, because "pushback" appears to be the talking point of the week on education policy. Yesterday, Anderson reports, President Obama "pushed back" against some civil-rights groups' criticism of Race to the Top, and Marcus applauded him when the president "took the opportunity to push back." Oh, wait: they do work for the same paper. Well, at least we know that at the Post, some colleagues talk with each other, unlike the one who fired Dave Weigel last month and the other who hired him this month. Then again, the fools at the Post, Inc., appear to be management and bull-male columnists, not rank-and-file reporters.

There are four major stories that dominated national education news in the past week, at least as far as I was paying attention:

  • The drama surrounding the civil-rights group report and non-presser and the two major education speeches this week by Duncan and Obama.
  • Continuing problems in trying to attach state aid to federal bills (after the emergency war appropriations, there's the inability to break the small business aid bill, which had jobs money attached).
  • Michelle Rhee's plans to fire several hundred teachers based on the IMPACT evaluation system.
  • The New York state testing cut-score embarrassment.

Pushback was used in the Post's coverage of the first story, but I think you can say it's a theme for the week. House and Senate members are now in almost open warfare over education jobs riders to bills (possibly extending to the FMAP aid to states on Medicaid, stuck in Congress since early this year). There is debate over how many teachers Rhee is firing and how bad a system IMPACT is. And Joel Klein is twisting himself in knots trying to explain how the mistakes in proficiency rates that he used to puff up his record really isn't a problem and, uh, Lady Gaga shows how good the New York City schools are. I'm half-expecting him to talk about New York's smog swampy beauty, the East River though, doesn't it split the Park Slope from the Palisades? Someone get Bill Shatner to read Joel Klein's ratiocinations!

Some things behind the headlines that seem obvious to this historian:

  • Part of the loose (and fragile) coalition criticizing the Obama administration's turnaround policy stems from unions concerned about due process for employers and community-based organizations worried about the closure of public facilities in poor neighborhoods and the role of public employment in providing a leg up to the middle class. That's not new, and it's complicated. The civil-rights group interest in public employees can be salutary (my understanding is that Black teachers were a solid core of local NAACP chapters in the mid-20th century) but sometimes at cross-purposes with other interests: I heard informally from some observers that part of the pushback against the decentralization of Chicago schools in the late 1980s was the role of the central school bureaucracy in providing a leg up into the middle class, and the reduction of the central bureaucracy threatened those positions. Today, the invisible risk is the position of minority teachers' aides and other non-certified employees. My guess is that they've been disproportionately affected by school-system layoffs that try to hold onto classroom teachers.
  • I still don't have a clue how much test scores played a role in the firing of DC teachers, and my guess is that you don't, either. IMPACT included test scores, but you'd have to look at the details of individual employees to know whether an individual firing is a case where all the indicators (including the required five observations) pointed in the direction of an incompetent teacher or whether test scores trumped supervisory judgment for any. Normally employers have broad discretion in evaluation systems, but the failure to bargain IMPACT may put the DCPS in some jeopardy of an unfair labor practice finding. (That depends on both the structure of DC collective-bargaining law and the details of what happened with IMPACT and WTU's requests for bargaining.) Double jeopardy for Michelle Rhee: the inclusion of the pseudoscientific "learning styles" in the IMPACT observation system. My guess is that the AFT (the national affiliate for the Washington Teachers Union) can quickly get their hands on well-known psychologists to rip that to shreds for any teachers where the tipping factor was a supervisor's judgment that they didn't cater to student "learning styles."
  • Joel Klein's dancing around the cut-score fiasco in New York illustrates once again that the performative setting of cut scores is often a result of the tension between bravado and "reform testosterone," on the one hand, and politically acceptable failure and the political need to game the system, on the other. We'd like to think that cut-score setting is arbitrary in the sense of arbitration, but it's too often arbitrary in the sense of caprice and politics. Two years ago, Jennifer Jennings and I wrote a commentary for Teachers College Record ($$ required) about the dangers of trusting threshold-based proficiency percentages as opposed to central tendencies such as means and medians, with New York City as the object lesson. She's too mature for this, but I have no such reticence with the last week's revelations: nyah nyah nyah, we told you so. And from those of us who warned years ago about the fragility of growth/value-added statistics? same message.

Bottom line here for administrators: test-based measures should only be used as a case to fire teachers or administrators where they strongly point in the same direction as observation-based evaluation instruments that are developed with some common sense, with unions and excising crap such as learning styles.

July 26, 2010

"Opportunity to learn" revived?

As Ed Week's Michele McNeil is reporting, a coalition of civil rights groups has issued a white paper today through a (new?) organization, the National Opportunity to Learn Campaign. Last night, Diane Ravitch was tweeting her reading of the paper as a gentle but firm rebuke of the Obama administration's approach to accountability. To some extent, I think she's right: the 17-page report briefly referred to the inappropriateness of judging schools and teachers primarily by test scores, but that was a brief reference.

For the longer and more committed passage criticizing policy prejudices towards school closures, I read the argument differently, because of the other arguments in the paper in favor of more money for early childhood education, wraparound care programs, and NCLB's public-school choice provisions and against budget cuts. And then there's the name that's a throwback to early-90s arguments in favor of opportunity to learn standards. To me, that all looks like a straightforward community-civil-rights approach more than an argument against high-stakes testing. In that context, the argument against school closure is an argument against withdrawing resources from a community institution that may be one of the few public facilities in a poor neighborhood.

That also fits with how the coalition's paper addresses Race to the Top: don't withhold resources or programs from poor children. Instead, combine formula grants with conditions. Notably, the paper states that a limited competition is acceptable, suggesting that the constituent organizations would not directly oppose Race to the Top as long as its structure does not permanently replace formula grants in ESEA. I know what others are going to say in response: we have plenty of conditions on federal funding, but the federal government almost never penalizes states for falling down on the job.

To a great extent, the politics of and posturing around education reform are all depressing to me: education reform policies are dwarfed by the state of the country's economy right now. In fact, that's a crucial part of the argument of the Broader, Bolder Approach. So you should maybe focus your efforts on the national economy right now? Or if not the national economy, maybe focusing on states, where the real action is going to happen over the next few years?

I think the coalition is moving about 15 months too late, if the key movers intended to shape federal policy. It's very likely that there won't be more RTTT, there won't be ESEA reauthorization, and there won't be a heck of a lot of things that should be happening from the perspectives of a variety of people on different sides of this debate. I wish I had been been wrong a month ago, but it looks more and more that I was right in predicting that David Obey's gambit last month was a stupid gamble instead. I was wrong in guessing that Obey would be frustrating George Miller, but I think I'm right on the general picture. To be clear, it's far from the biggest SNAFU of the Congressional session: that's the too-small size of the stimulus in early 2009 and the failure of the White House to nominate (or recess-appoint) enough Fed governors. But I'm still depressed, and puzzled by the strategic choices.

(One final puzzle is the group's website. The contact information is for the Schott Foundation in Massachusetts, which is consistent with the few blog entries (written by Michael Holzman) and the press-kit stuff. But there are no staff members or individuals listed on the website, just organizations. The whois entry for otlcampaign.org shows that the domain name has existed since sometime in 2009, but it's registered through a proxy, and the Internet Archive has no history of the website (blocked at the site). This is all perfectly legal, but it's odd.)  

July 24, 2010

Firings in DC

Andy Rotherham is correct that the termination notices in the DC public schools this week included about a third of the total who had not met licensure standards, and a greater number were rated in the highest classification in the annual evaluations. Nonetheless, what is newsworthy about the terminations is the public nature of outright firing of a chunk of teachers for nonperformance. It wasn't the firing of a third of the district teachers, but significantly less than 10%. Let's assume a similar number of those given notice of "underperformance" this year either quit or are fired next year. That would be the firing of around 13-16% of the teachers for nonperformance in two years. It's noticeable.

By itself, the number is neither good nor bad, though many will argue the point either way without additional information. I say we wait. First, we wait for the Washington Teachers Union to sort through the information to see if any teachers were fired without the five classroom observations required for the evaluations. The grievance mechanism that exists in the union contract is on procedural grounds, and here we'll see how careful Rhee's bureaucrats have been. Then, we wait to see if there are any examples of firings that don't meet a basic smell test--anyone who had won teaching awards and plaudits but were given low ratings for reasons of favoritism or obviously inappropriate application of student test scores. Either procedural errors or plausible miscarriages of justice are reasonable grounds on which the union will fight for members and has an ethical obligation.

Nor is that willingness to fight for individual members inconsistent with a union's willingness to try different methods of evaluation. My chapter can and does file grievances when we think an individual's procedural rights were violated in the tenure review process. That says nothing about the standards of review. It says that we'll fight for the integrity of the review process.

July 23, 2010

A more realistic view of standards

This week I've been spending most of each day in a workshop on the Spanish Civil War for area history teachers. In it, teachers learn about the war in general and also the involvement of American volunteers for both medical services and fighting on the Republican side (what's now known as the Abraham Lincoln Brigades). We've given them a number of books and other resources. They've had a chance to hear from and ask questions of an author of several books on the war and the aftermath (Peter Carroll), read both books and a wide sampling of primary sources, and yesterday they visited Ybor City's Centro Asturiano and listened to some older Tampa residents who had both direct and vicarious experiences of the war (such as that of Aida Gonzalez). Today they worked on developing specific lessons or assignments based on what they've been learning, such as DBQ exercises for Advanced Placement classes.

For those who have run or participated in such summer workshops, this is probably familiar (with the exception of hearing from eyewitnesses or participants, in the case of workshops on the Civil War, ancient civilizations, and the like). We've had some wonderful classroom teachers as participants in the two years I've been involved, and they tell us they appreciate both the chance to learn about a subject in depth and our treatment of them as adults. I just get to tag along, except for the bit about standards. And since there's an ongoing discussion of whether the common core standards in math and reading adopted by a majority of states mean much, maybe a practical discussion might help.


I'm not a "social studies methods" specialist, but when we were planning last summer's workshop, I knew what was missing: a connection for teachers between what they were learning in the week and the new state social studies standards in high school. I think this is all that justified my presence in the workshop because when I write, "we were planning," I am using "we" in the social convention form, not in the "I earned a significant chunk of the plaudits" form. Most of the credit for this goes to Peter Carroll, the USF history chair Fraser Ottanelli, and a former area teacher who is an adjunct at USF, Robert Alicea. I see the beautiful plans (okay, they were somewhat fuzzy until things fell into place in a practical schedule) and think, "Ah! They're missing the help-the-teachers-with-bureaucracy part. I can do that." And occasionally chime in to expand discussion.

Keep in mind that the participants in the workshop were already high school history teachers, the vast majority with experience in AP classes or in an International Baccalaureate program. We don't need to tell them how to plan a year, and we'd have been a fifth wheel had we done so. Especially in AP courses (and most especially for the drink-from-the-firehose AP world history course), teachers have to manage the coverage issue very carefully, and in many cases teachers explicitly used the materials for 1-3 days last year. (One teacher regularly has after-school enrichment opportunities, where he walked students through the James Lardner papers as an extended exercise in primary sources that tell a story.) So why hand out standards lists?

Last year, there were two reasons for me to sort through the new standards, identify which ones were related to the Spanish Civil War, and then sort those by some obvious themes (the narrative within Spanish history, world context, American involvement, art and popular media use, and historical skills). First, the state had approved the standards in 2008, but there had been almost no professional development, and this was an opportunity to show teachers what they were written like in a context when it has some use and it's not just a verbiage dump on teachers. (Teachers will know what I mean by that.) Second and most immediately, I reorganized the benchmarks so that they would help teachers generate ideas for lessons, assignments, or other ways to use the materials. In reality, I suspect I didn't need to do that much, since the primary sources and talking with eyewitnesses to history are far better inspiration than standards. Third, showing teachers how to tie a specific lesson to official state standards lets them justify doing what they think is professionally appropriate. In a large high school, an assistant principal for curriculum isn't going to push anything like a pacing calendar on teachers in most subjects, but some of them will ask what standards are met by a lesson, assignment, unit plan, etc. Giving teachers standards gives them something to put at the top of their plans as an official stamp of approval on lessons. (Well, it does if the standards make sense: the benchmarks mentioning Franco, the lead-up to World War 2, the Spanish-American War,* or the social movements coming out of the Great Depression are going to make more sense here than a benchmark on early federal history.)

This year we have some middle-school teachers, something I didn't know until Monday. So I felt horribly guilty when I realized my organized handout for high-school teachers was useless for them, except as an illustration of what high school teachers would expect from students. On top of that, the middle-school curriculum is up in the air with a legislative mandate to teach civics in seventh grade. That doesn't change anything about the middle-school standards, because civics is always going to be somewhere in social-studies standards (and is prominent in the middle-school benchmarks). But it does mean that many districts don't yet know how they're going to organize the middle-school curriculum into specific courses, though the standards provide some clear direction and emphasis on history and civics (ancient civilizations in sixth grade, U.S. history through 1877 in eighth grade, and now obviously civics in seventh grade). So teachers who had been focusing quite a bit on geography? They'll have to retool, and for now a great deal of their own initiative may seem like a waste if they'll be moving in different directions in a year or two. Yes, I've gone through the middle-school standards this week and identified a few dozen with clear connections to the Spanish Civil War, ironically more in social sciences than in history because of the topics selection for middle-grades standards. (Example: map use. Military maps in the war, historical maps as secondary sources, socially generated maps such as the map of mass grave sites and other war-related sites in Spain.) But the dynamics of "the curriculum is up in the air" are still prominent.

This is a commonplace about life on the ground with curriculum. The abstract talk about standards and alignment ignores the multiple layers that shape the taught curriculum, from idiosyncratic course expectations (e.g., the more deterministic nature of AP classes) to legislative mandates, textbook choices, the item specifications on state assessments, and the program du jour of the district that gobbles up curriculum either directly (Hillsborough county bought into the Springboard program several years ago, a decision that diverts a day or three each quarter for its mandates, if within the curriculum) or by absorbing time (by adding a tangential curricular module such as anti-drug education and forcing administrators to stuff it in some class). Curriculum mandates and pressures metastasize.

As a result of these multiple mandates and pressures, I am less persuaded than others either by the argument in favor of a common core curriculum or the philosophical or political arguments against a common core curriculum. First, the idea that something is truly a "core" that will only take up a small part of the year is pure bunkum; given the other structures of school, anything called a "core" will inevitably become "pretty close to all." And even then, there will be much slop between the formal expectation and what happens in a classroom and also what's assessed. Yet I am also unpersuaded by the argument that teachers should not have a structured curriculum, or that somehow a set of curriculum standards is evil. As I've written before, the first round of state curriculum standards was generally awful, but I don't think you could have expected them to be good, so that doesn't tell you what standards might look like, and there are now some reasonable examples of the right balance between generality and specificity. (My historical cynicism is out in force this morning.) Yes, standards advocates make a weak argument with the international comparison rationale (the claim that our chief international competitors have national standards, so we must, too), but that's not the central argument for curriculum structure. The most important arguments for some curriculum structure are (a) requiring teachers to design curriculum from scratch is cruel and unusual punishment; and (b) there are some overlapping content areas that most students would find fairly practical to get under their belt.

Why do I believe that requiring teachers to design curriculum from scratch is abusive? I'm a Ph.D., with more specialized expertise than the bulk of the American population, and I would find it extraordinarily challenging to design all of my classes from scratch every semester. I don't; and I would view it as an exploitation of junior faculty to ask a new assistant professor at a research university to prepare an entire curriculum from scratch at the same time she or he has to gin up a research program. There's one college I know that talks about creating new courses on a regular basis, Evergreen State College, but even there the courses (or "programs," as they're called at Evergreen) regularly reappear so a faculty member isn't completely designing things from scratch. And most of the faculty there are veteran teachers, and the programs are commonly cotaught by at least two faculty. For K-12? Let's just say we're putting in a whole week so teachers can design a single lesson or assignment each (and then share the fruits of the work). It is one thing to point out that many veteran teachers can design a class; it is another thing entirely to suggest that all teachers have to.

In addition, there is a legitimate argument that some overlapping content is important for students. It is an easier argument to suggest common material for a field such as math or U.S. history than in areas such as world history or English literature, which is why I'm using the term overlapping. But the point still exists that high school students are meeting some common expectations when they can correctly manipulate an algebraic expression, explain how evolution complicates medical treatment, and talk intelligently about the historical struggles of Americans to get the country to fulfill its ideals. (And for those who are wondering why I think algebra is a sensible expectation, it's less important for someone to be able to solve word problems about westbound trains from Chicago than understanding what Paul Krugman means by lower bounds for effective interest-rate policy.)

As I stated above, I'm a bit cynical about structural school reform, and I do not believe there is One True Way of constructing standards. To take U.S. history as an example, generally most state standards (and the effort by Crabtree and Nash) use a fairly common approach to periodization and important questions, and one that starts from the centrality of the nation-state. One could imagine equally legitimate approaches that focus on international context, and you can find such syllabi for college classes. But you have to construct a course around something, preferably something coherent, and the most common approach is not evil just by its being common. The practical question is how much of an overlap we truly need, understanding that every time we say X needs to be a common part of the curriculum, we're squeezing out something else.

* The U.S. victory in the Spanish-American War eliminated the bulk of the remaining Spanish empire, leaving a social and structural problem for the Spanish army at the beginning of the 20th century, stuffed as it was with a high proportion of officers and a much shrunken set of territories to control.

July 16, 2010

Gates in Tampa ... no, my daughter's school!

Two chances in one week to provide personal perspective on Gates' philanthropy. Along with a few thousand other AFT delegates, I saw Gates's speech last Saturday. Today's comment comes via the Business Week article on the Gates Foundation's education program. The article is one of the better journalistic portraits of the foundation, including historical perspective by Maris Vinovskis and some technical perspectives from Howard Wainer and Daniel Koretz. And then in the second half, the article quotes some teachers such as JoAnn Parrino and Kathy Jones. I expected the article to quote either Hillsborough superintendent MaryEllen Elia or Hillsborough Classroom Teachers Association president Jean Clements, and then suddenly the focus was on some teachers at Chamberlain High School, where my daughter graduated in the spring. Yes, she had both Parrino and Jones, as well as a few others mentioned indirectly in the article as Daniel Golden followed Hillsborough's Gates project staff into a teacher meeting at the high school.


Both teach AP social studies courses, Parrino with human geography (taken by ninth graders in Chamberlain) and economics (I forget whether it's micro or macro). Jones teachers the world and European history classes. Both have their student admirers within the school. In the article, Parrino is quoted in favor of random classroom visits, and Jones on a different topic, whether there is such a thing as a year-over-year growth measure when the class is a one-year class such as a topical social studies class. And the music teachers apparently scoffed at the notion that their competence can be measured by student performance on an end-of-semester music theory class. Most of the teachers I've met at the school are reasonably thoughtful at the least, and the article begins to touch on their perspectives and skepticism.

What is notable is that none of the discussion Golden reports is the type of "we can't be expected to do great things with poor kids" excuse that's the common straw-man argument by advocates of high stakes testing. Jones is right to be skeptical that there is any competent value-added measure for history, and the band and chorus teachers are absolutely correct that a music-theory class is an awful measure of their competence. Want to know what a Florida band or orchestra or chorus director pushes their students to perform in? Music Performance Assessments, or MPAs. These are juried festivals of school groups, and teachers in Hillsborough take them very seriously. To use music-theory paper exams instead of MPAs is a pedagogical crime. Do you think the Hillsborough High School band director should be judged by how well my son and his fellow sax players know a Napoleonic 6th, or how well they can blend in a performance of "Take the A Train"?

At some point, advocates of using student outcomes as part of teacher evaluation need to get some sense about implementation. Hillsborough is clunking along right now, and it'll need to adjust things on that part of the evaluation system. The rigid "everyone has to be evaluated in the same way even if it makes no sense" system is not viable in the long term. But it's what the mantra of "50% must be on student outcomes" will lead to unless Charlie Barone and others come out in favor of common sense in the use of student outcomes, and that includes telling their friends when they're wrong in a formulaic approach.

July 14, 2010

Fat tails and audit trails in Florida test scores

I'm starting the day behind on a bunch of things, thanks to a week at the AFT convention in Seattle and the beauteous handling of bad weather by Delta. I arrived in Tampa about 23 hours after leaving Seattle, and let's leave it at that.

So I'm a bit behind on the background behind the evolving controversy over test scores in Florida. NCS Pearson was way, way late on releasing scores, and part of the reason was what Florida DOE officials called glitches in the demographic files Pearson had on students, or how test scores are tied to students and then teachers.

I have a sneaking suspicion that's also behind the controversy that's developing, as first the urban and then a bunch of other system superintendents complained that the proportion of elementary students not making adequate progress year-to-year just didn't fit with any sense of reality (on the low side). Head to the St Pete Times for the published stories and blog entries, including new complaints that the organization auditing Pearson's work is a subcontractor of Pearson, but here's the reason why I suspect the demographic files are a good starting point: Florida's "growth" measure is not the mean or median growth year-over-year on some vertical scale, nor is it a regression-based measure of deviation from some version of expected growth. Instead, it is a jerry-built dichotomous variable of whether an individual student made a particular growth benchmark in a year: yes/no.

It's been a few years since I looked at the details of this "growth" definition, but there's some inherent sensitivity in any measure based on thresholds to variability around the relevant threshold. In the case of Florida's growth measure, the vulnerability is going to be less around the construction of a particular scale at a point in an individual test because the measure depends on a student's prior-year score. So a psychometric vulnerability is going to be two sources: the general characteristics of tests in two years, and the added variability that you get from comparing scores in two years (there's measurement error in both scores, and the measurement error when you compare the scores is going to be greater than the measurement error in either base year or following year).

Since the two-year-variability issue has been a fact of life for this measure for a number of years, I would be surprised if that were the issue. So then the question is whether this year's fourth- or fifth-grade reading test scores have unusual distributions that would cause interesting problems at the thresholds for "making gains" for students who were low-performing in the prior year. A particularly fat tail at the low end might cause that, but that's speculation, and I suspect an obviously fat-tailed distribution would have been picked up by the main auditor, Buros.

But you can have a non-psychometric wrench in the works, because Florida's dichotomous variable is highly sensitive to one other matter: the correct matching of student test scores from year to year. If the student data files were messed up, and student scores from 2009 were matched to the incorrect student scores from 2010, you'd have all sorts of problems with growth. I strongly suspect that's what tipped off problems with the data files earlier in the spring. If the failures were general, you'd have a skewed distribution of the dichotomous growth variable as the lowest-performing students from 2009 would be the most likely to be matched (incorrectly) to higher scores in 2010 and vice versa, so the first clue would be markedly high growth indicators for 2009's low-performing students and markedly low growth indicators for 2009's high-performing students.

But that's not what school districts are reporting: they're reporting unusually low growth proportions for low-performing students from 2009. I can think of a few different ways you'd have that after Pearson tried to correct any obvious problems it saw earlier, but that's speculation. What needs to happen is an examination of the physical artifacts from this year for a sample of schools: the booklets, the student demographic sheets, and the score sheets. We're talking about more than a million students tested, but we can start with a sample of schools that the urban-system superintendents are worried about and track the data from beginning to end with a small enough set to see exactly what happened to the satisfaction of local school officials, policymakers, and the general public.

And if Pearson destroyed all physical artifacts so you can't trace the path of data? Cue "expensive lawyer" music...

July 12, 2010

Gates speech at AFT

Originally written Saturday, July 10: I've figured out how to hang this electronic device onto the back of the chair in front of me while my old PDA foldable keyboard is synced and sitting on my lap, so I can write this blog entry in the middle of the AFT session. AFL-CIO President Richard Trumka gave a spirited speech before lunch, and then the floor approved a resolution on teacher evaluation without amendment.

This afternoon, we started with resolutions on community support and career/technical education (CTE) programs. For the most part, the resolutions this afternoon were neither going to be the controversial resolutions nor the controversial part of the afternoon session, which was Bill Gates' appearance at the convention. Very popular was a resolution urging public meetings for the national commission on fiscal responsibility and reform and giving AFT an official position in favor of progressive effective tax policy instead of Social Security benefits cuts that are regressive. As I've written before, a number of people simultaneously want policies that would end in significant layoffs of teachers over 50 and also significantly reduce pension benefits and contributions to public-employee pensions. Evidently, there is some group of self-defined reformers who are in fear that somewhere, someone is enjoying a retirement free from fear of destitution.

The Gates appearance started at 4:15. From what a colleague told me later, he helicoptered over from his island estate. Randi Weingarten at first started speaking from the sheet announcing Innovation Fund awardees and then turned to introducing Gates. She took care to quote from Gates's annual letter at points where he specified opposition to solitary use of test scores to evaluate teachers and supported evaluation as a tool to help most teachers. With a smattering of boos, Weingarten smiled and said, "I thought you guys were leaving," referring to the threats of a boycott by the small dissenting caucus By Any Means Necessary (BAMN). The majority of delegates roared. Later, there were about 25 delegates out of several thousand present who walked out as Gates stood at the podium. So much for the huge boycott of Gates's speech...

Gates started by publicly congratulating AFT for the approval of the resolution on teacher evaluation/development and on steps taken thus far, including the AFT locals who are working with the Gates Foundation on specific programs. He mixed in some misleading statements about "declining" graduation rates (as opposed to stagnation) with some fair statements and a clear statement that teachers must be included in reform. He spent a few moments discussing the failed small-schools initiative. The greatest applause lines came when Gates criticized the existing record of poor administrators' evaluations and when he acknowledged that people who have never taught in a classroom do not understand how difficult teaching can be.

The BAMN protesters then had pretty awful timing, coming back towards the hall shouting protests ... just as Gates said some teachers have challenges with students who are bored or engage in disruptive behavior. The hall erupted in laughter at the irony.

Gates's weakest argument was the individual teacher equivalent of effective-schools rhetoric: see what teachers do when students demonstrate great achievement. It's a high-risk claim, to assert that the development of a teacher evaluation system can also document which a priori behaviors are best. What may be easier is the collection of videos of different teachers, with a broad enough sample that some will turn out to be great teachers. Gates also highlighted two project districts in AFT: Hillsborough, Florida, and Pittsburgh, Pennsylvania. As is common with description of risky projects in early days, the rhetoric was a bit breathless, and I could hear a few oohs and boos in the audience when he mentioned merit pay, Race to the Top, and tying tenure to student achievement.

Gates ended with the obligatory reference to Al Shanker and the need for teacher voice in reform. "Don't give it back, take the risk, and keep it up." "No other union is doing what you are to make this [reform] happen."

Additional thoughts a few days later: Gates got some personal mileage by appearing at AFT. He spoke with a few reporters afterwards, and his appearance generated some newspaper stories at the St. Pete Times and Washington Post that were more about the Gates Foundation than the AFT convention. At AFT, I don't think delegates had their minds changed much by Gates, since they were likely to be aware of what he's done and where he agrees and disagrees with them.

Gates's rhetoric is compartmentalized. In a good part of what he said, teachers were at the center of what he describes as reform, including teacher evaluation. But then the sore-thumb statement popped out about tying due-process protections to student test scores, unmediated by professional judgment. It's as if there's a switch inside his head, where he can talk either about test scores or about better evaluation of teacher practice. Reform rhetoric as a quantum effect? I don't know. But it's poor strategizing and a poor contribution to discussion. One of the wealthiest men in the world should be able to be more sophisticated.

June 30, 2010

Ooh-datorians

Lovely: another faux trend story from the New York Times, this time about the honoring of multiple valedictorians, and then the easily-anticipated "standards must be dropping!" outcry from those who worry about these things. I remember that my school district started honoring multiple straight-A students in the late 1970s and early 1980s, and weighted AP classes began precisely to address this and make fine distinctions among a bunch of students who worked hard. As the person who was the official valedictorian of my high school class the first year of weighted grades in the school, I can truthfully state that a bunch of my fellow students generally worked harder than I did in high school, and that while I was reasonably proud of a consistent academic record, "valedictorian" has never been a part of my identity, nor has it appeared on any vitae I've written. Valedictorian honors are transitory by their nature: they become public the day they become largely irrelevant except as a mark of the irretrievable past. (The parallel in adult life: as Judith Martin writes, you really don't want your wedding to be the best day of your life, or it's all downhill from there.) To be honest, the most accurate and predictive honor I received as a high school student was the appellation on senior superlatives (the list of "most" and "best" and "most likely to" that many high school classes vote on): doesn't dress to impress. 

Okay, okay, there may be some relevance to the term for those who think that academic competitions are important marks of social values, so maybe we should coin a new term for the group of graduating seniors who have passed a threshold for impressive and consistent academic accomplishments. I nominate the term Ooh-datorian, as in "Ooh... the tenth-ranked student in this class worked harder and did more than the valedictorian in my high school class." And that's true for my daughter's graduating class. I know about half of the top ten students in her class by GPA because, well, my daughter hung out with the dangerous type of crowd that gets together over winter break to study calculus. If I remember correctly, the tenth-ranked student in her class was her friend who borrowed a copy of Calculus Made Easy because she was feeling uneasy (and I don't think she need have), the third-ranked student is her friend of many years who is the hardest-working high school student I've ever met, and so on. (There are plenty of graduating students at her high school whose grades were not as high but also impressed me in many ways, and if they are at all representative of this cohort of teenagers, I feel quite good about the next generation.)

In general, I think academic honors should be proportional to the relevance of the work to one's life. Honor people's academic work in one phase of schooling, but honor lots of things of similar importance, and keep it all in perspective. I am proud to know my daughter's friends partly because they work hard in academics but more because they work hard in more than one sphere of life, and I'd trust several of them with my life.

Yes, Virginia, there is a need for more science education

Today may be a lost day for me work-wise, thanks to a number of circumstances, including a raging thunderstorm overhead that threatens the power where I am. But I don't think I'm going to blame higher ed policy in Florida for all of that. I have a similar reaction to The Real Science Gap, which is an articulate explanation of the dissenting position on STEM education (hat tip). Beryl Lieff Benderly walks us through arguments about overcredentialing/overproduction of Ph.D.s, the abuse of H-1B visas, the STEM report equivalents of the old 1980s Mellon report on pending faculty shortages, the abusive treatment of graduate assistants and postdocs in many universities, the typical stereotypes about clueless faculty advisers, and so forth. She quotes Richard Freeman (who has been specializing in credentialing since the 1970s), and a number of current and former engineering students chimed in with comments echoing the article. And on one level, it is true that the usual cry for more doctorates is often based on misleading claims about national economic needs.

Nonetheless, there are a number of weaknesses in the article, generally of the okay, so STEM isn't immune from the troubles of the world, so what else is new? variety:

  • A significant part of the underemployment problem with graduates in general is the sum of broader economic woes of the world. That's as true for recent undergraduates as doctoral students. There are serious problems about the impoverishment of an entire cohort finishing formal schooling between 2008 and 2012, and I'm not going to pretend to have a great solution except that it has to address more than STEM graduate students.
  • Some of the policy issues can be addressed by institutional policies--for example, by universities' treating graduate students and postdocs better and by universities' agreeing to bargain with new grad-student unions. And that's more important for humanities students than in the sciences, which tend to have higher stipends.
  • Similarly, the "overproduction" of doctorates requires all disciplines to help students figure out career options that don't rely on tenure-track positions. In 1930, the two fields with the highest number of doctorates earned each year were fields where few doctoral students would have expected university jobs: chemistry and education. We need a little explicit "let's list and prepare for at least three options" planning. I'm not on the bleeding edge at all in this argument; some historians have been making this argument inside AHA for years, even if it's not as broad a practice as it needs to be. And I strongly suspect that STEM faculty advisers are among the most likely to have grad students head into industry. One physics doctoral student I know at USF was recently promised a job by Jabil Circuits, a local firm. Go, Jason, and go, Jason's major professor! One of my college classmates who was a physics grad student at Penn when I was a history doctorate student ended up in industry, spending some time on DNA computing and now working for a textile firm. Maybe the people I know are extreme outliers, but the idea of STEM doctorates having industry jobs doesn't strike me as either new or unknown.
  • If the "we need to double our output for economic competitiveness" argument is overblown, the arguments described by Benderly ignore the non-human-capital value of formal education. I am not sure that the type of (non-Clay Shirky-definition) educational surplus I've described before always justifies a huge social investment in postgraduate work, but it's not a horrid thing for society if an occasional chemistry doctorate ends up working in a computer company, and it's definitely a social good if she or he winds up teaching high school chemistry and inspires later generations. (Cue FSU physics professor Paul Cottle here on the need for better starting salaries for high school science teachers...)

And, speaking of high school, I'm in favor of policies that expand general science education. Yes, the "we need much larger programs for our economic future" argument is exaggerated and overpromises. Yes, the Benderly article is a good, thoughtful dissent. Maybe one of those alternate careers needs to be teaching...

I've got a bad feeling about this, Obey

So Rep. Obey (Wisconsin) is proposing to pay for a $10B boost to school districts in part by stripping hundreds of millions of dollars from RTTT and TIF? According to Ed Week reporters, the total offset from this move would be about 8% of the total cost, less than $1B, so I don't really buy Obey's argument that jobs come before reform programs. This looks much closer to opportunism than either addressing GOP objections on fiscal grounds or making the GOP back down sufficiently to get a bill through without offsets.

I'm not thrilled with some aspects of the offset targets, but there's something in this game of legislative chicken that doesn't smell right. I understand the principle of legislative sausage, and this is one move in a much larger policy and politics game. Nonetheless, there are a number of reasons why this filing of an amendment may be close to a thumb in the eyes of a number of federal players:

  • Burning bridges with states. This throws a few tons of egg yolk onto the face of state officials who cajoled school districts into working with them on RTTT proposals. They understand politics, but this is a particularly pointed move a few weeks after the second round of applications were due.
  • Burning bridges with the Senate. Yeah, I know, members of the House have little love for the Senate right now, but if I were in the House and saw HELP (education) committee members criticizing the Duncan four-option turnaround approach, the last thing I'd do is discourage Tom Harkin (HELP committee chair) from criticizing grant structures.
  • Burning bridges with others in the majority leadership. I wonder if Obey's office let George Miller know about this in advance, or if it was a surprise. If it was a surprise, it positions Obey as a problem child in his last half-year in the House. His state desperately needs all sorts of help from the rest of the country; this isn't going to help his leverage.
  • Undermining the credibility of unions. An AFT staff member quickly explained to reporters today the AFT position that they hadn't known what offsets would be used and that they preferred the original $23B bill without offsets. It really doesn't matter whether Obey had any contact with state or national affiliate officers from NEA or AFT; this is a notable distraction that no union leader needs right before the NEA Representative Assembly begins and the week before the AFT convention.

This is Obey's last year in the House, and I know there's a temptation for him to think there's little accountability for pulling stuff like this. Well, he may not have to pay for it, but others will have to. Own goal, anyone?

Accessibility and the e-reader

A brief note about the federal government's warning on college distribute of e-readers and students with disabilities: the concern is warranted, and it's much better for the warning to come now than to come after colleges and universities spend millions of dollars (or require students to spend millions of dollars) without the due diligence needed. Every time I use a piece of technology in teaching, I worry about accessibility for students with visual or hearing impairments. A few years ago, I spent considerable time preparing Flash-based presentations for the first part of an online course, to discover that a student with hearing impairments then had to contact my university's disability-service office for transcriptions, and I decided to switch to written "lectures" for the rest of the semester. I think the communication was better for all students, not just him.

Sometimes you learn through experience, but as Ben Franklin said as Poor Richard, fools will learn in no other school. There are now thousands upon thousands of sites built upon technology with limited accessibility, notably Flash. So, for example, Sandra Day O'Connor has spent untold hours helping develop several solid online games to teach civics, which you can find at icivics.org. But they're Flash-based. That limits accessibility. Yes, I know Flash has developed accessibility tools, but at least one of the games requires quick responses, and ... well, it's a great concept, and I hope that there's a paper version of it available for teachers who decide their students need a paper version to slow things down and make that game more accessible.

The safest technology wrapper for texts or other course materials is a plain-text file, which people can put into all sorts of programs to help them. Following that is a standards-compliant website. There are now tools to make websites touch-accessible for mobile phones, and focusing on websites will probably be a much wiser use of resources for most education technology outfits than creating Android or iPhone/iPad apps.

There is a possibility, as noted in the article, that this is a way for federal officials to use universities to push the publishing industry into allowing accessibility tools in all e-reader devices and programs. If so, it's no more an abuse of leverage than the use of colleges and universities to advertise e-readers (which is part of the role of these early-adopter "give an iPad to a frosh" programs).

June 24, 2010

Botched credit hours and blotted copybooks

A week ago, Ed Sector's Forrest Hinton asked six questions about higher-education accountability. Below are the questions, my quick responses, and some discussion about the fallout from this month's hearings on accreditation and for-profit institutions:


Q. As the monolithic traditional university begins to break down and diversify, should we continue to trust providers of higher education and accreditation agencies to provide meaningful accountability?

A. What "monolithic traditional university"? There's never been such a creature in history; there has always been tremendous diversity of institutions. For the performative culture of accountability, see this morning's IHE column by Cliff Adelman.

Q. Is there a sound method of measuring student learning outcomes in higher education that won't turn college courses into workshops where students learn simple facts, algorithms, and skills?

A. Yes, but whether it's politically robust is a different question, and I think you have to give up on the singular form. The Utah Tuning project report placed on the Ed Sector blog page is both promising in terms of the faculty engagement in the project and also curious in the very different levels of detail in the expectations laid out for the two disciplines in the project (history and physics). See below for the short-term landscape in more detail.

Q. Now that the federal government is providing a lot of higher education's revenue through student loans, how much responsibility does the government have to ensure quality and monitor costs?

A. The federal government has been subsidizing college loans for decades, and since the early 1980s the bulk of federal aid has been in the form of loan subsidies rather than direct grants. What has changed in the past few decades is the cost-shifting from states to students and their families. So maybe the federal government feels more inclined towards wanting something for the dollars because the cost-shifting also shifts somewhat more support onto the federal government, but lower state support hasn't been accompanied by lower demands for accountability.

Q. Is there a trade-off between innovation and regulating quality in higher education? If so, what is the appropriate way to balance these competing forces?

A. This question is ambiguously phrased, and I am interpreting as a question about for-profit schools. (The last question is about credit hours, which covers distance education given this month's politics.) As we learned from the shadow banking sector and the financial crisis of 2008, regulators are frequently way behind creative people who want to make a buck, especially if something has been deregulated because people think history has ended (e.g., the repeal of Glass-Steagall). Right now, for-profit companies of all types peddle degrees or the mirage of a college education with widely varying claims of success, and they have become remarkably adept at vacuuming up federal funds. Some administrators in different institutions tell me that's not only in terms of college loans but also G.I. Bill funds, though I don't have independent confirmation of that claim. To the extent that taxpayers and students are on the hook for loans, that practice has to have some scrutiny. In the early 1990s, the screening mechanism was default rates. Today, I think another measure is needed, but I'm not sure what that might be, maybe federal subsidy per graduate (with some calculation of the effective subsidy for loans).

Q. If higher education's traditional accountability structures are unable to provide adequate oversight, can we make use of other ways of ensuring quality instead, like through informed consumer demand?

A. Again, what "traditional accountability structures"? Name one that has existed for at least five years in any state and that has remained stable. With all of Adelman's caveats, maybe it's something we should try.

Q. When measuring course credit hours, how do we allow innovative new approaches in education to meet the standards implied from traditional rules on seat time, lab experiences, etc.?

A. This is obviously the hot topic du jour, since regional accreditors have looked the other way while some for-profits essentially bought accreditation by absorbing a few nonprofits and since at least one regional accreditor also paid little attention when a for-profit passed off a tuition-generating mechanism as real education. So let's explore this a bit more...

The difficulty with a Wild West of education is that you don't know what a course means. When an unaccredited institution is taking students' money but it's not telling the world it's anything but what it is, and if the students aren't lying about what the diploma means, it's just an experience someone pays for. Are you paying for classes in aerobic yoga weightlifting? That sounds like something taught by the Macho Yoga Instructor my mother once had years ago, but if it's your money and just your experience, that's fine with me. It's different once public funds are involved and once credentials have an exchange value in the labor market, and even more complicated if you're a student having taken courses at two colleges and wanting those courses to transfer into a third to help you get a degree.

Given the background to this morning's hearing in DC, there are clearly bad actors (or bad actors), and the temptation might be to have a rigid definition of what a credit hour is for course purposes, either for federal student-loan purposes or for transfer purposes. For the moment let's skip the question of entirely-online class and talk about about courses that blend some class time with other experiences such as online discussions, tutorials, etc. What counts as a credit hour: the time you spend in class, the time you spend actively working on assignments, the time you vaguely think about the course? What about people who read at different speeds: does the slower reader sign up for and pay for more hours for a course than the faster reader?

The reality is that the credit hour is an institutional convention that is malleable to help everyone account for student progress through programs as well as for tuition purposes. A good example of the malleability is in performing-arts programs. No matter how long a performance music major practices for it, my guess is that a college symphony orchestra class will always be one or two credit hours, no more, because that is the way to address the conflict between wanting students to be in performance classes every semester and also graduate without having to pay more than an engineering student: you require performance and studio classes every semester, but the total of all classes (including theory, music history/ethnomusicology, electives) doesn't add up to more than 15 hours in a semester.

One feasible way to address the credit-hour issue is to have disciplinary conventions for classes that need cursory vs. more extensive inspection, something that factors in both the nature of the discipline and the credit-hour load. A performance class that's one or two credit hours? Let's definitely not worry much about that. Undergraduate U.S. history class carrying three credit hours that blends one hour of lecture, one hour face-to-face discussion, and one hour of online activity? That's a conventional discipline and credit-hour load, with a slight bit of innovation: a little more scrutiny. Vague class in an unorthodox or vocational program that's 9 hours? Let's worry a lot more about that.

In terms of the giant leap that some are going to suggest: should we have institutional-level assessment for every class that can hold colleges and universities accountable? That is more likely to work for limited courses that everyone (or almost everyone) takes in the first two years than for the broad range of classes students take in their majors. Regional accreditors are now pushing institutions to develop such institution-wide assessment for general education programs, and while I am concerned about some of the consequences of that, it is at least plausible to have common assessments in composition, first-year calculus, and so forth. But something for the Celtic Civilization course my wife took with linguist Nancy Dorian? Good luck! (For those who are curious, it was a culture class, not a language class. I signed up for it initially but had to drop it, much to my regret.)

Some observers have argued that the likely fallout from the for-profit hearings will touch far more than the for-profits, and that's right for several reasons. One is that high-tuition institutions that are either for-profit or non-profit will be involved in a disproportionate amount of subsidized loans than low-tuition institutions simply because of tuition, so they will invite scrutiny. Second is that the questions about online classes in for-profit institutions are very close to the questions that you can ask about non-profit private and public institutions' online classes. Third is an institutionalized consequence an administrator and I were discussing this week: the hearings, any changes in law, and any changes in regulations will affect regional accreditors, who in turn will push additional mechanisms down on all of the institutions they oversee. I suspect that from a paperwork-burden perspective, accreditors will have to slice up the oversight mechanisms in some way to avoid peeking into every single course. It may not be my suggested slice, but if they don't perform some triage, oversight is just unworkable.

Finally, I think I misspelled Barmak Nassirian's name in a comment on IHE in the last week, but we all need to learn how to spell his name correctly since we're going to be reading a lot of what he writes and says in his capacity as associate executive director of the American Association of Collegiate Registrars and Admissions Officers (AACRAO). During the life of the Spellings Commission, Nassirian was the go-to person for many higher-ed reporters, and I suspect we'll be hearing a lot from him in the next half-year or so.

The value of college IV

In comments over the past few weeks, Glen McGhee has been doing a lot of work making the argument for a credentialist lens for higher education. And two weeks ago, Jose Vilson's Memorial Day blog entry raised the perennial question of what we're supposed to be educating our children for. So let me address the obvious questions that I haven't answered in the set of blog entries on the value of college. Roughly speaking, my argument is that in the absence of great social upheaval, social institutions tend to have an inertial relationship with the rest of society, and that the school-adult income relationship is an example of that inertial relationship. That's not to say it's just or predetermined or hermetic or even stationary. Rather, it's a statement of the importance of institutional structures once they're set up. Schools can maintain inequalities, they can help students change the world, and all sorts of mixes in between. In other words, formal schooling is a tool up for grabs.


We should not be surprised that without some sort of pressure otherwise, schools would tend to maintain social inequality. That's not because schools are particularly nefarious but because as Charles Tilly argued, humans tend to hoard opportunity for those close to them. So the more advantaged parents, families, and social groups in a particular society would use any childrearing practice as a vehicle for maintaining advantage, and without countervailing pressures, they'd have more options to do so than less-advantaged parents, families, and social groups. The same was true when work occupied more of the lives of children between 10 and 15 than schooling, so why should we expect anything different when formal schooling became more prominent as part of childrearing?

Except that things did not stay the same. As many have noted before me (including Karl Kaestle, Martin Carnoy, Hank Levin, Ira Katznelson, Margaret Weir, and others), the early nineteenth-century North witnessed dramatic expansion of school structures and, almost as importantly, a different way of talking about schooling. Horace Mann was not the first prominent advocate of education as a right; local Workingmen's parties were by the late 1820s, and over a few decades the advocacy of multiple parties expanded the education-citizenship link from a "schooling promotes citizenship" to "schooling comes along with citizenship." The story is long and messy in the nineteenth century, but among other things, that broadened connection was at the root of the mid-century lawsuit against racial segregation in Boston schools (that's mid-19th century), the relationship between compulsory education and compulsory attendance, the power of the state vis-a-vis parents, and so forth.

No one should pretend that Workingmen's parties said "education is a right" and all shouted "Hallelujah!" Far from it; the meaning of education as a part of citizenship was and remains contested. The notion of education became part of a citizenship bundle (what Europeans would call social citizenship, or the American version of it), and as Jennifer Hochschild and Nathan Scovronick point out, it's a mixed legacy. On the one hand, it provides a lever by which millions have been able to acquire better lives. On the other hand, it has also become the lever on which we rely too much, expecting one institution to solve so many social problems.

Those who wish to use education to address inequality need to think about the multigenerational long term. Part but only part of inequality can be addressed directly in a human-capital sense. Far more has to be addressed by equipping large chunks of the population to change society in other ways, and education is an indirect lever there. W.E.B. Du Bois understood the long game, and despite his young-adult romanticism with social-science research in the Progressive Era, he was persistently thinking about the long game for an entire population. His debate with Booker T. Washington was largely about teacher education: Washington publicly argued that primary-teachers for (and most community leaders among) African Americans in the South had to accommodate racism, with advanced academic training a luxury. Du Bois argued that the new colleges for African Americans (the core of what we call HBCUs today) would inevitably train a disproportionate number of teachers and had to support academic ambitions over multiple generations. His Talented Tenth argument was not about elitism but teachers for mass education.

In part the argument in favor of expanding college experience is not that it will pay off immediately for every student who attends college but that it will pay off for the society and for college students' children and grandchildren. On an email list some years ago, I expressed skepticism when one list member argued that formal schooling was essential for social activism. There were plenty in the 1950s and 1960s civil rights movement who had no more than an eighth-grade education and put their bodies on the line because they knew what was right. But it helped tremendously that some key roles were filled by African Americans and others who had a college education, a law degree, and so forth. Segregationists had some very well-educated, savvy people working on their side, and it was important to have equally well-educated, savvy people working on the side of civil rights.

That work shifted schooling in a better direction. Not perfect, but significantly better. The structures of formal schooling, including credentials, student aid, legal nondiscrimination requirements, etc., have left formal schooling moving in a different direction from 100 years ago, but the accumulated changes themselves have imparted a certain momentum to the relationship between schools and society. The role of schooling right now still is weighted towards wealthier families, but there are significantly more opportunities for poor children to improve their lives through schooling than 50 or 100 years ago. That doesn't leave schooling as a cure-all, nor does it excuse us from working towards improving the lives of people in other areas, but it gives me some optimism that we can change the way that schools provide differential opportunities, if we push hard enough and cleverly enough.

That leaves me in a somewhat odd mood towards expanding college, pushing an instrumental formal experience in hopes that all the stuff that isn't planned does even more than what is.

June 23, 2010

Collegiate Lying Assessment?

I should be asleep, but it's summer, we've released our teenage son from stricter bedtimes, and he is practicing sax. Most of the tasks on my plate require a little more concentration than I can muster after midnight, so I will write instead about lying, or whether we can tell much about someone's skills when we put them deliberately in a decontextualized situation where either the situation is a known lie or where the individual in question can get ahead by lying.


What put that into my head was the description of the Collegiate Learning Assessment (CLA) published by creators of the test. The CLA is one of the three supposed assessments of college learning that comprise options in a part of the Voluntary System of Accountability, and apart from the controversies over generic assessments of college graduate learning and the mediocre statistical properties of the supposed value-added measures of non-longitudinal samples, it's important to look at the details of the assessments themselves.

On its face, the CLA looks like a plausible assessment of reasoning skills in a written context. As described by its creators, the CLA sometimes consists of a performance task keyed to a simulated case with attended (fictional) documents, and sometimes it is a prompt to critique a specific argument. The samples provided were both from public policy--specifically, crime. Thus far, it looks something a cross between high school debate and the AP history "document based questions." I've constructed some assignments around fictional cases as a way to fine-tune what students have to confront and how it ties in to the issues they need to address. And it is common enough for essay prompts to quote someone's opinion in the topic at hand and ask for a critical assessment. As I said, it's plausible on its face.

But a funny thing happens once you remove either type of task from the subject in which it's embedded: those who are rating student responses do not have the substantive expertise to check student assertions. If someone responds to a simulated case in my class with statements about education research that are clear misunderstandings of course material, they're not going to get an A. Same with a response to a "please evaluate this statement" prompt. With the CLA, however, there is no such check unless the human rater happens to have substantive expertise aligned with the prompt (in the samples, criminology, sociology, or government). And even there, the scoring guidelines appear to ignore the veracity of student statements. It is entirely about whether someone can construct or criticize an argument in response to prompts.

In this particular case (with a prompt about crime policy), suppose a student lied about criminology research--made up four names and said that they were famous criminologists who had conducted research in effective deterrents. How should such a response be scored? I think I know the answer, because K-12 students in Florida are sometimes encouraged to make up details for the state's writing exam. As far as I am aware, such fabrication is rewarded as success in providing "artistic verisimilitude to an otherwise bald and unconvincing narrative" (William Gilbert, The Mikado). If you know something to the contrary about the CLA, please point me to it in comments, but nothing I've read thus far is particularly reassuring on this point.

Maybe that's what we should be doing in college, producing Sophists who can turn a nice phrase and fake their way through school, through job interviews, through a daily three-hour radio political talk show, or through professional reports about things like bridge safety and oil-drilling backup systems.

Or maybe we should acknowledge that if it's to have any value, a general-education program has to have some substance, and assessments of its success need to be rooted in the areas it putatively requires some learning in. Not writing and reasoning in general but writing and reasoning about the stuff that's in the gen-ed curriculum.

Designed to reward age discrimination?

The French government is proposing to raise the age when retirees can draw down full pensions, which sounds remarkably like the 1980s reform of Social Security, including the bump of two years, except that France is going from 60 to 62 and we're already at 67... and a logical compromise for extending the solvency of Social Security indefinitely would combine lifting the cap on income taxed by Social Security and another hike in the full-benefits age from 67 to 69 or 70.

But there's a problem, as Matthew Yglesias points out: the growing population of long-term unemployed in this recession are tending to be (or at least disproportionately are) older workers who will have a much harder time finding work than young adults with a college degree.

... which lands us in the lap of teacher layoff discussions and state pension plans. Many of the arguments against basing layoffs on seniority are focused on the presumed incompetence of some unidentified segment of older teachers. Sometimes that's thinly-disguised age discrimination, sometimes it's not. But let's suppose there isn't even the thinnest veneer of age discrimination. What would happen to someone in their late 50s who is fired or laid off by a school system? Sometimes older teachers have careers after they retire, but I suspect a good portion don't, and I don't think it's smart public policy to ignore labor-market discrimination. State or local pension plans with health benefits can provide a buffer against the labor market because a teacher who is eligible for a pension at 55 or 60 can probably make ends meet until Social Security kicks in, especially with at least a part-time job.

Then again, some of those who would love to destroy tenure also would love to remove public obligations to pension plans. So I have one question for the proponents of the combination: what do you expect someone to do if they're 57 and fired or laid off and the pension and retiree health benefits they expected suddenly evaporate? I can understand if an individual employer says, "That's not my problem," but it's lazy wonkery to propose a set of policies that make people highly vulnerable to age discrimination and then walk away.

June 22, 2010

I read Giroux recently and winced...

... and then wrote an essay on another blog on how to save teacher education. Or not if your mileage varies from mine, but at least you get to read me swear.

June 19, 2010

What uses of test scores will pass legal muster in teacher evaluations?

Legal considerations on the use of test score derived stats in teacher evaluation: Scott Bauries started an interesting discussion June 2 of value-added measures and teacher evaluations from a legal perspective. It's very important to read the comment thread, as he's challenged on his conclusions by Bruce Baker and Preston Green, especially with regard to disparate-impact claims. Bauries claims that employers need to defend the procedural due process but are probably safer on the substance, regardless of the problems with value-added measures.


Reading the main entry and discussion, I lean strongly towards' Bauries' conclusion, with one important caveat (below). My impression of the 2000 G. I. Forum v. Texas Education Agency case on the disparate impact of high-stakes graduation tests, which the state won, was that the plaintiffs were not prepared for the last burden-switching test on disparate impact. My rough impression of disparate-impact claims of illegal discrimination based on the Civil Rights Act: it's a series of penalty kicks/shots in soccer/hockey or maybe the games with alternating possession in overtime. I'm not a lawyer, and this is primarily based on my understanding of Title VI rather than Title VII law, but to the probably-inapt analogy: First, the plaintiffs try to demonstrate that a mechanism such as a test affected a property interest of the plaintiffs and had a disparate impact on one of the protected classes. If the plaintiffs succeed, the defendant tries to demonstrate that the mechanism meets an important interest, was properly constructed and applied, and members of the affected class had a fair chance at succeeding in the mechanism.

So far, we're describing lots of situations that have evolved in the past 25-30 years, especially with high stakes testing. Debra P. v. Turlington established the basic federal expectations in terms of student tests, and as a number of states created a new round of graduation tests in the 1990s, they relied on Debra P. v. Turlington as a guide to meeting the basic questions and getting to the final round all tied up. And this sort of makes sense if you think about the maturity of various mechanisms: you can argue that there is a rational state interest in a certain outcome (an adequate measure of achievement in the case of graduation requirements), and then satisfying the "fair chance at succeeding" is often a question of satisfying a set of criteria rather than perfection and that's often a reflection of the organization's experience and capacity.

The final test is whether there is a better option: could the defendant have feasibly chosen an alternative mechanism that satisfies the same interest with less impact. I've never read all of the materials in the G.I. Forum case, but the following is a key passage in Judge Prado's ruling:

The Plaintiffs were able to show that the policies are debated and debatable among learned people. The Plaintiffs demonstrated that the policies have had an initial and substantial adverse impact on minority students. The Plaintiffs demonstrated that the policies are not perfect. However, the Plaintiffs failed to prove that the policies are unconstitutional, that the adverse impact is avoidable or more significant than the concomitant positive impact, or that other approaches would meet the State's articulated legitimate goals. In the absence of such proof, the State must be allowed to design an educational system that it believes best meets the need of its citizens. (emphasis added)

In the end, the plaintiffs' lawyers in the Texas case were unable to provide a clear alternative to high-stakes testing that they could demonstrate was both feasible (i.e., wouldn't cost an arm and a leg) and would have a lower disparate impact. I'm not too worried about the state interest, since you can usually construct alternative mechanisms that have facial validity and that have roughly the same "noise" as whatever you're arguing against. And the not-an-arm-and-a-leg criteria is tougher to meet if you're arguing for portfolios, since it increases the cost... but it starts from a relatively low base of cost per-pupil. Ultimately, though, it is hard to argue that a prospective alternative would result in a lower disparate impact if it is only prospective and thus you have no evidence whether the protected class you're worrying about would be helped by the alternative.

So in the discussion over at EdJurist, Bauries's clinching argument is really that for all their flaws, value-added measures are going to look reasonable to a judge in that they try to adjust for incoming achievement of students and plaintiffs will have to put forward an alternative with concrete evidence that the alternative does a demonstrably better job at treating teachers fairly. The catch-22: without a working model of alternatives with that record, plaintiffs are going to be sunk on disparate-impact claims.

Bruce Baker has followed up on Bauries with a set of tongue-in-cheek impossible criteria to make the use of value-added measures reasonably fair. I understand the temptation, but he's onto one thing: ultimately, local K-12 unions will have to figure out how to respond. This will include whether they have separate evaluation procedures for the 20% of teachers for whom value-added measures are even possible, how to mix the data, and so forth.

And now for the caveat: a good part of the legal consequences of using student test scores for personnel decisions will depend on how stupid local administrators are in the first jurisdictions to use them, and the first that are challenged. I can imagine districts where administrators are careful to fire experienced teachers only where there is a record of several years of low statistical measures of student achievement and only where that is consistent with low marks in other areas, such as administrator and peer observations. I can also imagine districts where administrators purge teachers based on a single year's worth of data and with no checks of consistency with other sources of information. If the legal tests are in jurisdictions with the first set of practices, they're far more likely to pass muster than if the first cases are for terminations that don't meet a basic smell test of rationality.

June 14, 2010

Isn't Florida guaranteed a republican form of government?

The first draft of the new Florida Educator Accomplished Practices was released last month for public comment on a form that allows almost little substantive feedback. The accomplished practices are supposed to serve as the minimum elements of educator preparation programs, and I understand the balance between micromanagement and platitudes is hard to strike.

The draft is brief and reduces several of the current list's main items into subsidiary bullet points under seven main items in the draft revision. Ethics, critical thinking, and role of the teacher completely drop off the revision's first draft, and the first draft reduces diversity and cognitive research to bullet points. One item that worries me is the downgrading of cognitive research into subsidiary elements of main expectations. Given the prevalence of educational fads, teachers and administrators need to have a solid grounding in educational psychology.

As an historian, what worries me about the draft in addition is that you could transplant it to China or many another undemocratic country, and most of it would fit reasonably well. That shouldn't be the case: teachers in Florida should know that they're obligations are tied to the relationship between teaching and citizenship. So I've proposed an additional accomplished practice I've termed democracy and citizenship and covers issues such as ethics, nondiscriminatory treatment of students and families, and pushing civil debate. We'll see if the task force agrees with me in any way on this.

The currency of higher education in America

I winced reading today's Inside Higher Ed column by Arthur Levine and hoped for a second that someone was impersonating him. I checked it against my Higher Education Commentary Bingo Card:

bubble industrial society dinosaur unchanging decrepit
distracted digital native dumbest generation swirling millennial generation
seat-time anytime/anyplace

Free (by Chris Anderson)

mobile or ubiquitous learning individualized
helicopter parents passive/active dichotomy seamless outcomes pampered
reengineering students as consumers incentivized P-20 accountability

Looks like the piece hit every item in the second column.

Levine is consolidating a set of stereotypes about higher education that is only tenuously connected with real colleges and universities. The column is written as if almost every student is an 18- to 22-year-old with an iPad and an iPhone, a BitTorrent user, and a habitual plagiarist. For any who is tempted to describe college students in this way, please look at the real students in most colleges; a substantial fraction may fit this stereotype, but it's still only a fraction. And whatever flaws today's college students have, I suspect our predecessors saw them in spades 50 years ago. I would plead the same with the column's implication that college and university structures and curricula have not evolved over the past century; look at the proportion of courses taught online or by adjuncts and tell me again how universities don't change and don't see students as consumers. Even the one item I'd otherwise be willing to give a pass on--"All education is essentially remedial, teaching students what they do not know"--implies that education is the same as knowledge. Ouch.

The painful part of reading this morning's column is not only the blithe acceptance of stereotypes but the failure to see that higher education cannot avoid having some unit of currency. Like many other pieces I have read recently, this morning's column calls for a move away from the student credit hour. With the millions of transfer students in the country, colleges and universities need some currency system to treat them fairly and process the request to bring some of their work from other institutions into the new institution. That is unavoidable, unless you want students to start from scratch at every institution. But let's imagine a world where colleges and universities no longer count seat time. So the student credit hour would be replaced by what, precisely? Some propose a list of competencies, but that's still a countable currency (if in tests/assessments passed rather than courses), and then you'd have to create competency assessments for every conceivable course in the world that a transfer student might have taken somewhere else. Does anyone really believe that's a more viable structure than credit hours/courses?

June 13, 2010

The teacher jobs bill is still needed and sensible

Rick Hess's blog continues to be interesting and well-written, and while he is out of the country, he's recruited as substitute bloggers two of the new generation of political scientists focusing on education, Patrick McGuinn and Paul Manna. They have been a welcome addition to the field, and I look forward to what they have to say in Hess's space.

Hess continues to talk about the $23 billion education jobs package as fiscal profligacy, and I gather that he agrees with those arguing that additional short-term stimulus spending would be a significant contributor to long-term national debt, in addition to delaying a (presumably deserved) day of reckoning for local school districts. I've made the argument before that the colonic model of school reform (and public-sector reform more generally) is an attractive hypothesis but one without an empirical basis put forward by its proponents (including Hess). Yeah, I know, Rahm Emmanuel talked about not letting a crisis go to waste, but we in education tend to declare too many crises for that to work too well.


I suspect I simply disagree with Hess on short-term debt and long-term national debt. I am persuaded by a number of economists that the far greater threat in the long run is posed by either health-care costs (Medicare and Medicaid) or by the shortage in revenue caused by the recession. In addition, I am also convinced that inflation and interest rates are too low for us to have too many options right now. Like Paul Krugman, I think the initial stimulus was far too small, and I see the mini-packages rolling through Congress in the past half-year or so as a necessary supplement.

In that context, saving teachers' jobs is a plausible component of additional macroeconomic stimulus, and those arguing about whether it should require elimination of last-in-first-out layoff priorities are playing chicken with a Mack Truck: damn it, get the package through, or the only useful function you'll have in the fall will be the proper orientation of your index finger.

From a macroeconomic standpoint, I suspect the question is what would provide the greatest bang for the buck, keeping people employed or spending the money on any other function. I suspect that you could find a group of people who have a bigger stimulus impact than teachers and other school-system employees: the majority of civil servants are probably paid at least a little less than the median teacher, and more of their income might be recycled into the economy. That's an empirical question, though I suspect that differences between the stimulus effect of keeping on "relatively low-paid professionals, clericals, and service staff" vs. "even lower-paid professionals, clericals, and service staff" are going to be swamped by any differences between "keeping public employees working" and other ways to spend money.

It's important to keep in mind that Hess is far from the only conservative downplaying the budget woes of schools. About half a year ago, Arthur Peng and Jim Guthrie penned an article for Education Next about how everyone was constantly crying wolf over public-school funding. There is a significant truth in the article along with several important flaws. What Peng and Guthrie have correct is the history over 60 years of generally buffering public-school funding in crises, if you look at national-level data. That is because we associated schooling with citizenship in both legal and political ways, and that helps insulate schools from the effects of economic downturns. (They also mention a number of other mechanisms.) There are some important weaknesses in their article:

  • The past is not prologue: I'd love for treatment of public schools in minor downturns be perfectly predictive of what happens as we (very slowly) leave the worst worldwide economic crisis since the 1930s. Peng and Guthrie imply that the behavior will continue automatically, and I am unconvinced.
  • Per-pupil revenues do not address infrastructure/capacity: In the late 1970s and early 1980s, two recessions in quick succession hit districts while many were also struggling with a decline in student enrollment with the back of the Baby Boom. Looking at per-pupil revenues obscures the fact that a number of districts engaged in repeated years of RIFfing teachers (RIF = "reduction in force," the term commonly used at the time), with small reductions in per-pupil revenues magnified greatly by enrollment declines. We're again on the downslide of a baby boom (if the baby boom echo), and regions with enrollment declines will face greater budget woes as a result of compounding events.
  • The nation is not the district: even in prior recessions, there have been school districts and states where school funding suffered far more than what you may assume from the charts in Peng and Guthrie's article, and demographic unevenness compounded that lumpiness. Enrollment declines were not felt in the budgets of Sunbelt state schools in the early 1980s, but they were prominent in the Northeast and Midwest.
  • Law is not reality: Peng and Guthrie state that education is treated budget-wise in the way that constitutions are written (i.e., children and education first, sometimes explicitly so). I wish! In reality, while education funding shifted towards the state level in the 1970s, states have also greatly expanded funding of health care and prison systems. I suspect that Blago will shortly be enrolling in a graduate program at Statesville U., if you know what I mean, but apart from the education of juveniles in state custody, that's not really the type of spending we should think of as schooling. (Yes, I know Statesville is a maximum-security facility, but Joliet closed in 2002.)

Bottom line: I wish I could share Peng and Guthrie's view of schooling as largely buffered from recessions, and I wish that I could fear long-term debt from a stimulus more than I fear lingering high unemployment, the way that Hess does. But as long as inflation and interest rates are close to zero and unemployment lingers in the 9-10% territory, we'll need fiscal stimulus, and supporting teacher jobs is a logical way to do it.

June 11, 2010

Remedial/development education, required reading

If you're interested in community-college remedial/developmental programs and you haven't yet, go read an excellent feature by Bill Maxwell that appeared last Sunday in the St. Petersburg Times Perspective section. Because it's in the paper's opinion section, Maxwell is free to add his judgment, but for the most part this is just a good feature, detailed and thought-provoking. It deserves more attention than it's gotten thus far in the week.

Why is a college education like a tulip bulb?

Dean Dad has one plausible response to the latest installation of the "college is the next asset bubble to burst" argument, and every time I come across it I grind my teeth, think of ravens and writing desks, and go on. At least Glenn Reynolds is neither an economist nor an historian, or I'd accuse him of professional incompetence. Hint to all who might think he's right: a college degree is not an excludable good that is the type normally resellable on a speculative basis. But at least I have material for this Out of Left Field Friday entry...

Some part of the argument regularly floated on this topic is an anticipatory taste of Schadenfreude: "I just can't wait for the bastards to get their due," with higher education standing in for all bastards here. As many people before me have pointed out, Schadenfreude isn't a wise basis for public policy, and desire for it tends to blind one to analytical details. Most students are not in the type of tuition-dependent institution that Dean Dad rightly points out is the only part of higher ed vulnerable to a "oh, we can't spend as much as we'd like" change in behavior. Millions still want a college education, and if they can't afford private tuition or out-of-state tuition somewhere else, they'll pop for a four-year university degree or start at community colleges.

At some level, the dissatisfaction with higher education leads to grumbling and sometimes structural changes in public higher ed (e.g., calls for accountability, today more about attainment than cognitive outcomes). Concerns about family costs have led to the changes in student loan policy. Grumbling has not yet led to changes in tax laws that would move the needle on athletic departments or large endowments. And given the labor-market queueing advantage of those with college degrees, you're not going to see people leaving colleges in droves, or at least not "college" in the abstract.

In other words, this doesn't look like an asset bubble to me in any way I'm familair with.

Of side deals, soup and sandwich, and prayers

A few days ago, the St. Petersburg Times uploaded all of the second-round RTTT local MOUs in Florida they had been able to acquire to one of the relevant blog entries, so I can talk about the elements that are common among them. First, a mea culpa: when reading the Broward document on Safari, the browser only showed the paragraph discussing impasse, and I wrote a blog entry based on that assumption. After seeing several of the MOUs, I checked the file format, realized it was not a PDF, and put it into a different app. So my erroneous description of Broward's and Hernando's MOUs as dramatically different is my fault. There are some interesting variations (as I explain below, Hardee should not be included in the broader list), but I'll stick with the commonalities.


I'm going to start with the local MOU signed in January by stakeholders in Hillsborough and other MOUs signed in a handful of other counties where locals of the Florida Education Association signed onto the first state application to Race to the Top. The Times blog entry at the time could be read to suggest that the Hillsborough Classroom Teachers Association (HCTA) wheedled the school board into the language. I suspect the reality is that HCTA asked FEA's bargaining support team to give them something that would allow HCTA to sign on and still secure them (or try to secure HCTA) against impasse proceedings and being locked into an experiment after the money ran out.

So what did the January MOU in Hillsborough include?

  1. The parties agreed to negotiate in good faith to allow them to participate in the state's RTTT program.
  2. The parties agreed that they would not take discretionary bargaining subjects to impasse even if that impeded participation in RTTT (and if the state rejected whatever was bargained, Hillsborough would not participate).
  3. Any contract modifications agreed to as part of bargaining with RTTT would cease either with the end of the grant or the end of the contract. (In Florida law, public-employee collective bargaining agreements can last no longer than three years.)
  4. The school district committed itself to finding non-general-fund sources of money "to continue implementation" if RTTT funding was insufficient.

Again, the effect of this agreement was to guard HCTA from being exposed to impasse proceedings and a default permanent commitment to contract provisions for a program that might not be fully funded after RTTT moneys ran out. The first protection was incorporated into the state-level MOU for the second RTTT application. It is not entirely clear that the state-level MOU addresses the second issue directly, though it does state that the MOU (and the commitments) end with the end of the grant, if the state receives the award. (I'll get a little more into the weeds on the doctrine of status quo below.) In any case, the new state-level MOU was sufficient for the vast majority of counties and local unions this time around, and FEA's leadership encouraged locals to participate (hedging that around a bit in public language).

The existence of local MOUs the second time around is an indication that the reassurances were not enough for at least 10 counties and their locals to sign on without a local MOU. (Stakeholders in the rural county of Hardee signed an MOU that essentially duplicated the state MOU: they'd attempt to bargain to meet RTTT requirements, but they wouldn't declare impasse. So I am not including Hardee in the group of other counties whose MOUs put a termination date on contract modifications.) Three of those counties have large populations: Duval (Jacksonville), Lee (Ft. Myers), and Broward (Ft. Lauderdale). The others are much smaller.

So what parts of the local MOUs match up to the MOU signed in Hillsborough in January? I'm going to match up some of the language of the Broward MOU (why? Broward's the largest system in the set, and I can copy and paste language from its MOU) against corresponding elements in the January Hillsborough MOU and see what remains:

  1. Good faith bargaining: The parties will use best efforts to develop a negotiated, mutually agreed upon implementation plan in the areas identified by the parties as part of the Plan (from item 2 of the Broward MOU).
  2. No impasse: Any items relating to the RTTT Application or Plan that are unsuccessfully negotiated between the parties specifically for the purpose of applying for or receiving the RTTT grant award will not be subject to the impasse procedures set forth in Chapter 447. The impasse procedure is herewith deemed waived by the parties as to negotiations which are for the specific purpose of applying for or receiving the RTTT grant award (item 4).... Should there fail to be a fully ratified MOU by the bargaining-unit and non-imposed agreements after good faith negotiations for RTTT, the parties are released from any obligation to continue participation in the Race to the Top Grant (item 8).
  3. Termination date for contract modifications: In the event that negotiations for RTTT result in modification to the existing CBA, such modifications will expire upon either the expiration of the RTTT grant or upon the expiration of the funding of the grant whichever occurs first (item 5).

Those pieces parallel three of the four elements of the January MOU in Hillsborough (and I've put the Broward MOU text in the same order as the list above). That doesn't mean that they don't have policy implications, but it's not as if this language came out of nowhere; the provisions clearly came from the January MOUs. (Discussion of policy below.) What's new?

  • Recognition that RTTT may require impact bargaining as well as mandatory subjects of bargaining: If an RTTT grant is awarded, any items in the Plan that impact wages, hours or terms and conditions of employment or that may modify the current CBA are subject to bargaining in accordance with Chapter 447 (item 3).... Portions of the RTTT Application and Plan implicate mandatory subjects of bargaining under the Florida Constitution and Chapter 447. The parties acknowledge that limited issues such as performance pay, salary schedules, and teacher placement can best and most effectively be addressed at the local level (item 7).
  • Agreement to the state MOU does not commit the parties to specific contract changes: [A local stakeholder's signature on the state MOU] does not constitute agreement to modify the existing CBA or to negotiate additional language consistent with all elements of the FLDOE Preliminary Scope of Work (item 1).... The signature of the Union President on the FLDOE MOU does not constitute an agreement to (a) reopen or otherwise modify the CBA, unless and until a subsequent negotiated time specific waiver or other agreement has been mutually agreed upon by the BTU and SBBC or (b) limit or waive its rights and protections under the Florida Constitution, the Florida Public Employees' Relations Act and other applicable laws (item 2).
  • Contract modifications for RTTT do not become the default contract language that extends beyond the end of a contract while negotiations continue: If bargaining according to this MOU results in any modification to the current CBA in order to comply with RTTT requirements, then such modification(s) will not operate as the status quo and shall have no precedent setting value, face or effect, unless to the extent agreed to be the parties therein (item 6).

The first topic is mundane and would probably be accepted as a statement of reality by most involved in Florida public labor negotiations. The second category protects parties (primarily unions) from the argument that a signature on the state MOU was the same act as signing a labor contract or a binding commitment on any specific element of a contract (from the first numbered paragraph of the MOU). More interesting is the language from the second numbered paragraph that refers to "a subsequent negotiated time specific waiver or other agreement" that results from bargaining. Any part of a collective bargaining agreement that includes a time-limited waiver of union rights (or agreement on a contract provision that is discretionary under Florida law) would satisfy that language. I am not a lawyer, but this may duplicate established protections that unions can assert at the table.

What appears to be truly new in comparison with the January Hillsborough MOU and also a potential substantive difference from anything else is the language clearly stating that contract modifications specifically for RTTT do not become the status quo that continues after the end of one contract if bargaining continues on a successor agreement. Here we're truly getting into the weeds on Florida labor law: A public-employee labor contract continues to operate after the last date if there isn't agreement on a successor, and the terms and conditions of employment continue as if the agreement had been extended. There is some controversy about whether and which waivers of rights by either party continue as the status quo, but the status-quo doctrine changes the nature of bargaining, as a deadline that might otherwise be facing one party at the end of a contract just does not exist, and that provision continues. If I understand correctly, the public-interest rationale for the status-quo doctrine is that maintaining the terms and conditions of employment during a contractual interregnum is easier to administer, and since the parties agreed to those terms and conditions at some point in the past, one can assume that the contract provisions are not contrary to the public interest. But it does theoretically remove bargaining leverage from a party who conceded a term that would otherwise cease at the end of the contract.

Given the language elsewhere in the Broward MOU (and others) that the RTTT-related contract modifications cease as of the end of the contract or RTTT funds, I'm trying to figure out what the additional language on status quo adds. Is this the legal equivalent of putting on long johns on a winter day in Philadelphia just in case the temperature drops another 40 degrees F.? It roughly parallels the same issue--"we're not locking ourselves into the experimental contract language we may agree to"--with different language.

The two questions that have been raised have been of process (transparency in the state's application: will federal reviewers see everything that's relevant?) and the substantive question of commitment to contractual changes after the grant money runs out.

I don't think the transparency issue is affected by the language about impasse, since I'm persuaded that the local MOUs' language effectively duplicates the state MOU language on the issue without adding anything substantive. And the state has the authority to include or exclude specific counties based on information state officials have; remember that this is an application by the state for money, not an application by individual school districts. So we're down to one issue: does 10 local unions' unwillingness to be locked into specific contractual language in advance of the grant directly contradict the application's claims of stakeholder buy-in?

Well, that's not quite it, either, because I don't think the other locals want to be locked in either, but they didn't sign local MOUs with language on the issue. So it's really whether the application reviewers have the information they need to decide if the state as a whole has sufficient stakeholder buy-in to earn the relevant points in that part of the evaluation. Suppose for an instance that we should wipe out those ten counties from buy-in based on signing the local MOUs; would Florida lose any points? Perhaps a few.


So a man walks to the corner deli to get lunch and orders a soup and sandwich. He pays and waits at the pickup counter. When the short-order cook rings the bell and puts the tray on the counter with the order, the customer asks the cook, "Did you pray while putting together my meal?"

The cook squints a bit. "Did I pray?"

"Yes. Did you pray for me while putting together my meal?"

The cook thinks. The customer's a stranger, there are seven billion people in the world, and she's busy making lunches. Should she pray for all the strangers in the world or make sure she doesn't cut her fingers off? But the customer's always right, the boss says. "Sure, I prayed."

"What'd you pray for me?"

"I prayed for you to give me a good tip."


Now for the big picture on transparency: you can't read minds. At most what we know is that officials in these ten counties are more suspicious of the state MOU than FEA President Andy Ford and other members of the governor's task force. Do you think that local officials in other states are equally suspicious? Is there a way for a state to suss that out? Officials in every state have assumed that a signature on an MOU is a legal commitment, and the same has been true in Florida. There could be all sorts of local agreements of the nudge nudge, wink wink variety in multiple states, and that's true of every grant program where the state is the applicant and locals participate. If someone wants Florida to be penalized because state officials were unaware of local suspicions and we now know about it because the local stakeholders put those suspicions in writing, do we need to set up mechanisms to find any agreements that are unwritten?

There may be one other issue in the grant application development playing into this: the state asked local stakeholders to sign the MOU before the state application was finished, and there is a section of the state MOU that allows local districts to back out if they discover after having signed the MOU that they don't have the capacity to follow through. (That's shortly after the zipper clause that lets the state exclude local districts.) A few national stakeholders have pointed out that RTTT required states to buy into the common core standards before they were finalized, and the development of the Florida RTTT application had a parallel problem of asking local stakeholders to sign on before knowing all of the details of the state plan. Given the escape clause in the MOU, that shouldn't have fed into discussions about any local MOU, but I wouldn't be surprised if that upped the suspicion level.

The policy question remains the same: what happens after RTTT funding runs out? There are (at least) two ways to look at this, if you're a supporter of the RTTT program. One is to see the state application as a way to lock in certain policies that can't be undone. The January MOU in Hillsborough and the ten MOUs that the Times has uncovered would greatly irritate anyone with that view, because it's evidence that local unions and possibly school boards are going to want to backtrack just as soon as the legal commitments for RTTT evaporate. But there's a second way to look at RTTT: they force states and local districts to experiment in certain ways and have certain experiences. If you think that the experiences with different policy structures are going to change the conversation and loosen up opposition to a bunch of things, you're going to trust that the proof of the pudding is in the eating. Or you have to think that at some level, you can't force local districts to behave in a certain way forever and stakeholders have to agree with you at some point.

I think the first way of viewing RTTT is incompatible with its being a voluntary grant program. If you want to force certain behavior, you can't hide it behind a temporary pot of money, and people are going to seethe and figure out ways to undo it if you try. If you want to be top-down, you have to be top-down and open about it, and limit it to things you are reasonably certain you can control. If you want to encourage experimentation, you need to acknowledge that some of the experiments will absolutely fail and let local districts escape from those failed experiments.

One other matter I haven't been able to resolve: I vaguely remember some of the news coverage this month reporting that both the first and second Florida applications for Race to the Top mentioned at least a few of the local MOUs, but I couldn't find any reference to the local MOUs in either (at least by searching within the PDF files). For the record, can anyone tell me whether my memory is going, or if I just missed everything?

June 9, 2010

Get your performance-pay evaluation report bingo cards here

So another few evaluation reports have been released with little evidence of student achievement flowing from performance-pay systems. This is going to sound like a broken record from me, but I don't make too much out of one or two studies in policy research. These studies on systems in Chicago and New York confirm something any historian (or anyone who's read education historians) could have predicted: even if there is some benefit from changing a pay system, it's a darned hard thing to try. This is one of the reasons why I dislike the boutique, closed evaluation tradition in education research: every evaluation collects data, walls it off, and then presents (only) conclusions to the public. When there are millions of dollars being spent through the Teacher Incentive Fund in addition to privately-funded efforts (or any program with an interesting but untested theory of action), there have to be data archives so that other researchers (those not on the original evaluation team) can conduct secondary analyses.

But having put forward these caveats, I'm going to guess that most studies of performance pay are going to show negligible effects on test scores. This may be my inner cynic (okay, not very inner), but the long-term questions on performance-pay policies revolve less around whether it is consistent with the theory of action proponents have but focus instead on whether the politics demand something regardless of effects and what is workable from a variety of standpoints.

June 8, 2010

The value of college III

Part of the value of a good college education is that much of it is surplus. In the same way that the early nineteenth-century education of women could have been perceived as superfluous, a good deal of what students learn could be seen as not directly or immediately useful in their lives. To some economists, this may smack of inefficiency: why should we educate anyone beyond what we can see as an immediate payback on the job or in life? To others, this gets absorbed in a metastatic notion of human capital, where everything good in life is redefined as investment. (Read the new introduction in the 1993 edition of Gary Becker's Human Capital if you doubt me: not only are schooling and standalone job training considered human capital, so is love from one's parents.) Claudia Goldin and Larry Katz refer generically to education as critical to handling changing technology on the job, which makes a certain amount of sense as long as you're not operating a picture-based point-of-sale register (technology can deskill jobs as well as require greater skills). Goldin, Katz, and Uwe Reinhardt are definitely well-meaning, and I'd want them all at my back in an unlit economics-department hallway. But at some level, the economic justification of surplus education is troublesome because it is a black box (how the extra education works exactly isn't modeled); the slop between formal schooling and economic utility (which I've termed surplus) is a fundamental problem for how economists approach education.


An inefficient education as useful play

So let's turn from economics to anthropology for some help. In 1973, American Anthropologist published Stephen Miller's "Ends, Means, and Galumphing," which explored the social and evolutionary purposes of play. It's reasonably well-cited for a social-science article, but more importantly it's widely cited in areas as diverse as educational and social psychology (where you might expect it to be cited) and... well, it's cited in "Marketing in Hypermedia Computer-Mediated Environments" (1996, in the Journal of Marketing). In other words, it's got legs. Miller argues that one can define play within multiple species as activity that is deliberately inefficient and where the individuals involved gain pleasure from facing challenges that stem directly from the inefficiency, whether we're talking formal inefficiencies such as the rules of baseball and chess or informal make-believe... or activities one might find in college such as analyzing a real or fictional company's operations, writing a history paper, spending ten or more hours talking about a single play of Shakespeare, and so forth.

More importantly, Miller argues that play has some advantage for a species in that it turns specific skills into general problem-solving capacity. In play, one uses skills repeatedly and in a range of combinations. (One could argue a little differently about some videogames I know, but I'm describing his argument, not making my own, and the point would still be important even if you removed videogames that require nothing but exactly-repetitive behavior.) Play looks remarkably inefficient in one way, but it has important adaptive value in another.

So too with much of formal education. I could make the same faculty-psychology arguments on behalf of studying history that many people do: not only does it provide specific knowledge of certain times and places, it also prepares you for any career that requires the presentation of linear arguments with specific time- and place-bound evidence. (Legal brief, anyone?) It teaches you about human foibles and prepares you for situations where you have to suspend antipathy towards individuals to identify potential motives and key interests. David Brooks makes all of those arguments in his column today.

But that type of argument has always struck me as beside the point, not because history majors do not have practice in those skills but because any faculty-psychology argument is easily turned into a nebulous "this will help you learn critical thinking" claim, which my time-and-place-specific training makes me skeptical of. Yes, majoring in history will help you in a lot of fields more than not going to college at all, but it's hard to argue that a history major is better suited to a professional biochem lab's gruntwork than a math or physics major, even if the gruntwork has occasional public presentations attached to it requiring linear arguments with detailed evidence (see above on that refrain).

(Margaret Soltan argues a different point today, asserting that the value of the humanities is in the embodiment of human frailty, not its rational analysis. She writes, "For [William Arrowsmith], a prolonged encounter with the humanistic tradition amounts to a more and more sensate anguish at the recognition of our own chaos." I'm not going to argue with her or Arrowsmith, since I'm sure many a student in a Milton seminar has probably had crises of faith, and I had the odd experience of The Painted Bird as a soothing read at the end of my first semester in college. I'm just making a different point that can stretch beyond the humanities.)

An honest explanation of the value of college acknowledges that when college accomplishes what it can, a good part of that achievement is teaching students how to play with ideas in thoughtful ways and follow up that play in a reasonable, rigorous manner. This is neither a comprehensive nor exclusive way of thinking about college: formal schooling doesn't guarantee this result, and there are plenty of wise people in this world who can play with ideas without having finished secondary school, let alone college. But you're far more likely to get adults who can play with ideas in a productive sense if some critical mass of them have attended formal schooling where that was one of the outcomes.

I think Stanley Fish and gaming-for-learning enthusiasts are some of the more extreme proponents of this view, though they may not like being put in the same bin. At some times eloquently and inarticulately at other times, Fish argues (or just implies, as in yesterday's piece) that playing with ideas is the purest and highest aim of college and university life. That's a good part of the reason why he is allergic to some other conceptions of teaching (such as passionate engagement in the world). Those who have pushed for the insertion of game design in teaching likewise see value in gaming in and of itself, and they have the well-intentioned goal of spreading that joy to students through the use of gaming in teaching.

I do not think the promotion of intellectual play is the sole purpose of higher education, which is why I do not agree with Fish on his save the world on your own time refrain, which would place a wall between classes and any concern with what happens off a campus. Nor do I think that constructing game-like structures inside classes is the only way to promote intellectual play, which is why I have only experimented in a tiny way (and not that well) with game-like structures inside classes. Instead, what a good college (and many a good high school course) provides is the foundation, tools, and time and space for students to play with ideas.

This play needs to be rooted in specifics: some critical mass of specific knowledge in an area, which includes stuff we might call factual information and also knowledge about important questions that have been and continue to be asked in the discipline or field. In most (but not all) colleges and for most (but not all) students in those colleges, that foundation and set of tools require some breadth and some depth. You can't be a great student of history without knowing a sufficient amount about some critical mass of places and time, or without knowing a sufficient amount about some critical mass of other fields that bring other questions to bear on the ideas you're playing with.

And then you need the opportunities and encouragement to play with ideas in important ways. Sometimes these come in structured assignments that look playful, sometimes in serious assignments that engage students in the flow that positive psychologists write about, and sometimes the opportunity comes in extracurricular activities. Again, none of this necessarily requires formal schooling, but the playful autodidact must discipline herself or himself, and a formal school can provide structures to encourage this type of engagement. The institutional nature of a school can often grate on those within its walls, but it can also provide helpful structures. From an historical standpoint, the amazing feature of non-mandatory secondary and postsecondary education is not that one-quarter of teenagers leave high school and two-thirds of young adults do not complete a B.A. but that so many finish when there is no law requiring it. Normative expectations play an important role, and that is as true for shaping behavior within a school as standing outside it pushing students towards school.

Costs

Justifying public subsidies

Okay, some of you must be thinking, I'll follow this argument about the play of ideas as far as formal schooling doesn't cost much. But why should taxpayers subsidize this, and why should someone incur more than $100,000 in debt to learn how to play with ideas? Taxpayers should subsidize surplus education because it's worked for society in the past, which may seem highly unsatisfying but is true with one caveat (below). More pragmatically, the obviously-useful parts of higher education easily justify the subsidy, and what appear to be "frills" are comparatively cheap: try to tell a provost that the English department or history department is a money-waster, and she or he will laugh in your face with good reason: humanities faculty are generally the cheapest dates in any place, in part because of their low salaries and in part because even at the ritziest research universities they don't require several hundred thousand dollars in start-up money each. Doubt me? Go ask your local university the annual maintenance costs per student of a intro-chem lab and an intro-languages lab.

Costs to students: the car rule-of-thumb

Student debt is a different issue. I don't think someone should incur more than $100,000 in debt for an undergraduate education. However, that issue is complicated by stories about new college graduates with mountains of debt that come from enrollment in private schooling, either non-profit colleges and universities or for-profit programs. We need to watch the debt issue, but the streams of student debt origins are concentrated away from public colleges and universities (i.e., not what the solid majority of students face). There are plenty of public colleges and universities where the average debt for graduates carrying debt is under $20,000, and that's a reasonable debt to incur for the part of a college education with likely immediate payoffs in the job market (assuming that there's a job market in the next few years). In addition, the creation of income-based repayment plans is a buffer against college debt peonage if debt begins in the federal loan programs that are captured by income-based repayment. Again, that's easy when you're talking about public colleges and universities. Fortunately, a very large majority of high school seniors and their families are skeptical of mountains of debt, which is why (for example) two of my daughter's closest friends are going to the University of Florida next year rather than Rensselaer, Rutgers, or Georgia Tech (some of the other places one or the other was accepted, where they would have paid out-of-state or private tuition).

(As I've noted, private loans and gigantic debt coming from attendance at private institutions comprise a different matter, in addition to credit card debt. Part of the role of Pell grants, the new GI Bill, and federal loans is to encourage families to take on both subsidized and unsubsidized loans. That may sound remarkably like the type of public-private partnership that's become common in economic development, except that here, families and students incur substantial risk. Private non-profits and for-profits are in the same boat here, receiving a federal subsidy that's often bundled in with additional unsubsidized loans that families and students carry forward, something NYU is struggling to respond to, at least. And all university administrators who approve privacy-invading deals with credit-card companies should rot in Purgatory for a very, very long time.)

There is another way in which student debt is taken out of context: for full-time students and a number of part-time students, a significant part of the cost of college is the opportunity cost of not being in the labor market (or giving up some job opportunities, for part-time students). That can end up in debt if students borrow to pay for living expenses while going to school, and in any case, it reduces income and the accumulation of job experience. For a few years, that's more than balanced by expected greater earnings. The opportunity cost of not gaining job experience becomes a larger issue for someone who is out of the job market for an extended period, as happens with longer graduate programs (such as programs that have an average time-to-degree of nine years for students who finish, and that would be on top of the time spent in an undergraduate program).

A few rules of thumb, to summarize on debt and opportunity costs of attending college: if the direct debt incurred by going to college is on the order of magnitude of an economy or low-priced midsize car, it's justified by the anticipated concrete returns, so the chance to play with ideas isn't a giant financial risk. Don't go into debt on the order of a house note unless the degree leads directly to a lucrative career (e.g., medicine or law, and even there I have some questions). And if you're going to spend more than ten years out of the labor market as part of getting an education, definitely get that economy-car-sized education.

The assessment dilemma

Let me return now to the issue of public subsidies in part for what might look like surplus education. Part of the justification for public subsidy (concerned with value) is taken care of by the parts of college you can identify concretely as human capital, specific bits of skills and knowledge with clear social benefits. Part of the justification for subsidy (concerned with cost) is taken care of by the fact that the more expensive parts of college and university academic programs are concentrated where you see more clearly identified returns (the "humanities are cheap dates" principle). (Athletic programs and student affairs are different subjects.)

That might be enough from the perspective of some faculty (and Stanley Fish and David Brooks, at least this week), but the push for accountability in learning outcomes in higher education can easily be turned into the type of mechanism that squeezes out opportunities and structures for playing with ideas. For the foreseeable future, there will be key actors in several states who would be willing to impose reductive standardized testing on colleges and universities. That is the alternative to the current set of assessment mechanisms embedded in regional accreditation. So let's look at assessment and accreditation with regard to playing with ideas.

The black hole of accreditation-centered assessment

Assessment in the context of regional accreditation is best thought of as meta-assessment, where accreditors hold colleges and universities responsible for having a curriculum and assessing how well students learn it. That putatively gives institutions the freedom to create a structure consistent with a unique mission as long as there is assessment of student learning. In reality, this type of meta-game can be difficult to navigate, and the default behavior leans heavily towards mimesis: many colleges and universities hire consultants familiar with a particular regional accreditor, and they tend to suggest whatever structure has enabled similar institutions to pass muster. In addition, because consultants (or former consultants) are sometimes brought in-house to handle the logistics, they focus on the parts of the process that are most easily managed and cause the least hiccups internally... and that often turns into a small universe of reductive measures available commercially, especially for general-education goals. (Want to assess writing? Let's try the ABCXYZ. Want to assess problem-solving? Let's try the ABCXYZ. Want to assess critical thinking? Let's try the ABCXYZ. Yes, of course we can create our own in-house assessment, but we'd also have to justify its use to our accreditor, and it's just easier to use the ABCXYZ; why don't we at least try that as we're developing our own...) There's a reason why the Voluntary System of Accountability specified one of three cognitive measures: it piggybacked on existing trends in accreditation and institutional inertia.

My general concern is that the mechanisms of assessment through regional accreditation can become the black hole of faculty time, absorbing everything around it and making it difficult to plan a structure for more engaged projects or the type of activity I have described as intellectual play. In addition to what else I could say about that narrow range of measures, the long-term problem with institutional meta-gaming is that the rules of the game can change, sometimes with nasty consequences for faculty time. Every time that an accrediting body changes the rules by which institutions have to set rules for students (i.e., the curriculum), faculty have to rework their lives and often entire programs of studies to accommodate the changes. Every time my state reworks licensing requirements for college-based teacher education, or changes the rules for state review, faculty in my college have their time stolen by the logistics of meeting the rules. (Please don't ask a Florida dean of education to describe the double-standard between the rules for college-based teacher education and alt-cert unless you have a few hours.) One of the consequences is an overburden on both faculty and student time. Let me stop talking about faculty time and focus instead on student time: Look at a few random programs of study for baccalaureate programs in nursing or education. Count the number of elective courses. Compare with a program of studies in any social-science or humanities major. Then pick your jaw up off the floor.

On the one hand, the licensure requirements make a certain amount of sense from the perspective of professional training: you want teachers, social workers, and nurses to have the tools to do the job. On the other hand, an undergraduate education that is devoid of anything but instrumentalist technical courses is job-training and nothing else. And especially for teachers, that is inconsistent with one central purpose of college and dangerous for what we'd like them to do on the job. And the Holmes Group's proposal to shift all teacher training to the masters is unrealistic for working-class students if you apply the car-cost limit to student debt for future teachers. I am not sure there is a good way out of this problem for elementary teacher education, and it is on the extreme end of the "no room for thought" problem we face with accreditation-based assessment.

Outside elementary teacher education, there are a few escapes, but none are palatable. Ignoring assessment requirements of accreditors is either fatally brave or foolish, so what's left? Assessing intellectual play. You can stop groaning now. Yes, attempts to assess "creativity" make you tear your hair out, and the thought of assessing intellectual play makes you want to punch me out for the oxymoron or the threat of one of these projects unmoored from substance and rigor. But from an institutional standpoint for a faculty member in one of those regions with an accreditor that threatens micromanagement, you can either tilt at windmills or see what the power might be used for. I've got a limited appetite for windmill-tilting, and I've got enough blunted spears in my garage for a lifetime, thank you very much. This may sound like squaring the circle or getting out from within the horizon of a black hole, but the ability to assess intellectual play would allow faculty to justify all sorts of projects within an existing accreditation framework.

Defining and assessing a challenge

First, a reminder of Miller's notion of galumphing, or play: pleasurable activity that is deliberately inefficient and encourages the combination of existing skills to accomplish the self-defined or agreed-upon goals over and around the obstacles presented by the constructed inefficiencies. The tricky part of assessing such activity is not focusing on the issue of pleasure but instead on the meta-rules that characterize the nature of the activity. For this purpose, it's best to think about a circumscribed type of intellectual play: a challenge that is at least partially well-defined, based in considerable part on what others have done (i.e., not entirely reinventing the wheel), and that requires putting together at least a few skills. Then the assessment of the student activity has two levels: the level of the meta-game, where you assess how well the student defines the challenge, shows where and how the project relies on other work or is new, and how well the student used multiple skills; and the level of the project itself, where disciplinary conventions come into play...

And for history, at least, the disciplinary conventions match fairly well with the first level: having an appropriate historical topic, using the historiography in a sensible way, and handling a range of evidence and argument structures. The guts of most undergraduate history papers are in that last catch-all category: "handling a range of evidence and argument structures." There are a number of more idiosyncratic and less comparable assessment frames (such as student reflection on engagement), and this short essay is about the larger picture, not a detailed (let alone a tested!) framework for assessing intellectual play. And this sketch is about a narrowly-defined type of challenge, with lots left out. But it's a way to think a bit about the issue... or play with the idea of assessing playing with ideas.

Tools to explore

A few words about some recent developments to watch in this vein. The Lumina Foundation's Tuning project could have begun within a regional accreditation context, but it's geared instead towards a proof of concept that a faculty-driven definition of outcomes and assessments can simultaneously honor disciplinary conventions and also satisfy external constituencies (thus the term "tuning" to get everyone singing in the same key: I've got to ask Cliff Adelman sometime whether it's harmonic or tempered tuning). If I remember correctly, the first discipline-specific reports should have been available on the foundation website sometime this spring, but it's not there now (just a cutesy cartoonish presentation of the idea along with Cliff Adelman's concept paper and other materials from 2009). At a first glance, it looks like an application of the accountability framework of the American Association of Colleges and Universities (i.e., the liberal-arts office in One Dupont Circle). But without sample exemplar projects, it's hard to judge at the moment.

Then there's the movement for undergraduate research. When my daughter and I were visiting colleges over the past few years, it's clear that every institution devoted resources specifically to undergraduate research, whether they were public or private. Then again, these were generally small colleges where undergraduates were the only research assistants that faculty would be getting. On the third hand, undergraduate research is a type of operation that both liberal-arts colleges and universities are trying to develop and promote, albeit with different understandings of student engagement. I think my alma mater (a small liberal-arts college) now requires seniors to engage in a major thesis-like project. At my current university, that's expected only of Honors College students, and the resources of the Undergraduate Research office are available to all in theory and would be totally swamped if every student asked to be involved. Again, neither the development of Tuning and undergraduate research are models in any practical sense of the word, but they're something to watch and, if nothing else, they provide a few rocks on which to stand and survey the landscape of playing with ideas.

June 4, 2010

More on so-called "side deals"

Andy Rotherham has responded to my blog entry early this morning. Let me skip for now the question of why he was the sole person quoted in the article and address the local MOUs in Florida on Race to the Top. Rotherham wrote in part, "If these agreements have no bearing on the state's application or implementation then why go through the laborious exercise of crafting them[?]" I wasn't in the room for any of these, but having observed Florida schools for almost 15 years, I can imagine a number of reasons, including distrust by some party in a county in the judgment of FEA President Andy Ford and other participants in the task force about the clause excluding non-mandatory subjects of bargaining from impasse. I said as much in my prior post. I'm not a labor lawyer, and neither is Andy Rotherham, but I do know something about the dynamics within FEA, where there is often a healthy internal debate. The argument that local MOUs are an inherent evasion of the grant is something that requires examination of the actual language at issue.


Now, to my comment about Rotherham's being used as the sole source for Wednesday's story in the St. Petersburg Times. Why did it seem curious to me? Partly it's a matter of sensitivity to these issues on a number of fronts. For more than a year, Rick Hess has been pointing out the potential for all sorts of perception problems with a competitive process that's the result of (enormous) discretionary authority. In April, Liam Goldrick noted that the New Teacher Project was simultaneously advising several states on RTTT and then commenting on the process (something he thought was unwise from an organizational standpoint).

So when I read a story with a single source commenting on the issue, where the source may have had business interests at stake and where the disclosure of that in the story was vague, and then the term "some say" with only that source as documentation, it looked odd. It was a good journalist quoting someone who is open about disclosure in every direct piece of writing of his I've read. And it wasn't the larger point. But I don't remember anyone correcting the impression earlier in the spring that TNTP had been a disinterested observer. But I could also have thought of a number of reasons why there weren't more people quoted or more disclosure about Wetherbell (Rotherham's firm): maybe Matus got the information late in the day and couldn't reach more than Rotherham; maybe the material was in the submitted story and an editor chopped out additional disclosure; maybe Rotherham didn't have access to the text of the local MOUs or didn't have a copy of the state MOU and was relying on Matus's over-the-phone description of "hey, this looks like it could be different, especially in Hernando." But on the other hand (I think we're on my fourth hand here), the failure to correct the stuff on TNTP is a lapse for education journalism more generally, and it has to stop. So I decided to note what I had observed, call it minor compared with the other issues, and go on. If it looks like I'm being hard on the Times, it's because this local newspaper is one of the top papers in the country on education, and I think I can expect great reporting. But this is a minor error, I meant the observation as such, and I explicitly said so. If I were going to point out that the alleged transparency problem with Florida's application is a distraction until better researched, maybe I need to be consistent and explicitly say my concerns along parallel lines were less important, nu?

My central point was that absent some more solid analysis of what the local MOUs actually meant and whether they conflicted with the application packet, the larger issues were not about procedure but the sustainability of whatever happened with RTTT, assuming it was beneficial. Here's what I wrote this morning:

It's a legitimate question to ask what the right balance is in collective bargaining on the scope of bargaining, on the relative power of the parties, and on state law that can essentially dictate terms and conditions of employment outside bargaining.... It's also a legitimate question to ask about the commitment of parties to reform after the money runs out.... Those issues are still out there, and they're out there whether or not a particular state has an MOU like Florida's.

And here's what Andy Rotherham wrote:

[T]here are two big outstanding questions on RTT that we won't know the answers to for several years: First, how durable will the policy changes be?  Will states relax things when the money is gone and/or will "loser" states undo the reforms they put in place in an effort to win? Prize theory is built on the idea that the progress generated in an effort to win is built upon.  That idea has not been fully tested yet in the public/political sphere.

Surprise: we agree on the importance of that question! No, it's not really a surprise. It's just that a lot of electrons have lost their lives this week in what thus far looks like a nonstory.

Correcting the facts on so-called RTTT "side deals"

In Wednesday's paper, St. Petersburg Times reporter Ron Matus relied on the sloppy language "some say" to spin a mountain out of a molehill about county-specific MOUs between school boards and local FEA affiliates. In the article as well as two blog entries Wednesday and Thursday, Matus stated that there were a number of counties with local memoranda of understanding (or MOUs) and quoted one individual who said that the existence of what Matus called "side deals" might be a problem for Florida's application. Matus stated that his source Andy Rotherham had helped other states with RTTT applications, but the article did not state whether that was in the context of consultancy contracts (i.e., whether Rotherham's new organization had its reputation and business at stake in competition with Florida's RTTT application).

The omission of any mention of Rotherham's business concern is minor compared with the failure of the Times to look at the content of the side deals and see whether they modified the obligations of local parties vis-a-vis the state Memorandum of Understanding that most districts and unions signed across Florida. Since Ed Week has gotten into the story, albeit without quoting Rotherham, it's important to look at the facts.


First, the issue of impasse. Language from the state MOU (part of the RTTT application):

Only the elements of this MOU which are contained in existing law are subject to the provisions of section 447.403, Florida Statutes. (p. 3)

Explanation: F.S. 447.403 is the part of Florida's public-employee collective-bargaining law that covers impasse. In other words, if there's a part of the MOU that is not already a term and condition of employment under Florida law, it's not susceptible to the impasse procedure. That point is clarified in the attachment the Times education blog noted was part of many counties' documentation.

The Broward side agreement in its entirety:

Any items relating to the RTTT Application or Plan that are unsuccessfully negotiated between the parties specifically for the purpose of applying for or receiving the RTTT grant award will not be subject to the impasse procedures set forth in Chapter 447.

Can someone explain to me why this is any different from the state MOU? But there's a second issue that Ed Week's Michele McNeil discussed: "these side deals also say that any changes successfully negotiated because of Race to the Top will expire once the funding does," and refers to the Hernando County MOU. But on p. 4 of the state MOU, Part IV explicitly states that the state MOU expires "upon the expiration of the grant project period, or upon mutual agreement of the parties, whichever occurs first."

The question one might logically ask is why some counties and locals felt they needed extra language. The FEA had a long weekend discussion with state leaders and local leaders about the state RTTT application this time around, and from news coverage it looks like FEA President Andy Ford was strongly encouraging locals to sign on. The reason why was pretty clear: he had had a seat at the table in the task force Charlie Crist set up in the week after Crist vetoed SB 6. When you've had a hand in crafting changes, you've got a stake in success. In addition, the additional language had taken care of one of the legal concerns of FEA bargaining-support staff, because the MOU from the first application in Florida looked like it might give school boards the ability to impose contracts on matters beyond what is currently in state bargaining law. Unlike in many northern states, Florida school boards have the authority to impose contract terms under impasse for a the duration of a fiscal year, but only on mandatory terms and conditions of employment as defined in Florida law.

In January, FEA had cautioned locals not to sign the MOU, and it crafted language for the few locals who wanted to sign (including one large county, Hillsborough). The language FEA crafted for the locals in the first RTTT round? It specifically exempted issues from impasse when the issues were not already in state law. (I don't have the exact wording in front of me, but I am sure an intrepid reporter could ask the Hillsborough press rep for it.) In that case, it's clear that the local MOUs created legal conditions different from what would have been the case with a signed state MOU and no local MOU. So when similar language appeared in the state language, why did some local teachers unions sign essentially redundant local MOUs? Let's just say a generous level of suspicion about the process.

The greatest problem with this coverage of the county-specific MOUs is that it's a distraction from serious issues of reform implementation with RTTT. The issues Matus and McNeil have raised in the context of local MOUs exist with the state MOU. But instead of focusing on the substance, the reporters are focusing on the process issue. It's a legitimate question to ask what the right balance is in collective bargaining on the scope of bargaining, on the relative power of the parties, and on state law that can essentially dictate terms and conditions of employment outside bargaining. In a state like New York, bargaining authority leans more towards unions than in Florida, and likewise state law. Northern states are the ones to have seniority-preference laws that trump the bargaining process, and Florida has had several statutes trying to mandate all sorts of things unions would be very unlikely to agree to in local collective bargaining.

It's also a legitimate question to ask about the commitment of parties to reform after the money runs out. That is one of the critical questions with the DC teachers contract: what happens if/when the billionaires pull out? The billionaires' support of DC along with RTTT present a theory of action all about inertia: if we can just budge districts away from current practices, we'll accomplish long-term structural changes. In contrast, Denver's ProComp was in the context of a permanent funding stream and a political deal with voters: give us the money permanently, and there is a permanent change in compensation practices.

Those issues are still out there, and they're out there whether or not a particular state has an MOU like Florida's. I understand why this reporting on process exists, especially in a rush to print news, but am disappointed that two good reporters have perseverated on an apparent process issue without checking the details of their assumptions (i.e., whether the local so-called "side deals" are substantively different from the state MOU).

Disclosure: I am a former member of the FEA governing board. (I have not corresponded with elected FEA leaders about the reporting on this story, but I want to be open about my former position within my state affiliate.)

Update (9 am EDT): After I posted this earlier in the morning, Valerie Strauss published an entry on the topic in the Washington Post blog she writes, largely repeating what the Times had said. I also corresponded in the last hour with one reporter interested in the story, and one of the empirical questions is whether the local MOUs in Florida are more like Broward (short and redundant) or more like Hernando (which was much longer and with elements McNeil discussed in her blog entry yesterday afternoon). There's also a broader question about state administrative authority. Suppose a superintendent of a district in any state receiving RTTT funds decides she or he isn't going to follow one of the requirements. She or he just didn't put it in writing. Does the state's obligation to eliminate participation and cut off funding for that district change? As I said earlier this morning, the broader and more interesting questions are not really about local MOUs.

June 2, 2010

Quips on Race to the Top and Common Standards

From late morning until a few minutes ago, I've spent the day under a cloud of variously-intensive headache pain. I'm free of it but also exhausted, so in the zeitgeist of evidence-free punditry about RTTT and Common Standards, here are some thoughts:

  • If my memory and reading are correct, none of the national press outside Florida noted the collaboration involving the Florida Education Association in the state's RTTT2 application until... there was a chance to criticize the FEA. Have national ed reporters been captured this spring by a particular narrative frame on unions? For Lesli Maxwell and others: if you describe everything unions do vis-a-vis RTTT as explicitly or implicitly obstructionist, how can you can distinguish the good actors from the bad? (Disclosure: I'm a former member of the FEA governance board and a VP of a higher-ed local affiliated with FEA.)
  • There is a point to criticism by Rick Hess and others that RTTT required state buy-in to the Common Standards in math and reading one day before they were finalized, effectively creating a cheap zipper clause in the application. At least it's not requiring states to buy in to Texas-approved textbooks. Texas's Board of Education would not have liked the idea of Lincoln's second inaugural as one of the English/Language Arts required texts without balancing Lincoln with Jefferson Davis.
  • And speaking of copouts, at least give credit to Virginia Governor Robert McDonnell for harkening to Virginia's own curriculum standards as an excuse for not putting in a RTTT2 application. He can think on his feet faster than some other governors I know, or his own attorney general. At least until a sharp reporter asks the governor to repeat a single line in the Standards of Learning he's so proud of.
  • Notice that since Sarah Palin's resignation as governor, Alaska has returned to its status as one of the states you put in a map inset or use to miniaturize Texas? Texas and Alaska are the only two states that outright refused to participate in the Common Standards project. They also have mutually exclusive cuisine: you can't get moose or wolves in Texas, and you can't find armadillos or liberals in Alaska.

After hearing some of these, a family member said, "Between that and your sinus headache, please don't breathe on me." More serious thoughts as soon as I can find them.

Update: mea culpa on misidentifying the Ed Week reporter who wrote about Florida yesterday. The attribution should have been and now is Lesli Maxwell.

June 1, 2010

The value of college II

An offhand reference I made last week to Lisa Delpit is nagging at me this evening. It's the part of Other People's Children (1995) where she talks about the existence of codes of power (what others would call tacit knowledge) and how one of the jobs of good schools should be to lay those bare, damn the accusation of selling out to an instrumental view of schooling. Her argument is that middle-class parents and educators too often talk in a Romantic discourse about schooling, ignoring how advantaged parents teach a great deal about the codes of power explicitly and how unfair it is if you hide some of the secrets of power from poor children. When I began teaching at USF, Delpit's book had been published recently, and I used it for several years. It never failed to stimulate healthy debate, especially since the majority of my undergraduate students are usually of the temperament and philosophy Delpit was trying to discomfit.


While her argument was more about primary and secondary education, a great deal of it could apply to college, yeah, even to junior faculty. Earlier in the spring, SUNY Buffalo sociologist Lois Weis visited USF, thanks to the Alliance for Applied Research in Education and Anthropology (Kathy Borman's group in the anthropology department here), and one of her talks briefly referred to Delpit as a jumping-off point to a realistic discussion of what research-heavy universities are looking for in faculty. You think I was unrealistic in urging assistant professors to wait until they're tenured before sinking a lot of time into experimental forms of scholarship? Go listen to Weis; I saw at least one colleague looking to apply for promotion to full absorb every word, and I thought that was wise. Weis's talk was unabashedly instrumentalist: if there's a game to be played in academe, let's not pretend it doesn't exist, and let's make sure that the people we care about can play the game with a full understanding of the rules.

Beneath these arguments is a realistic assessment of how schools combine instrumentalism and the potential for change. Delpit doesn't worry too much that children of color will sell out; let's give them the skills to succeed, and while some may want to sell out, we'll probably learn a great deal about how many won't. Weis didn't talk about that much in the hour-long presentation, but given the type of work she does, I don't think she's on the side of getting a bunch of sociology grad students to join Wall Street. Being successful as academics mean they can make arguments for a better society in general.

One of my friends and longtime colleagues talks about the time John Hope Franklin visited USF many years ago and when asked about radical change in society, Franklin reportedly said, "Go to the library!" What he meant, or what my friend drew from what he meant, was that the textbooks reach the next generation, but to be in the textbooks, you've got to publish research that's read and influences those who write textbooks. And to publish research, you've got to go to the library. It's a conventional view of academic research coming from one of the great African American intellectuals of the 20th century, someone who grew up in Oklahoma, went to college in Nashville in the 1930s, was denied opportunities in WW2 because of race, helped Thurgood Marshall prepare cases in the late 1940s and early 1950s, and stood by his association with W.E.B. Du Bois in the middle of anti-Communist hysteria as he was ascending the academic ladder. Of course, you might say that it's easy to take that view if you're John Hope Franklin, but I suspect it was not easy to be John Hope Franklin, at least not before the 1970s.

The point of all this is that schools simultaneously serve as a vehicle for hoarding privilege and also for breaking it down. The first part is going to exist not because schools exist but because those who currently have privilege are going to use whatever institutions exist to maintain that privilege. So Romantic notions aside, you don't get a choice in that fact, in any society with formal schooling. The choice is whether we take the tools that currently exist and make those tools available to people broadly. When I first saw a link to the May 16 New York Times article on Vedder's and Murray's anti-access view on college, my thought was that Vedder and Murray were arguing that poor families should give up half the tools at their disposal for improving their lives. Are college degrees sometimes used as credentials without reference to what graduates learn? Sure, but you don't eliminate the use of credentials by refusing to gain one. Are college programs sometimes light on substance or disconnected from the job you might get within two or three years? Sure, but you get to keep what you learn for the rest of your life, not just the job you get in the next few years.

And is formal schooling sometimes mind-numbing, discouraging, depressing, oppressive, disillusioning, lock-sync, and whatever other term you want to call lit? Sure, and that's a consequence of a structured curriculum that also provides millions of children with access to the life of the mind. If you've got the resources and the background to teach your children at home.... hmmn, where might you have gotten it? ... sure, you can be a successful homeschooling parent. Of course, if you're a homeschooling parent, you might well use a prepackaged curriculum that makes your kid's education fairly close to the structured system that you just called mind-numbing, discouraging, depressing, ... well, you get it. There are many, many ways in which formal schooling can improve, and there many ways in which schools carry a political burden that is unreasonable. But that's no reason to avoid or fail to use the instrumental value of schooling as formal schooling. First let's graduate the next John Hope Franklin, and Franklin's readers, and we can also worry about the tortured, contradictory nature of higher education.

May 25, 2010

"...and thereby to secure a more arbitrary and unlimited authority"

Yesterday afternoon (at least afternoon in California, where the radio station operates), Sara Goldrick-Rab and Richard Vedder debated who should attend college on KPCC's Patt Morrison Show. I am disappointed but not too surprised that Vedder skipped over who he generally thinks are the types of people who don't benefit from college: other people's children. (Amy Slaton made a similar point in this morning's IHE column: "These two assertions [of the not-everyone-should-aspire-to-college crowd], the first based on very selective logic and the second baldly elitist, become particularly nasty in tandem, making the college aspirations of minority or poorer Americans seem positively uppity.") Let me step away for a day from the question of who should attend college today and see how that logic would have been applied in the past--discouraging formal schooling for those who would not necessarily finish a certain level and for whom there wasn't an economic payoff.


To put it bluntly, that logic would have prevented the coeducation of primary schooling in the nineteenth-century North. As David Tyack and Elisabeth Hansot explain in Learning Together, the first half of the nineteenth century witnessed a quiet revolution in formal education, from schooling being the domain of boys and men to coeducation in the first few years of schooling (which was generally what was available for most children in the North). There had been some colonial examples historians can identify of coeducation and women teachers outside dame schools, but they're the clear minority of experiences. When Benjamin Rush helped John Poor obtain a state charter for the Ladies' Academy of Philadelphia in the early 1790s, he was using his influence to break down existing barriers. Five years before, he had spoken at the new school, and the written report of his remarks starts with a justification for the school that (at least to me) looks to men as the audience, and only at the end does he start to speak to the students in the audience:

To you, therefore, YOUNG LADIES, an important problem is committed for solution; and that is, whether our present plan of education be a wise one and whether it be calculated to prepare you for the duties of social and domestic life.... I have sometimes been led to ascribe the invention of ridiculous and expensive fashions in female dress entirely to the gentlemen in order to divert the ladies from improving their mindsa nd thereby to secure a more arbitrary and unlimited authority over them. It will be in your power, LADIES, to correct the mistakes and practice of our sex upon these subjects by demonstrating that the female temper can only be governed by reason and that the cultivation of reason in women is alike friendly to the order of nature and to private as well as public happiness. (pp. 91-92)

To us more than 220 years later, this quaint and charming language obviously lacks the fire of Tom Paine and the righteousness of Mary Wollstonecraft, but for all its gentility it is an affirmation of common humanity and educability that Rush and his audience knew could not be taken for granted, even in Quaker-influenced Philadelphia. Tyack and Hansot struggle somewhat with the question of how coeducation could happen without significant public debate, and I struggle with it as well: how much to ascribe to the coeducational experience of dame schools, to early-national ideologies of Republican motherhood, to a practical "I want the girls out of my hair, too" attitude of rural Americans (who often sent children as young as two and three to tag along with older siblings), to the Second Great Awakening, or to the fact that rural apprenticeship was a system of sharing childrearing that included girls as well as boys (if the girls were often distributed to neighbors' houses to help with domestic responsibilities).

Whatever the causes, there are two undeniable facts about the coeducation of primary education in northern states: the expense was not easily justified by the legal or economic role of women at the time, and it had enormous benefits for the entire society for generations to come. I am sure Vedder and others would contest the first claim, but there are plenty of agrarian societies where the majority of work is or has been done by women who have little or no formal schooling. Why do you need to read and write if you're in the fields all day? Just go to a taro-harvester certificate program for a few weeks and get a job! Oh, wait: no community college currently offers a taro-harvester certificate.

More seriously, one could imagine a different history, closer to the history of the South, where coeducation happened much later, incompletely, or not at all. Primary education was an expense for communities, and coeducation was an added expense either for the community or for the parents who paid extra tuition (or private payments to schoolmasters on the side). We know that formal education was a considerable expense in part because even in Massachusetts, communities resisted the creation of high schools until late in the 19th century. The 1860 town vote of Beverly, Massachusetts, to abolish the high school was notable because it was a clear violation of state law (Beverly was one of the towns sued by the legislature earlier because they didn't have a high school) as well as because its public recording of individual votes has bee the subject of two books. Only a relative handful of students could continue to high school, and the majority of voters at that town meeting clearly thought the benefits of high school did not justify the expense. Yet by 1860, most towns in the north had coeducational primary schools, and thousands of parents had been willing to pay extra money (and had been willing to pay it for decades) to get their daughters some education, though the daughters would never be able to repay them in any concrete sense.

Yet despite the lack of immediate calculable returns, the coeducation of primary schooling in the North was one of the smartest social policies for the long term. The education of girls doubled the pool of potential teachers one generation later. Combined with lower fertility over the 19th century, the increased pool of potential teachers dramatically shifted the ratio of children to potential teachers in favor of children and education. Apart from arguments I could make about lower fertility's being a consequence of coeducation, the combination effectively provided a bootstrap for American mass education, making it easier for states to expand formal schooling generation by generation. Some parts of that bootstrap were not what we'd choose today, since it partly depended on restricted employment opportunities for educated women generally and educated men who were not white. But it would not have existed without primary education for girls and without the willingness of parents and communities to spend money that they could have easily not spent.

Part of the case against expanded educational opportunities is a show me what it'll do today argument. That's a narrow reading of the potential of people who don't currently attend college, a narrow reading of the purpose of education, and a narrow reading of the consequences of education. Yes, I think a lot more children from poor families can succeed in college than do currently. Yes, I want the people who pick up my garbage to read Shakespeare and pick out the lying statistician on a witness stand. And, yes, I am confident that there will be positive consequences for expanding college opportunities far into the future, consequences we cannot imagine today and that will dwarf the real costs of expanding those opportunities in the institutions where they will exist.

May 20, 2010

Texas and reality

Despite what I promised a few hours ago, this entry is not about coeducation, but current events in Texas are pushing my thoughts away from the value of college, at least for now. The rolling disaster that is the lame-duck-infested Texas Board of Education is both agonizing and fascinating, or one step above the formerly-creationist Kansas Board of Education (when the majority was in favor of teaching creation myth as part of science). Reading and listening to the more conservative board members leads me to conclude tentatively that while they will not say so explicitly, they really would like a curriculum that is based on a providential understanding of historical cause: America (and Texas) is blessed, and history shows how God has favored us, especially when we have been Godly.

That desire for providential history in public schools is wrong for two reasons. First, public schools in the United States should not be teaching religion as truth. (Teaching about religious beliefs and organizations as an important part of history is different. Teaching about religious beliefs as part of the cultural background for literature, myth, etc. is likewise different and perfectly acceptable.) The majority of the Texas board obviously disagrees with my interpretation of the First Amendment, but there's a second reason to avoid providential explanations of history: it is incompatible with the type of historical argumentation that is professionally acceptable to historians.

There are all sorts of historical explanations, metanarrative structures, and assumptions about human nature that professional historians would find plausible or at least acceptable to discuss as part of historical writings. But history as practiced today is about human nature and observable events, not providential explanations. That's as true of historians who have deeply-held religious views as it is for nonbelievers who write history. We just don't write deus ex machina history.

I know: we've been down this road before with debates over creationism and its close cousins: evolutionary biology is not a religion, and neither is standard history. But there's something that we can learn by thinking about history rather than science: the type of incommensurable perspectives that exist in the evolution/creationism divide is not there just because we're talking about fossils rather than human beings. That's close to the type of distinction that some refer to as mind-independent vs. mind-dependent phenomena. And I understand the appeal of that distinction.

But I have a different way of looking at the detritus of poststructuralism, and perhaps it's because I knew in writing Creating the Dropout that the bit about the construction of dropping out was sloppy in terms of handling the idea of social construction. I was focused on writing the story as detailed as was appropriate in an historical sense: when did "dropout" become the dominant term for adults who didn't have a high school diploma, what was the description that became associated with that, what did the choices at the time foreclose, etc. But as an historian who is generally more focused on the details than the meta-meta-level assumptions, I didn't do much more in talking about the construction of social problems than wave at Hilgartner and Bosk and go about my work. Did I mean that the stereotype associated with the term dropout was one of those paralinguistic structures that foreclosed alternatives, or that would spread and become an overturned irony over time? Was it part of a growing hegemony about the value of education? I apologize to anyone who was disappointed, but I was not up to the meta-para-hypertheoretical work that might have been involved. And no one really called me on that gap: reviewers generally acknowledged the story in the first few chapters and poked holes (some real and some virtual) in other pieces.

That doesn't mean that I am unread in relevant literature. I took my first-year proseminar with Lynn Hunt, and she walked us through Foucault, White, and a number of others who fall in the poststructuralist/deconstructionist canon (irony intended). But the question of whether language in the abstract performs the type of cultural work that some attribute to it paled in comparison with what people actually said about high school attrition in the 1950s, 1960s, and since. Given what Lynn's written since that year in criticizing the extreme forms of historiographical deconstruction, I think I may have made the right choice in how to spend my time, at least when it came to my first major research project.

But there is a larger question here of how to handle the fuzzy and malleable categorizations of (what we think of as) reality. Do we make choices about how to frame reality? Yes, of course, but in a relatively mundane sense of having to make some choice in how we investigate or describe the world. We can't avoid that choice, and for the moment I'll be agnostic on whether investigation is with scientific instruments or textual analysis (or something else), or whether communication is with language, mathematical symbols, or whatnot. Once the choice is made, that creates some structure about how we view reality, and it imposes at least a minimal cost on looking at things in a different way.

At this mundane forced-choice level, I'm essentially arguing that intellectual work is like the policy options for a country choosing whether you drive on the right or left side of the road. If you want most people to get anywhere on the road quickly and safely, you have to make a choice. We can debate whether the choice is political, economic, rational, irrational, etc., but a choice has to be made to get both quickly and safely, and there are consequences that flow from the choice, including signage, standard car equipment, and so forth. Note that this analogy doesn't touch issues such as correspondence with any underlying reality: It would be silly to claim that the choice of left or right has correspondence to Reality or Truth.

Instead, let me focus on the question of whether the choice at one time for the convention of driving on the left forecloses changing the convention, and what's required for such a change. At one level, the choice is mutually exclusive: a country cannot pick both rules and expect anything other than carnage when people drive faster than 5 mph. But at another level, the choice is resource-dependent: it's possible for England to change its rule so everyone drives on the right. It just would be a royal pain in the tuches.

So you can measure the rigidity of a convention in one sense by asking how expensive it would be to change it. Changing the side of a road for driving is expensive but possible. But you could imagine setting a rule that is impossible to change in the defined context. Unless you are driving on the Autobahn, most jurisdictions limit your speed to under the escape velocity of the planet. I don't think we could reverse that and require people to drive on the surface of this planet at greater than 7 miles per second.

Let's move away from driving conventions and back to how we talk about the universe. In both physical sciences and humanities, there are ways of classifying our fields that are nonexclusive and can be mixed; there are categories and ways of describing objects of interest that are exclusive but that can be switched from one to the other with some cost (i.e., exclusive but resource-dependent intellectual choices); and there are some choices that cannot be changed within that context (i.e., exclusive choices that you can't undo in the context you've created). Race, class, gender, disability, national origin, politics, language, etc., are all classification systems that can be mixed in the same context. No big deal there: we may choose to define categories of interest in different ways, but even if you call your categorization by the term class, and I call my different categorization class, we can just say they're different notions of class (or, as Ira Katznelson says in City Trenches, different layers of class). For the mathematically or notationally inclined, we could even index them as Class1, Class2, Class3, etc.

As I wrote at the top of this entry, I think there are exclusive choices that you can't undo in a specific context. If you're an academic historian, your arguments are going to eschew providential explanations of events. You can't undo that and still be in the field of history as I understand it. Regardless of whether the surface disagreements between me and some Texas education board members appear to be political or pedagogical or something else, I think the deep difference is that a number of them truly think public schooling should be teaching providential history or the "intelligent history design" equivalent (i.e., papered over). Again, that does not mean that historians or history teachers have to be agnostic or atheist, just that what they write or teach as historians isn't providential. (My high school history teacher Mr. Knowlton was one such person, a conservative evangelical who taught American history using primary sources and definitely non-providential arguments, though I know from conversations with him outside class that he clearly had providential beliefs outside his professional role.)

What I haven't talked about are examples in history (or other disciplines) of the exclusive but resource-dependent ways of categorizing reality. I'd be tempted to draw from physics (designing experiments to observe electrons as either particles or waves, but not both at the same time), but that's cheap. I will admit that it is late, I am tired, this entry is long as is, and maybe leaving this open-ended will draw interesting comments or enough suspense to keep you reading my blog. But please chime in on comments: am I all wet, on track, and can I be both at the same time without the universe exploding?

Update: The prayer at the start of today's meeting confirms my tentative conclusions about at least the member saying the prayer.

The value of college I

Over the past week, I had been collecting a number of references to recent online discussions of the value of education when the New York Times column by Jacques Steinberg highlighting the views of Richard Vedder and Charles Murray appeared. Claus von Zastrow (among others) has already pointed out that given the fact that the advocates of the "you don't need college" position are highly educated, this reads as an argument that other people's children shouldn't go to college. I sometimes have a bit of fun when Bill Gates talks about the importance of college--"do as I say, not as I do"--but Gates errs on the sides of generosity in terms of what he'd like others to accomplish. Not so Vedder or Murray.

I'm going back over Goldin and Katz's The Race between Education and Technology with a finer-toothed comb than when it first came out in 2008, and I'll probably write a number of posts on this topic. Generally, the literature on the value of higher education (or formal schooling more broadly) is not particularly nuanced. It's human capital and a boost to income! No, it's a queueing process! No, it's a confirmation of inherited intelligence! It's a floor wax! It's a dessert topping!

I'll start with an historical perspective, a warning about loose generalizations: let's stop talking about "higher education" in the abstract, as if it's the output of a utility. Colleges and universities are specific institutions, and the value that students receive from them are dependent on context. In the nineteenth century, a number of states and some cities created normal schools, or schools designed to train teachers. But as Chris Ogren and others have pointed out, in addition to the teacher-training function, public normal schools were often the nearest place where anyone could get something beyond rudimentary schooling, so they inevitably became general schools. When normal schools became teachers colleges, you saw the same phenomenon; Lyndon Johnson attended a teachers college because that was where he could go to college, period. To see the history of normal schools and teachers colleges entirely through the lens of teacher education would be historically inaccurate and narrow.

Or, to take another example, the history of vocational education is not just about narrow trade schooling and the denial of educational opportunity through tracking. That's a large part of it (e.g., ), but again, context is everything. Ira Katznelson and Margaret Weir point out the disputes between labor and business in the structure of vocational education in Chicago. And both Kathryn Neckerman and Bill Graebner have pointed out that in many northern cities, vocational-technical high schools were one cut above comprehensive high schools, sufficiently so that working-class whites fought to keep African American students out of them. In my archival research for my dissertation and first book, I saw something similar in the Atlanta-area vo-tech school in the early 1960s, where administrators fought to prevent it from being what they perceived might be the dumping ground for area schools. Again, institutional context is important.

The reality is that higher education serves several "functions." Some of that could be considered human capital, but the only way to call all of the value human capital is to make the term meaningless. And plenty of higher (and other) education also helps advantaged families hoard those advantages, but it's far from a hermetic process and far less tilted towards the wealthy than plenty of other areas of life (housing, the labor market, tax codes, health-care access, etc.). And all of this type of analysis is predicated on the ability to identify predictable consequences of education. But that's not true, and the prime example is the coeducation of primary education in 19th century America. That's the topic for the next entry on the value of education.

May 14, 2010

The Thursday's-child research design

This morning's St. Petersburg Times is reporting on a working paper on the Florida class size initiative by Harvard researcher Matthew Chingos. This paper is now available on the PEPG website, and the Times asked me to read it and comment, and there are a few things I can say:

  • It's stronger technically than a previous paper by Chingos and West that I commented on recently. As a former editor, I'd guess it's one thorough scrub away from being publishable at a reasonably selective journal.
  • It's based on a clever concept, comparing districts and schools that had "far to go" on class size with districts and schools that were very close to the starting mandates and framing the comparison as one of "if you give districts and schools more money, do you get more bang from your buck by mandating that they use that money to lower class size?" That's not the question on the ballot this fall, but it's a reasonable slice given the difficulties of disentangling policy effects. And if you know the "Monday's Child" rhyme, you'll understand why I'm calling this a Thursday's-child research design.
  • The paper focuses on grades 4-8, which is an important contribution to the literature because of the focus of much class-size research on primary grades. But that focus comes because grades 4-8 was the policy interval with the most variation in starting points among districts, and so the paper can't tell us about class-size effects at other grades.
  • Chingos addressed a number of potential weaknesses in reasonable fashion. There are two weaknesses remaining that struck me on first reading the paper -- the failure to address student migration (both interstate and intrastate) and the failure to consider "contamination" of the design by student experiences with primary-grade class-size reduction. The first can be addressed by some alternate samples, or even identification of students who moved from "control" districts to Thursday's-child districts. But the second is tough to address: while the majority of districts didn't have far to go on grades 4-8 starting in 2004, most of the "control" (or not-far-to-go) districts for grades 4-8 were still "far to go" districts as far as primary grades were concerned, and for intermediate-grade students in the third and later years, their test scores would have reflected experiences with class-size reduction in primary grades.
  • There are some interesting variations in tables and effects by specific grade.

As I told reporter Ron Matus yesterday, I'm sure this will be spun out of all proportion to the study because there's little solid research on Florida's class-size mandate, the stakes with the ballot measure to change the mandate are high, and the proponents of changing the mandate are evidently starting well behind the eight-ball with public opinion. So the temptation to exaggerate the findings is high. Jeff Henig would not be surprised.. and speaking of which, congrats to Henig for the AERA award for Spin Cycle! It's highly deserved.

May 2, 2010

The theater of basing a majority of evaluation on test scores

Now that SB 6 is dead, that a governor's task force on RTTT came to a compromise in a single day, and it looks like there is some direction for teacher evaluation in Florida that's acceptable to Florida's K-12 teachers unions, it's time to take stock of the rhetorical stance SB 6 supporters had that a "majority" of a teacher's evaluation had to depend on student test scores. I've seen this pop up in other states, so it's a common rhetorical stance. Let's get a few things off the table first: this is not based on any research, and the supporters have no clearer idea of what "majority of a teacher's evaluation" might mean than supporters of the "65% solution" had any clue what spending money in a classroom meant. For that matter, neither did I as a skeptic (about either proposal).

So the "majority on test scores" stance is political, then. That's fine as a minimal statement; almost all decisions about pay structures are political in a broad sense rather than based on research, and to some extent they're reactive. Teacher pay scales became standardized to protect bureaucratic structures from (and sometimes in response to) accusations of corruption, and the single salary schedule is a response historically to gross pay inequity.

I'll go further: I don't think there's a way to avoid political values embedded in pay structures. Once you involve public money and a service most people connect with citizenship (education), you've got politics, however well structured and justified by reference to neutral statements of organizational need. On that level, performance pay is justifiable from the sense of satisfying public perceptions about how teachers should be paid. That was explicit in Denver's ProComp plan: the voters approved higher taxes in return for a performance-pay structure.

The problem with the "majority based on test score" position is twofold. One is the obvious one: it's divisive, and many parents and other community members are offended by the idea. Here, Diane Ravitch spoke for millions when she criticized SB 6. But there's another problem: it obscures the evaluation process rather than clarifying it. By reference to an implied point-based system, it fails to focus on what matters in a teacher evaluation system in terms of either an algorithm or underlying concepts.

I've written a bit about point-based systems, and because the focus of my paper was elsewhere, I didn't have a chance to talk about the limit of point-based scoring systems: it matters not where you can earn points but where you might lose points. I learned this in high school when I was a debater: individual raters have an implied comfortable range for scores, and it's the range of scores that matters, not the total number of points available in different categories. If raters have different effective ceilings as well as ranges (i.e., it is impossible for people to earn perfect scores with some raters, while others commonly hand out full marks), then the raters with the largest ranges of scores exert more power over final results than raters who have a very narrow range.

Similarly, components of any point-based system will have differential impact on final results when they have broader ranges in practice regardless of the proportion of the scale that derives from individual components. Imagine a teacher evaluation system with 100 points. Suppose 60 points comes from student test scores, and the range is restricted for most teachers to between 52 and 60 points. On the other hand, suppose 30 points in this hypothetical evaluation comes from direct observation, and the range of scores is between 10 and 30 (and more than a handful of teachers may earn the low score). Which component has the greatest influence on final results? It's the 30-point direct-observation component in this thought experiment, because in this hypothetical example teachers can lose more than twice the number of points there than through student test scores.

But the "majority of evaluation" rhetoric does more than obscure the real power in point-based systems: it obscures the question of what teachers are responsible for. "Outcomes!" says the supporter. Right, I say: that doesn't say a darned thing about the types of outcomes that will make the difference in evaluation. In Florida, Louisiana, and other states where people have pushed a majority from test scores approach, the push has been to create a mandate and defer the implementation to a regulatory process. That's a nice illusionist's trick if you can get away with it, but the process of implementation always mediates absolutist mandates, and then the legislature is giving up what mediates the test scores.

There are three ways I can see that test scores' impact on evaluations would be mediated in any system (and yes, I'm including SB 6 here): ad hoc (i.e., caprice), by reference to student disadvantage (i.e., blame-shifting), or by reference to teacher behaviors in classrooms (i.e., standards of practice). Without any legislative guidance, ad hoc and capricious mediation is likely (probably by the temperament and philosophy of the administrator with the greatest authority over evaluation). More destructive than ad hoc mediation would be blame-shifting: a teacher would be held blameless if someone else/something else (poverty, language, presumed parental neglect, etc.) can be blamed instead. Bad, bad idea.

Of the three options that come to mind tonight, mediating test scores by professional standards of practice seems the most productive. But then that raises the central question: if the use of test scores is inevitably subject to mediation, and the best choice for that mediation is through professional standards of practice, why not base evaluation on professional standards of practice to begin with--for example, to let an evaluation that documents effective practice create the rebuttal presumption of effectiveness?

The answer here is two-fold: one is that there is no agreed-upon standards of practice for teaching more generally, other than by crude and obvious standards (don't beat your students) or by reference to effects (keep your students' attention). The other explanation is that even if there were agreed-upon standards of practice, the process would be sufficiently messy as to irritate the sensibilities of those who advocate the putatively cleaner "majority from test score" approach.

The result is that instead of getting a messy but constructive system based on developing standards of practice, any such system that putatively bases the majority of a teacher's evaluation on test score is going to get ad hoc or blame-shifting mediation through the back door.

Update: Linda Perlstein noticed the 50% rhetoric and should get credit for the pattern recognition. Consultants' advice? Hmmn... looks like an interlocking-directorate phenomenon (no conspiracy needed).

April 25, 2010

Veritas? Caveat!

There's a new paper by Harvard researchers Matthew Chingos and Martin West on what ex-Florida teachers make, and relationships between post-teaching income and during-teaching value-added measures that I'm sure some will tout as proof that SB 6 and any like performance-pay plans are desperately needed. Err, no. Fortunately, Chingos and West do not make that argument, but they also don't tell the reader how many student scores were excluded from their rules that would tend to eliminate special education service recipients, nor how they justified combining value-added measures across multiple grade levels... nor why they used a linear measure of age in both the labor-participation propensity measure and income when the labor-participation (and thus income) effects of age for reproductive-aged women are not going to be linear.

Then there's R2 for the key non-public-school labor market equation: .06 (see the last column in Tables 4-6 on pp. 39-41 of the MS). This is an underwhelming amount of variance the models explain.

Unfortunately, the breathless reporting of this study by Joanne Jacobs does not pay attention to these details.

April 22, 2010

Dorn reviews Ravitch

My review of Diane Ravitch's new book is now up at the Education Review website. I should have finished it a few weeks ago, but the fragmentation of my time this spring has interrupted all sorts of usually-short-term projects, such as book reviews.

If there is one benefit to the delay, it was my ability to watch the sales keep racking up while the book climbed several bestseller lists. At one level, I think, "I wish my book on the topic had sold a tenth as many copies!" But that's silly; I'm glad someone was able to meet the clear need for this book in a way that's been rewarded.

Bottom line of the review: read the book. In writing the review, I made the choice to skip much of the contemporary discussions around the book and focus on Ravitch's historical arguments. As usual (with Ravitch), she writes a highly appealing argument, and it's important to look at the claims dispassionately. I should say that I dearly wish she were correct in her claim that Lynne Cheney's attack on the voluntary national history standards in the 1990s was a primary cause of mediocre curriculum standards and our current policy obsession with high stakes testing. At the time (as a new scholar in the field) I was very upset with Cheney's distortions of the record, and at one level it is attractive to see her in the villain's role. But I think it's more complicated.

April 15, 2010

Misinterpretations of Crist's veto, and where to go next

I suspect that a number of observers will spin Charlie Crist's veto of Senate Bill 6 to the point where the representation doesn't come close to reality. By a quirk of timing, I was in Tallahassee today talking with legislators and staffers in the morning. In other words, I was at Ground Veto. Yep: I came, and Charlie caved. No, that would be a post hoc fallacy, even if his veto message used the same word (overreach) that I used to describe the bill. Wait: he used a hyphen (over-reach). Or maybe I don't own the term, and the idea had been floating around the state for the last few weeks, including in newspaper editorials, and it was one of the options available for a governor vetoing the bill. So I can't claim credit as being the person who killed the bill, though I was one of thousands who contacted Crist in the last week.

In the meantime people are spinning this as the Event that Destroyed Florida Education, or the Victory of the Union(s), or the Resuscitation of Crist's Senate Campaign. Maybe one or all of those labels is true, but I doubt more than one is. (To calculate the probabilities, we need to use quantum spin dynamics, a new field that melds political science with nuclear physics.) Whoa, friends, and maybe you should take a step back. Here are the reasons why Crist vetoed the bill:

  • Thousands of Floridians from both major parties contacted Crist to urge a veto.
  • His sisters who teach probably told him they hated the bill.
  • The Republican legislators and former Governor Bush who were pushing the bill had largely sided against him in the primary against Marco Rubio.
  • Crist prefers consensual processes.

Crist's veto kills this particular bill, in this form. It does not signal a victory of teachers unions over performance pay, and it does not mean that the Florida Education Association will oppose either performance pay or alternations in the process leading to due-process protections. In fact, if you're on Facebook and "friends" with Andy Ford (he's a nice guy, and the ironic quotation marks are about FB, not Andy), go ahead and see what anti-SB 6 groups he joined... and which he didn't. If you're a reporter, go ahead and talk with Commissioner Smith and ask him to repeat the first thing Ford said at discussions about Race to the Top.

Where do we go from here? It depends largely on whether the FEA executive cabinet will support Andy Ford in negotiating with other stakeholders and politicians, on what the administrator and school board associations push for, and whether the business groups or the Republican sponsors of SB 6 are willing to negotiate in good faith. Here are some obvious questions that don't correspond with any hypothesized litmus tests:

  • Can the key parties agree that a performance-pay framework can exist?
  • Can the parties agree that a performance-pay framework cannot force budget cuts to current operations?
  • Can the parties agree on a performance-pay framework that addresses student outcomes on a "pass a smell test" basis but does not depend on blue-sky assumptions about assessment for students with disabilities, English language learners, and every subject in the curriculum?
  • Can the parties agree that teachers should not automatically receive continuing-contract status (with due process protections) without a more serious evaluation than usually exists (i.e., by default after three years regardless of the scope of evaluation)?
  • Can the parties agree on the scope of personnel contracts that can be negotiated at the local level?
  • Can the parties agree on what due process protections are workable for experienced teachers who have demonstrated effectiveness in the classroom?
  • Can the parties agree on what must be part of teacher evaluations and the range of options for those evaluations?
  • Can the parties agree on what constitutes a proof of concept for their pet ideas?

Disclosure: I am a 14-year member of the United Faculty of Florida and thus a member of FEA. I am firmly convinced that if you are a Florida teacher and want a future with no performance pay, and if you somehow persuade your local and state leaders to agree with you, you will be at the policy table... as the meal. I am equally convinced that if you are Jeb Bush or one of his close friends and want a future with no job security for teachers beyond a single year, you will succeed... in turning a great number of people who would otherwise agree with you into political enemies. And if you think that there can either be a future in state education policy with no high-stakes tests or a future in state education policy where there is a quantified high-stakes test for every subject and grade level... well, I'm not legally licensed to give my opinion of that response.

In other words, many of the questions above have yes as an answer, but only if people who would otherwise hold extreme positions are willing to work on problems rather than positions.

April 14, 2010

Concern trolling about union democracy

Over at Jay Greene's blog, Greg Forster points out that the majority of weighted votes in the last United Federation of Teachers election were not from current classroom teachers. This is corruption! is the implication. Er, no. It's called following the legal bylaws of an organization. I haven't heard Forster call for the reweighting of general-election balloting so that all ages are represented in proportion to their actual population, nor have I heard his call for the abolition of the U.S. Senate, which gives small-population states such as Arkansas power far beyond their relative size, nor any concern from him that in some cities, a single voter controls all of the ballots for school board elections (some people call that mayoral control).

There are potential problems when retirees form the majority of a union's membership, but it's also a problem if retirees who depend on the fulfillment of their pensions have no voice whatsoever in the running of the primary organization defending their pension rights. The weighting in UFT is one of many plausible ways to address the dilemma. From someone who received a Ph.D. in political science from Yale University, one might expect an analysis based on the existing literature on power in different voting systems, and it is disappointing not to see any evidence of such a perspective in what is essentially catcalling.

How not to reform K-12 teacher tenure, and how to do it right

The blog management system here tells me that I started this entry on April 2, which tells you something about the intervening 12 days. We're on the edge of a potential governor's veto of Senate Bill 6 in Florida. As I've written before, SB 6 takes several issues and marches several bridges too far. Most of the concerns with the bill have been covered extensively in print media, and the protests have been covered in both print and broadcast media. I'm going to poke away at a few quirks below as an illustration of the type of problems that are both important and also ignored in the larger discussion and then discuss an example of an alternative policy that would accomplish the bill sponsors' goals more effectively than what the bill says.

Quirk the first: Senate Bill 6 could threaten Florida's eligibility for federal aid on the maintenance-of-effort (MOE) requirement for ESEA. The bill forces districts to sequester 5% of the state financial package of operating revenues for two years in a way that is explicitly not in the classroom. The federal government granted a waiver from MOE for stimulus funds, but there are still requirements, and though the ARRA funds disappear the day before the 5% sequestration rule exists, there are still MOE requirements in ESEA.

Quirk the second: Senate Bill 6 forces districts to spend money to develop end of course exams that Senate Bill 4 refused to schedule because ... er ... the state doesn't have money. If you've been following Paul Cottle's blog, you'll know he's not happy that the Senate removed a provision from SB 4 that would have set a date certain for EOC exams in chemistry and physics. The explanation given in the senate is that the creation of state-level EOC exams should wait until there's money. But ... a provision of SB 6 forces districts to spend money developing appropriate assessments in every single grade and subject for which there is not currently a test, such as ... high school chemistry and physics. By the same deadline that was removed in SB 4. So the state doesn't have money certain for a state assessment, but districts do? And 67 school districts will be more efficient in creating 67 different EOC exams than the state?

These issues don't touch on the larger questions about the constitutionality of SB 6, but they still boggle the mind a bit, since failure to pay attention to MOE requirements could threaten hundreds of millions of dollars for education Florida receives from the federal government.

Now, let me answer the obvious question in response to my argument that SB 6 overreaches, starting with the question of due process for experienced teachers. How else could a state address the question of job security and due process? Let's take the issue of probationary status (i.e., can be fired at any time), one-year contracts (where a teacher can be released without cause at the end of any year), and permanent status (where the burden shifts to the employer to show cause for termination). A number of states have taken different approaches, from extending the probationary period to requiring a certain number of strong evaluations before a teacher shifts from probationary to permanent status. One could also imagine a hybrid of a year or two of probationary status and then a shift to one-year contracts until a teacher has met certain benchmarks of effectiveness.

But why is it in the public interest for teachers to have some job security beyond a one-year contract (the maximum that would be allowed under SB 6)? Consider high school biology teachers first: Do you want biology teachers willing to talk about evolution in a district that is socially conservative and where the school board majority often is opposed to teaching evolution? Do you want teachers willing to give poor grades to students who don't do the work, including if the students are children of school board members (as the father of St. Pete Times columnist Robyn Blumner was able to do with job security)?

Due-process protections provide protection against capricious or malicious disciplinary and termination decisions. There is nothing in Florida law or union contracts that provide for "employment for life," which is what Senator John Thrasher claimed on a public radio program. Florida provides for a 90-day correction period for teachers found to be ineffective, after which a school board can fire the teacher. Neither Florida's collective bargaining agreements nor state law require months and months of legal proceedings to fire an ineffective experienced teacher. 

But let's assume that there is something inside the black box of administrative decision-making that somehow doesn't work with the 90-day correction period for teachers with professional service contracts. There are a number of other options that's far removed from SB 6, including a rolling multi-year contract... say a three-year contract where a satisfactory rating in the first year of the contract means that a teacher has a fresh three-year contract in the following year... or an effective teacher is essentially always in the first year of a three-year contract.

All of these are options that address either the probationary period or the question of job security after a probationary period. It seems that SB 6 could lead to less honest evaluations by administrators than the options I have laid out, because administrators would want most of their teachers to feel secure in their jobs, especially in a school where jobs are hard to fill. This would undermine the claimed intent of SB 6's sponsors.

If Governor Crist vetoes SB 6, I will be relieved, and there will be a chance for a more inclusive discussion that solves existing problems rather than creates new ones.

April 4, 2010

Brief note on iPad: bye, Flash!

The iPad is definitely awkward in some respects. I am currently typing this entry on a software keyboard, which is a bit clunky in large part because I am a touch typist. But this will definitely take off, and the primary result is that all of the educational sites built on Flash are inaccessible. So-called standards-compliant web design will have its revenge! (This refers to XHMTL standards, not curriculum standards.)

April 2, 2010

Florida House budget wants public employees to delay retirement

This one's an odd cut-your-nose-to-spite-your-face maneuver: the budget that the Florida House passed yesterday cuts health subsidies to retirees because legislative leaders are desperate to balance the budget without raising taxes, but the House budget maintains complete state support of premiums for about 27,000 state employees, including state legislators. (Disclosure: I'm not among those who get premiums completely paid.)  

Let's think this through a bit: suppose you're 65 and are thinking about retiring. Before this year, if you retired you'd be eligible to have a health-care subsidy. If the House budget provision on those items remain, you'd probably think twice about retiring, because by staying at work you're covered for what Medicare doesn't help with, and your salary pays the premium, but if you retire you don't have a health-care subsidy.

Now let's suppose you're also one of the 27,000 employees whose premiums come out of your employer. If you stay at work you don't pay for health care premiums. If you quit, you don't get any health-care subsidy.

This reverse the usual incentives that pensions set up to encourage retirement: you lose some income, but you gain some security. The health-care subsidy is not a significant amount of money over one's entire lifetime, but it's something that older public employees had been counting on, and the loss of the anticipated benefit might tip the balance for some to staying in their for a few extra years. Is this what Florida legislative leaders want? Have they asked anyone to estimate the long-term costs of delaying retirement for those who might change their mind based on the health-care subsidy?

April 1, 2010

Hilda Turner and why teachers are skeptical of John Thrasher's motives

In Tampa, there is a five-year-old elementary school named after the late Hilda Turner. The students attending Turner Elementary may not know why it's named after her, or who she was. Most legislators in the capitol probably don't know about her case against the all-white Hillsborough school board in the early 1940s and why the long history of politicized teacher evaluations give Florida teachers reasons to believe that Senator John Thrasher's bill is an attack on them.

But my friend and colleague Barbara Shircliffe knows, and she reminded me of the case today. She published a history of Tampa's desegregation case a few years ago (The Best of That World), and she's currently researching the history of teacher desegregation in the South. In the early 1940s, teachers across the South faced a split between what the federal courts had decreed and what the reality on the ground was. In 1940, Melvin Alston had won a lawsuit against the Norfolk, Virginia, schools for having separate salary schedules for white and black teachers, because the (federal 4th Circuit) court had ruled that unequal salaries were wrong. (In the decision linked above is the salary schedule that shows high school teachers were paid more than elementary teachers, men in high schools were paid more than women teaching in high school, and white teachers were paid more than black teachers.)

But most school systems didn't change anything until they were sued, and it took quite a spine for a teacher to take on her or his employer. Maybe the teaching shortage of WW2 made a difference. Certainly the fact that black soldiers were bleeding for their country played a role in growing militance (including the "Double V" campaign of the Pittsburgh Courier). Or maybe this sham of an evaluation for Hilda Turner in 1942 kicked her into action (Turner v. Board of Public Instruction, reference exhibit 3). The case quickly became messy and ugly, and I'm going to leave the story of that for my colleague's next book. But this wasn't isolated. Black teachers in Florida were treated unfairly and unequally for decades, often by their white colleagues. It probably wasn't until the mid- and late-1960s that teachers of all races in Florida started working together to address teaching conditions in the schools.

Nor were the types of spurious judgments in that evaluation uncommon. The fact that an annual evaluation was one of the lawsuit exhibits may be a legal quirk (since it was damning evidence of how the system treated black educators). But it also illustrates the controlling way that systems treated all teachers, and that continued for decades. In the 1950s and 1960s, they were subject to attacks by the state's anticommunist legislative committee, run by Horace Johns, which eventually turned to outing gay teachers. (If I remember correctly, current U.S. Rep. Bill Young was a member of that committee when he was a state legislator starting out in politics.) Teachers in general were attacked in 1968 for striking, but gay teachers were the target of another attack in the 1970s by Anita Bryant. In the following decade the state imposed a generic evaluation instrument (the Florida Performance Measurement System), designed before the recognition that there was subject-specific expertise in teaching. And all of that came before the Sunshine State Standards in the mid-1990s, Jeb Bush's A+ accountability program, vouchers, No Child Left Behind, the Bush Recession of 2008, and finally John Thrasher's bill. I can point to a number of events or policies that supported teachers, but the background has always been a recent history of blaming and judging teachers.

Because there has never been a sufficiently well-grounded system of teacher evaluation, the experience of teachers on the ground has been ineffective, useless evaluations... or worse. And what teachers see in Senator Thrasher's bill is the "worse" category. Combined with the elimination of tenure (a topic for another entry), the mandate of a formulaic approach to teacher evaluation is too much for many teachers to swallow. This is not the result of hyperbole on the part of the Florida Education Association. This is the result of Florida's history of education.

(For more on the local context of Turner's actions, see Doris Weatherford's history of women in Tampa, pp. 287-288.)

March 30, 2010

Race to the Top winners and losers

So officially, Delaware and Tennessee won (note, Andy Smarick: I spelled both states and your name correctly). But in the side competition (including brackets and sidebar bets), who won and lost?

Those who predicted political decision-making were wrong. I know Mike Petrilli has wondered if politics has intervened in the reviewing process (and thought the secrecy of reviewer identity was political suicide). When New York, Ohio, and Illinois are frozen out, it's hard to spin the choice of Delaware and Tennessee as political (though Petrilli takes a half-hearted stab at it). Addendum: Rick Hess takes a firmer stab at it, though I think you could take any possible RttT awardee list and fabricate a post hoc "this was all politics" explanation.

Those who predicted a "low bar" in getting money were wrong. In the end, when Arne Duncan said USDOE would give the money to a small number of states, he meant it.

Those who predicted "reforminess" as the secret criterion were wrong. All the cool kids were assuming Florida and Louisiana would win because, well, they're the fair-haired boys this year. Wrong! While stakeholder buy-in (or the lack thereof by Florida's unions) was part of the reason for Florida's four-place finish, there were other ways Florida's application lost points, and Michelle Rhee's application for DC fell at the bottom of the Tweet 16.

Here's who won in the side competition: the reviewers. At least at first reading, the reviewers' comments on Florida's application were serious in comparing the application to the scoring guidelines. I'm sure you can quibble with scores here and there, but I think any sane journal editor might be tempted to kill to have this quality of effort from manuscript referees.

Especially in Florida, there's a great deal of second-guessing and spinning after the announcement of results. I'm tempted to pitch in, but I'll decline, at least for today.

March 25, 2010

Florida House committee chair calls security?

Wow: Florida House PreK-12 policy committee chair Rep. Anitere Flores apparently threatened to call in security when a Democratic representative complained that his amendments were not being heard or voted on, and at the end of the committee a group of Capitol security guards stood at the front of the room, between the audience/witnesses and the committee members and staff. This apparently after several hours of mostly hostile testimony on the House equivalent of Senate Bill 6.

There's no claim in the Miami Herald blog report (linked above) that the audience was threatening any of the committee members, so I'm curious why the guards were there.  spent a few minutes watching passionate, civil testimony earlier today over the streaming connection from Tallahassee. The Florida Education Association had declared this a Rally in Tally Day (or Virtual Rally in Tally Day) and asked all teachers in Florida to wear red to protest Senate Bill 6, attacks on retirement funds, and budget cuts. Over the video feed, I saw several members of the audience wearing red shirts, and my guess is that Rep. Flores has had little experience with running meetings when substantial audiences didn't like what her side of the meeting was doing. 

I understand that last year, several thousand community members confronted Senator Mike Haridopolos in a town-hall meeting over expected budget cuts. I have no clue what the political consequences of Senate Bill 6 might be, but I have heard from a number of K-12 teachers that they see the bill (along with proposed changes to the state's retirement system) as a direct attack on them. 

In better news, bipartisan bill passes Florida Senate reforming high school testing

In addition to Senate Bill 6, the Senate also passed an amended form of Senate Bill 4, which moves the state's high school testing program away from comprehensive exams in 10th and 11th grade and towards end-of-course (EOC) exams. Senators from both parties finally "get it" that the so-called comprehensive science exam was counterproductive, and a well-implemented EOC exam system is significantly better than the one-size-fits-none eleventh-grade test. But that doesn't mean the bill is perfect: FSU physics professor Paul Cottle has been diligent in explaining his concerns with dilatory clauses placed in the bill that eliminate any deadlines for physical-science exams.

It's important to keep in mind that only part of the purpose of these exams is to encourage students to go into STEM fields, though it's important to raise the floor of science courses students take in part to reduce inequalities in access to lab-based courses. The purpose of pushing all students to take more math and science courses is because they are going to be adults when they leave, citizens who vote on issues where they should be informed. I want elementary-school teachers to have stronger math and science backgrounds, and so should you. I'd like someone in charge of a venture fund or pension fund to be able to recognize fraudulent science claims without wasting other people's money. And when my oldest nephew finishes his graduate program in astrophysics, I want a ready source of groupies fanatics educated readers willing to pay oodles of money for listen to him to talk about microwave inferometers and the early universe.

Okay, maybe the last isn't a public purpose. But the rest is. We all benefit when high school students have a well-rounded academic education not only in "skills" such as reading and arithmetic but in history, literature, math, and science, and moving from the FCAT to EOC exams is the right step.

Florida Senate overreaches on changes to regulation of teaching

Yesterday, the Florida Senate voted for Senate Bill 6, which would dramatically change the structure of teacher evaluation, contracts, pay, and licensure in the state. A few amendments were approved on the floor of the senate, but only three appear substantive, and the largest changes happened in committee, in part to address concerns about constitutionality for the initial bill. 

As the Washington Post's Valerie Strauss has, most observers have focused on the evaluation, pay, and contract issues, and that's because the intent of the bill is to elliminate any form of tenure, to reorient evaluation around student test scores, and to eliminate the ability of school boards to pay teachers in part based on experience. For a variety of reasons, legislation such as SB 6 is policy overreaching, and as it has in several other ways in the past decade, Florida has gone far beyond any other state in education policy. In part because it is so hostile towards the Florida Education Association, I suspect that some observers will praise the senate even if this turns out to be horrid policy. That way lies Thrasymachus, and it's not pretty.

SB 6 is overreaching. Instead of reducing the protections of tenure, it eliminates all meaningful due process related to job security. Instead of mandating that student outcome data be a part of teacher evaluation, it requires that test scores form the majority of any teacher evaluation system. Instead of moderating the influence of job experience on pay, it completely prohibits any such factor being used.

As a result of this overreaching, school boards are going to be motivated to work with teachers unions on workarounds for most of these issues. For each area where school boards and union locals agree the state has gone too far, they'll figure out another way to provide for some job security, to moderate the effect of test scores on evaluations, or to create a legally defensible proxy for experience in salary structures and call it performance-based pay. It took me about 10 minutes to come up with a few mechanisms for these issues, and I'm not nearly as clever as highly-motivated union officials and superintendents. But as a result, you're going to see highly variable treatment of teachers across the state, which I don't think is the intent of legislators.

There is only one area where the state has an undisputed right to regulate teaching, either in Florida or elsewhere, and that's in licensure. Regardless of what happens in collective bargaining at the local level, any state can decide who has the right to be licensed as a teacher, and at least at first, the part of SB 6 that is least amenable to mediating influences is in the requirement that teachers demonstrate effectiveness to have their professional certifications renewed. Does that mean that it will be tied closely to test scores? That's what I fear. While there's a substantial academic literature on the problems with using either test scores or growth measures, Daniel Willingham's video remains the clearest short explanation for a lay audience. But I'm sure there's going to be lots of testosterone-laced talk about getting tough on teachers, at least until the State Board of Education has to decide what proportion of experienced teachers it's going to non-renew licenses for... and wait for things like lawsuits and backlash from parents and districts.

I expect that I might find a few additional nuggets of unworkable details in the bill, but that's the big picture. If the Florida House passes SB 6 without substantial changes, there's going to be a great deal of turmoil in schools over the next few years, and until the questions raised by the bill are settled about local bargaining authority and the use of test scores in teacher evaluation, there's going to be a substantial cost of the bill in terms of instability.

March 23, 2010

The sugar-daddy amendment to SB 6

Note (March 25, 2010): This entry was written on March 23, before the Senate adopted the Thrasher/Crist amendment. For my thoughts about the version that passed the senate on March 24, see my entry describing it as overreaching

.Among the amendments to Florida Senate Bill 6 filed today is a short amendment sponsored by John Thrasher (Jacksonville) and Victor Crist (Tampa) to address a concern I raised Saturday (and I assume others have also raised): As originally filed and then approved by state senate committees, Senate Bill 6 would essentially punish the Hillsborough (Tampa) school system for having won a Gates Foundation grant because the carving out a portion of teacher evaluation for trained observers would reduce the amount accounted for by student outcomes below the statutory minimum in the bill.

So along comes the bill with a possible solution to this individual problem: a school district can apply to the State Board of Education for an exemption if it's constructed in various ways that match Hillsborough's situation... including the first requirement: "Any school district that received a grant of at least $75 million from a private foundation for the purpose of improving the effectiveness of teachers within the school district may seek an annual exemption..."

In other words, only Hillsborough need apply. If you've got a sugar daddy, you're eligible for the exemption. If you don't, even if you're a school system willing to invest your own money in a similar system meeting all the other requirements, you can kiss any exemption goodbye.

March 20, 2010

Would Florida SB 6 criminalize Gates grant to Hillsborough schools?

Note (March 25, 2010): This entry was written on March 20, about an earlier version of Senate Bill 6. Early this week, the bill was modified to allow Hillsborough to seek an exemption; the amendment was crafted so that no other district could apply, even if they replicated Hillsborough's efforts using local funding. For my thoughts about the version that passed the senate on March 24, see my entry describing it as overreaching.

In the past year, supporters of using student test scores to help evaluate teachers have expressed incredulity when some teachers union officials have been opposed to those moves in states such as California. "We're not even talking about having test scores dominate all evaluation!" has been the tone of such comments, "but student achievement should be one of the important factors."

Whether or not you agree with that position, it's intellectually defensible. This month, though, I suspect DFER members and Obama administration officials are going to do their best to avoid writing or speaking about Florida Senate Bill 6, which takes the approach that student test scores should be an absolute criterion for continuing professional licensure, and undefined "learning gains" should "comprise more than 50 percent of the determination of the classroom teacher's performance" (ll. 1197-1198 of the 3/19/10 version), no matter what subject the teacher is responsible for and whether anything like a value-added measure is technically feasible.

This majority-of-evaluation position is essentially what the state department of education wanted districts and locals to sign off on for Race to the Top, and Commissioner Smith's public support of Senate Bill 6's approach is inconsistent with his earlier claims in December and early January that the department would be flexible about how districts and unions could implement the RTTT MOU. As the head of the Florida superintendents association wrote in a letter to the commissioner, "you and your staff have emphasized flexibility in implementing these elements" (Bill Montford to Eric Smith, January 8, 2010).

In fact, Senate Bill 6 is less flexible than the text of the Memorandum of Understanding on the use of student outcome data for teacher evaluation. Here is the relevant MOU paragraph:

(D)(2)(ii)(1). Utilizes the Department-selected teacher-level student growth measure cited in (D)(2)(i) as the primary factor of the teacher and principal evaluation system. Primary is defined as greater than 50% of the evaluation. However, an LEA that completed renegotiation of its collective bargaining agreement between July 1, 2009, and December 1, 2009, for the purpose of determining a weight for student growth as the primary component of its teacher and principal evaluations, is eligible for this grant as long as the student growth component is at least 40% and is greater than any other single component of the evaluation.

The second sentence beginning with However appears to be framed specifically to allow Hillsborough County to participate; Hillsborough and its teachers union won one of the Gates Foundation multimillion-dollar grants in the fall, and one of the provisions of the grant is to construct teacher evaluation around three components: student data, an administrative review, and observations from a trained classroom instruction evaluator (the last part of the Gates initiative to develop such evaluation expertise). And in the January letter noted above, Montford wrote that all districts should be able to do what Hillsborough and its union had agreed to for the Gates grant.

So what happens if Senate Bill 6 passes?  Well, there goes any value of the Gates award in Hillsborough; the arrangement in Hillsborough would violate the law because less than 50% of the teacher evaluation structure will use student outcomes. Is this really what DFER and the Obama administration wants? Teachers union and district take a risky step in a joint commitment; state punishes district.

Keep in mind that SB 6 is a moving target: on Thursday, a state senate committee changed the bill to eliminate constitutionally-dubious provisions in the original that would have forced local school districts to raise taxes if they didn't do what the bill rquired and that would tie half of teacher pay to test scores. And thus far there is no House companion. But the teacher-evaluation and licensure components of SB 6 are based on a fantasy of assessment data and state authority that is unrealistic and is a slap in the face of administrators and teachers who are working at the ground level to develop better teacher evaluation systems. 

I can't expect Commissioner Smith to acknowledge openly that his public support of SB 6 is a political calculation that he has no choice if he wants to keep his job. His capitulation is sad, since I like Smith and he's done a considerable amount of work in the background to educate members of the state Board of Education and legislators. But those outside Florida are free to criticize overreaching on teacher evaluation proposals, and this is a chance for them to prove that they are not as absolutist as teacher union activists in California and other states claim. So, is anyone from DFER or the Obama administration willing to speak up against the excesses of SB 6?

March 19, 2010

ESEA reauthorization blueprint, the CliffNotes version

I have several meetings today, but I want to write down my thoughts on Duncan's ESEA reauthorization "blueprint" before I forget them. As I wrote over the weekend, Mike Petrilli is reading the substance of the blueprint correctly; the Obama administration is proposing that federal policy walk back a few steps from NCLB's absolutist mechanisms and disentangle the different issues involved in accountability. Petrilli is also correct in seeing a connection between the administration's ESEA reauthorization proposal and the promises by both Duncan and Russlyn Ali to be more aggressive in the department's Office of Civil Rights (OCR). That's essentially the implicit deal the administration is putting out for review by stakeholders: "We won't force states to label the majority of schools as failing, but we will require states to intervene in the worst 5% of schools in each state, and we will be aggressive in monitoring equity issues in other schools." 

At least in theory, this fits with my argument in Accountability Frankenstein that schools have three different types of challenges: the challenge of truly mismanaged schools in crisis, the challenge of inequality, and the challenge of making sure the next generation is smarter and wiser than we are. I argued that NCLB tried to address all of those challenges with the same mechanisms, and it looks like the Obama administration is recognizing that they need different policy approaches: requiring states to identify 5% of schools in crisis, using OCR to address inequality, and pushing for common curriculum standards for the next-generation challenge. 

That's not saying that the proposed mechanisms are going to work. I am less worried about using testing to screen for schools in crisis than others, but I agree with Diane Ravitch that educational euthanasia is a simplistic response. That doesn't mean that states should allow schools with deep problems to fester but that both states and the federal government need to be much more humble about their ability to "turn around" schools in crisis or even replace them with putatively brand-new schools. It's the proposed four-option turnaround mandate in the blueprint that bears the most resemblance to NCLB's cookie-cutter interventions, and that's a matter of deep concern for me. 

Then there is the effective-teachers piece of the blueprint, which is less bureaucratic than NCLB's "highly qualified teacher" approach and the trigger for NEA's and AFT's critical responses to the blueprints (though I think Andy Rotherham is correct that the Obama administration's pushing of a health-care excise tax, abandonment of the Employee Free Choice Act, and passiveness with regard to NLRB appointments is definitely playing a role). The blueprint is very general with regard to its treatment of teacher effectiveness, and it could be consistent either with something like the Toledo peer-review system and Denver's ProComp, or with the problematic Senate Bill 6 in this year's Florida legislature. 

The generally positive response to Duncan's presentations this week (especially from rural-state senator Tom Harkin) suggests that Duncan's hit a number of right notes, at least politically. That's not the same as effective policy, but it's a long way from a 40-something-page document and a law.

March 18, 2010

Constitutional questions on SB 6

Note (March 25, 2010): This entry was written on March 15, about a substantially different version of Senate Bill 6; the obvious constitutional problems were removed by an amendment late in the week. For my thoughts about the version that passed the senate on March 24, see my entry describing it as overreaching.

While the edublogule is chattering on about ESEA's reauthorization blueprint (which Cheryl Sattler and I agree is a misnomer), there's a battle royale in the Florida capital as former state House Speaker and new state Senator John Thrasher pushes S.B. 6, which seeks to take apart existing patterns of teacher certification, pay, and job security. 

After reading news coverage of the issue over the first half of March, I think the bill may have some constitutionality problems. I am not a lawyer, but it strikes me that the type of strong-arm tactics that the bill has may run afoul of several provisions of Florida's constitution:

  • Separation of powers: The bill delegates a task to the Florida Department of Education that may go beyond the agency's authority -- deciding whether a school district's teacher and administrative pay plan meets the statutory requirements (be performance-based with at least 50% of teacher pay based on test scores, and not use years of service or degrees in calculating pay). The consequence for failure to meet statutory obligations is the removal of 5% of the state's basic funding formula from a district a mandatory local referendum to replace that 5% with higher local property taxes. But with the exception of budgetary provisions tied to local tax rates (mentioned in the Florida constitution), I don't know where the state constitution gives an agency the effective power to direct tax rates at the local level. 
  • "Control" of  local school board vs. "supervision" by Florida Board of Education. S.B. 6 would direct certain actions of local school boards in terms of half of teacher and administrator pay and limits to job security for teachers (in an attempt to eliminate tenure). The question here is whether that goes beyond the legislature's authority in limiting constitutionally-defined powers. In 1998, voters approved a number of amendments, including the replacement of an elected state board of education (comprised of the governor and other statewide elected officials) with an appointed board of education, and the current language in the state constitution says that the state board supervises the state education system. But it also says that local school boards control education at the local level. In previous cases, the Supreme Court has decided that the legislature can carve out pay provisions that are not covered by collective bargaining (distribution of bonuses related to the state's accountability policy), but those are generally at the margins of pay issues. Would S.B. 6 go too far in encroaching on the constitutional powers of school boards to control local systems?
  • Right to collective bargaining. The Florida constitution grants public employees the right to collective bargaining, and in the 1970s it took a threat by the court to write the rules before the legislature finally wrote a collective-bargaining statute. Does S.B. 6 violate the collective-bargaining rights of Florida teachers? I suspect that the weakest provision of S.B. 6 in terms of the constitution is an attempt by S.B. 6 to determine half of teachers' base pay; that's far more than the marginal one-time payments that has previously been ruled to be outside collective bargaining. I don't know about the attempt to eliminate tenure. The legislature tried to do that a few decades ago, and the provisions were changed but due-process rights of experienced teachers were not eliminated. 

I suspect that the strong-arm tactics of S.B. 6 (and the state senate's greasing of the bill's path to the floor) may give school boards an incentive to work with teachers' unions to figure out workarounds if it does pass, and before any legal challenges are resolved. A far less radical bill would be a smaller legal target and be less likely to stimulate backlash by school boards. Later this month or in April we'll see what the House response is to the senate's choice.

Addendum: Obviously, since I am not a lawyer, I am not professionally qualified to predict how any court might rule on that or know offhand what precedents might rule. On the other hand, I suspect it doesn't take a jurist to know that S.B. 6 is heading into unique territory, recognized by the long passages of the Senate staff's bill analysis (beginning on p. 16) devoted to two constitutional issues (one of them collective bargaining). This also says nothing about the merits of any part of S.B. 6 (and there's a lot in S.B. 6 that didn't catch my eye in terms of potential legal issues).

March 14, 2010

What the iPad will and will not be

Last time I wrote about electronic readers, it was before the announcement of the Messiah Tablet iPad. Well, it's Pi Day, and whether or not the circle has been squared, for the first time in my life I've given money to a Steve Jobs company for hardware. As I noted in January, I hate reading PDFs on my laptop, I can't read them comfortably on my Sony Reader, and I really need to read PDFs for my job or kill a lot of trees in the process. The iPad costs about the same as other devices that would do the job, and it'll be far more likely to just do its job. And that's the end of the story, at least as far as my purchase is concerned.

But since there is an enormous amount of myth and hype about tablets/larger readers from both technophiles and technophobes, maybe a little realism is in order. After watching the January 27 unveiling video (and tremendously enjoying the Doritos Canada parody--it shows you how far Lorne Michaels has fallen that something like this didn't appear on Saturday Night Live January 30), I've been thinking about what tablet-sized readers could do and what they cannot do.

First, some genres will do well with little additional effort or reworking of production systems. Comics are likely to be successful on at least one tablet/large reader, as is anything that is already produced for a large-ish page size. Some magazines will survive in this way, and I can easily imagine museums producing electronic catalogues. In general, image-intensive texts will benefit. All of this is easily encompassed within any ebook distribution system, but the more visually luscious books and magazines that will benefit from the iPad and other tablets are also resource-intensive to produce, either by artists or the publisher.

With some tinkering (and yelling and screaming), students will get what they repeatedly complain is lacking in ebooks: easy ways to highlight and annotate texts. The lack of annotation capacity in the EPUB ebook standard is a fixable problem, since EPUB uses xml. The ability to share annotations would be even better. I've written about my use of Diigo in teaching, but that's a workaround, and it's awkward every year that passes, with new versions of Diigo and new problems in sharing annotations.

Apart from annotations, it is not clear what interactive systems will work well on a large tablet that doesn't exist already on websites. There are some good tools for interactive exhibits, such as the Omeka package for museums (see its use in the Inventing Europe exhibit) or the WordPress Digress.It plug-in, which allows reader annotation of any paragraph. Omeka is interactive in a navigational sense. Digress.It is interactive with the content, but the paucity of comments on the Digress.It port of Ivan Illich's Deschooling suggests that it is largely theoretical. 

Craig Mod's essay this month on the infinite canvas (a la Scott McCloud) is interesting, but I'm not sure how that might translate into reality. There's an interesting alpha-level website called the infinite canvas that is infinite in the horizontal dimension. Its showcase includes a cute short comic by Neil Gaiman and Jouni Koponen, The Day the Saucers Came, but the interaction consists of clicking on forward/back buttons with simple PowerPoint-style slide transitions.

And then there will be plenty of resource-intensive development efforts that create one-off apps, many of which will be interesting pedagogically and culturally but will be one-time-only projects. If I were interested in managing the creation of an interactive project, I'd probably create it on a website using tools that I know the iPhone, iPod Touch, and iPad could read -- that is, no Flash and no Java. I know there's an App Gold Rush on, but the non-Flash, non-Java, smartly-designed website is going to be useful no matter what's in people's hands or on their laps or desks.

In other words, the iPad has one very obvious tool that's more than an ebook reader (anything that is visually intense), and there will be an obvious extension for tablets and readers in general (annotations), but the rest is not yet clear.

Petrilli nails ESEA reauthorization proposal

After finishing the last entry, I realized I should write something about Friday's USDOE proposal on ESEA reauthorization. But procrastination is sometimes a serendipitous thing, thanks to the Fordham blog: Mike Petrilli's analysis is correct, at least on first approximation. A narrative framework is not statutory language, Duncan's proposal isn't George Miller's, and other Beelzebubs squatting in the filigree, but I had the same general reaction Petrilli did.

I'll write more about ESEA reauthorization later in the week.

Health care reform: how to save lives and money and maybe defuse debates about teaching

Another reason for the House to pass the Senate's health-care bill and both houses to pass a tweak through reconciliation: it would expand existing comparative-effectiveness studies. Currently, massive advertising by pharmaceutics is feeding Americans' existing tendency to ask for huge amounts of wasteful spending on imaging/testing, drugs, and surgery. While NPR has highlighted the cooptation of a research term (osteopenia) in the service of Merck drug sales, it's important to see drug advertising as taking advantage of a broader tendency to overtest and overtreat, not the sole cause. Some other examples: older men take protein-specific antigen (PSA) tests to detect prostate cancer though you'd have to test 1400 men and possibly treat and thus give more than 40 men a substantial risk of impotence and incontinence to save a single life (National Cancer Institute PSA fact sheet). And apparently every year 75,000 people have cement shot into their vertebrae though sham surgery gives close to the same results

The "safe" and thus ineffective way of changing treatment is to give the advice, "ask your doctor." Yeah, right: practicing physicians who see patients 40-60 hours a week are always up on the latest studies published in obscure journals every week or two, and everyone knows that a doctor's advice is always followed. Consider three effective changes in health behavior prompted by research: smoking reductions, switching how parents put their babies to sleep (in terms of positioning), and a reduction in the proportion of older women taking hormone-replacement therapy. 

For example, it took decades for research on the harmful effects of smoking to filter down to behavior. You want to know why my mother quit smoking before I was born? My older siblings told her that it was disgusting, and she became convinced that not only was it unhealthy, it also represented a character weakness. I'm happy that I wasn't exposed to smoking when growing up, and the beginnings of postwar research on smoking's harms was a part in that but not the whole cause. More recently, the Florida Truth campaign was reasonably successful in persuading teenagers that smoking was uncool. Unhealthy? That was going to change behavior on the margins at best. Another social-marketing campaign changed parental behavior on the sleeping position of infants. "Back to sleep" was based on solid research about the relative risks of sudden-infant death and hammered a simple, actionable message rather than talking endlessly about the research. 

If there is a case for research's changing behavior directly, it may be the reduction in hormone-replacement therapy as a result of studies such as the Women's Health Initiative 2002 report on relative risks of using hormone replacement. Even here, I suspect that the drop in use was both from changing recommendations of doctors (the first link in this paragraph is to an article that suggests that the drop in HRT was primarily among those at risk of cardiovascular disease) and possibly also older women's thinking of themselves as savvy consumers--and that can work both in favor of and against cost-effective medical treatment. Fortunately, there is some evidence that the drop in HRT use is leading to a decline in breast cancer. This is a substantial victory for large-scale public-health research.

Why then focus policy on comparative-effectiveness studies rather than rely on the existing hodgepodge system? Insurance companies already try to limit treatment, and they often rely on existing research to justify their decisions. Well, I've got first-hand experience of why bureaucratic mechanisms based in private industry are no more rational than public bureaucracies; though I have a family history justifying early colonoscopies, Blue Cross/Blue Shield of Florida spent several months denying claims. More importantly, the evolution of private decision-making about treatment has led to a lengthy cat-and-mouse game that has not changed the basic tendency of American medicine to overtest and overtreat those with coverage while we fail to cover those who need preventive care and treatment. Then there's the problem with hodgepodge anything: there needs to be balance between investigator-initiated studies and a systematic program of research.

More broadly, there are several benefits of comparative-effectiveness research. First, it provides a level of transparency that industry-generated decisionmaking never can. This is highly dependent on how resistant a comparative-effectiveness program is to corruption, but the private-insurance cat-and-mouse game is a structure guaranteed to lead to distrust and extra costs of operating a system of benefits. The Women's Health Initiative study publication is a case study of why comparative-effectiveness research is not only important in controlling costs but also in saving lives. The WHI study was large and credible, and the reports were published broadly in the general press. Second, the results of comparative-effectiveness research can be the foundations of more secure efforts to change behavior. We're always going to have bad medical-research reporting (quick: is there a research consensus on the effects of coffee drinking?), but it is going to be easier to write guidelines, communicate a message, and gain funding for publicity efforts if it is clear and credible. (Small aside: that's an obvious and appropriate role for foundations, not to fund marginal research but to fund public education efforts based on a solid research consensus.)

Third, a comparative-effectiveness research program can lead to professional standards of care that are less susceptible to manipulation based on context. Yes, doctors will sometimes grump about that. But Atul Gawande might have a few things to say about the value of checklists and the dangers of assuming professionals can just "wing it" when in an examination room. In doing so, health-care reform will move us one step away from thinking about professionals as a hero-artiste, and in turn that will move us in the right direction on talking about teaching.

So, to teaching: Having professional standards of care/practice based on research is a reasonable alternative to either laissez-faire approaches to teaching or assuming that the black box of incentives will magically improve results. That doesn't mean that it's easy. Larry Cuban's response to the story Elizabeth Green wrote for the New York Times is correct: the history of micro-teaching advice is long and not particularly successful. And I have no illusions that just because you say you're in favor of professional standards of care and practice means that there will suddenly be a body of rigorous research.

But anyone who believes in the hero-artiste model of teaching in the public schools needs both a political and ethical reality check. If you're paid by the public purse, you have an obligation to the public. Public school teachers need protection from corruption, unreasonable demands, and retaliation in response to whistleblowing. But that protection doesn't mean that an elementary school teacher should be able to teach what he or she wants, when he or she wants, how he or she wants. The practical and political tradeoff for some autonomy in the classroom is the adherence to recognized norms of professional behavior. That includes how teachers treat students, how they respond to a formal curriculum, and the instructional tactics used.

It's the latter that Green's article addressed. My guess is that teachers can argue either that they should be evaluated based on results or based on professional standards of care/practice tied to research, including research in the future. But you cannot argue that there should be no professional standards, or that a good chunk of them should not be tied to research. The "incentives" focus of much current accountability puts instruction in a black box. I think that's inappropriate public policy, but there has to be an alternative for at least political purposes. Changing the talk about doctors, checklists, and comparative-effectiveness research is a way to show that professionals do not have to be hero-artistes, and that's a healthy direction for the country.

March 12, 2010

Health care and financial-aid reform as a package

Wednesday's rumor has turned into Friday's semi-confirmation: Democratic leaders in Congress are looking very seriously at packaging together the changes to the Senate health-care bill with the Student Aid and Financial Responsibility Act (SAFRA) through budget reconciliation. SAFRA would end federal subsidies for bankers that initiate loans for college students and return an estimated $67 billion over many years to be used for better purposes, such as giving poor students Pell grants. Since taxpayers foot the bill for the banker subsidies that currently exist, students end up paying twice for their own loans, once in interest to servicers and a second time in taxes that go to banker subsidies. It's time to end the double taxation of students.

Politically, the packaging is a good move for multiple reasons. Matthew Yglesias argues that putting SAFRA in with the health-care bill changes will reassure House progressives that one of their priorities will get a vote in the Senate, and it might get SAFRA over the hump of the small number of Senate Democratic naysayers who are siding with lenders over students. Last night, Sara Goldrick-Rab explained the shame of the anti-student bank subsidies, and it sort of burns me that one of the Democrats signing the protect-the-poor-bankers letter to Harry Reid is Florida's Bill Nelson. 

To be honest, I expect the package is more likely to attract support from the Nebraska Senator Nelson than Florida's Senator Nelson, because Ben Nelson (NE) now wants his embarrassing Cornhusker Deal for health-care off the table. But both Senator Nelsons are on the wrong side of the issue with SAFRA. I e-mailed Bill Nelson to that effect early this morning, but each time I've called his Washington office today, it's been busy and the voicemail is full. Time to call his local office on Monday...

March 11, 2010

A health-care mandate for education

Over the next week I'll be writing a few entries about health care. Today, it's the benefit of an individual mandate for college completion. The individual mandate is the package deal that goes with universal coverage, to get healthy people into the insured pool, and it's also important to help college students finish. Every year, USF and every public college and university loses students because they get sick or have a financial crisis because they or a family member get sick. Even if imperfectly enforced, an individual mandate would give colleges and universities the political ability to require proof of insurance upon enrollment, and that would safeguard both the individual investment of the student and her/his family and the public investment as she or he starts college.

Yes, there are alternatives, but they're all bad: Many colleges offer a very high-premium plan for students because the pool they can compose out of their students (or a fraction of their students) is tiny. Together with the option to stay on their parents' plans until 26, an individual mandate would give college students more choice by letting them enter the exchange markets instead of having one horrid option for health insurance in college. An individual mandate would make sure that our public investment in higher education is not wasted by a spurious event that no one can control.

(Obviously, someone in the White House reads my blog because they're emulating thought of the same idea as they echo my uninsured-death-every-24 21-minutes entry in their final push, highlighting key numbers on the issue:  625 Americans who lose health insurance every hour, 8 health special-interest lobbyists for every member of Congress, 8 Americans denied coverage every minute either by loss of insurance or other means, and $1115 paid every month on average for a family premium.)

Kristof and the public purpose of feel-good years

Charlie Barone is right: Nicholas Kristof's column yesterday comparing TFA and the Peace Corps shows the practical limits of TFA (as well as Kristof's ignorance about VISTA, but that's a different story). There's something important about consistently reminding reporters and other naive folks that TFA is not scalable. Regardless of what you think of it, there is a vast difference between the needs for a professional long-term teaching corps and matching up a few thousand new college graduates with positions that would be filled at best with long-term substitutes. There's nothing wrong with short-term backfilling (heck, that's what ARRA and other stimulus bills are for), but that's not a main solution for much.

Barone's point is not really about Kristof's central argument, which is more about how young Americans need to experience more of the world. Kristof is right about that, though maybe they should also see more of their own country? Nor is it about the side benefit of TFA participation in terms of giving a broader group of young adults experience in the public sector.

I think the last is the lasting impact of TFA. I look more favorably on TFA than a lot of other education researchers, not because I think there's significant evidence of great results but because a backfilling role in urban systems is acceptable and because social movements need well-off and well-positioned allies, people who had formative experiences that led them to empathize with others. In Inventing the Feeble Mind, for example, James Trent documents how WW2 conscientious objectors' experiences in state institutions helped lay the groundwork for a postwar change in attitudes towards cognitive disabilities. That's not a pre-law internship, as some accuse TFA of becoming; regardless of naivete, two or three years represents a serious commitment for someone who's 22. I don't know where TFA alums are going to be, but few of them are like Michelle Rhee either in temperament or future careers. Somewhere in 10 years, a TFA alum far outside public education is going to make a difference in a different sphere of life because of those two or three years.

March 9, 2010

Florida v. Georgia -- in budget crises, not football

Today's revenue-estimating conference in Tallahassee is probably going to confirm prior state revenue estimates, which are slightly better for 2010-11 than 2009-10, but that's like saying two broken legs are better than two broken legs and a broken floating rib. The state revenues are still far below 2006, and there are three sources of pressure on the state budget: increased demand for Medicaid, the federal maintenance-of-effort requirement for education (even with the waiver for absolute maintenance), and declining property-tax collections that support K-12 school districts.

Last year we kept reminding ourselves that we weren't in California. And this year, Georgia's picture is worse. Plus a few other states I could mention. But that's cold comfort: Schadenfreude doesn't pay the bills.

Updated (5:45 pm): Yes, today's Florida state revenue estimates are almost identical to the last round.

March 8, 2010

Sour-grapes agreement

Michael Olneck and Peter Sacks turn petty in letters to the editor about Diane Ravitch that the New York Times printed today. Wow. I agree with Ravitch on a number of things and disagree with her on a number of things, some of which is in our area of expertise (history of education) and some of which falls outside the history of education. But I'm not sure why Sacks in particular is turning on the venom spigot. Well, actually, I do have some hypotheses about general hostility to her I've occasionally seen (as opposed to disagreement): she caricatured the field of history of education in a sloppy late-70s publication sponsored by the National Academy of Education, and along with Patricia Graham she was a woman to get high-status national recognition in the 1970s for her work in education policy at the national level, which heretofore had been a male bastion. (Graham was director of NIE from 1977 to 1979.) The first is a seriously flawed work, but that's several decades in the past, and in any case, a particular work should stand or fall on its own merits. I've never seen the second item discussed or even acknowledged. 

There's a related issue here, which is Ravitch's position outside traditional faculty. As far as I'm aware, she's never had a tenure-track or tenured faculty position, and she's one of the few historians who can say that they published their dissertation commercially before receiving the Ph.D. (The Great School Wars was published in 1974; Ravitch received her Ph.D. from Columbia in 1975). For the most part, her books are far more widely read than those of us who have full-time faculty positions, and I think she and Graham are the only historians of education to have held political appointments in the federal government. That's an interesting combination of insider and outsider positions. 

When Meier and Ravitch started their joint blog/conversation three years ago, I briefly referred to this history in writing, "Regardless of various professional views of her scholarship, Ravitch is a recognized voice on education policy. There are plenty of people I correspond with who have fewer claims to expertise, so I can either have a snit-fit about that or deal, and at this point, having a snit-fit is darned close to sexism and uber-testosterone in education policy studies." I'm sorry Olneck and Sacks, and especially Sacks, have made a different choice.

For the record, Sacks is factually wrong when he states, "Dr. Ravitch fashioned herself into the Ayn Rand of educational policy and rose to fame as a result of a free-market ideology that came into fashion in George W. Bush's administration." Ravitch's appointment was during the first Bush administration, and whatever you might think of Ravitch's historical arguments in different books, she's a much better writer than Rand.

March 7, 2010

Historians' automaticity, part 1

Concerns with science and math education are nothing new, and although the rhetoric today focuses on saving the planet and the economy, the argument for urgent intensification of STEM education is remarkably similar in structure to the Cold War era debates in the 1940s through the early 1960s: our country is in crisis, we need science and technology to solve the crisis, and so we must reform education. A 1959 forum about science and math education at Woods Hole was summarized by Jerome Bruner in The Process of Education (1960), which essentially was an argument about education in the disciplines. (Bruner later was instrumental in creating Man: A Course of Study [MACOS], and fellow Woods Hole conference participant Jerrold Zacharias was a key mover in MIT's Physical Science Study Committee, whose materials were used by my high school physics teacher.)

For a number of reasons, MACOS flopped as a curriculum project, but the central question raised at the 1959 Woods Hole conference remains: what's necessary for students to be successful at learning disciplinary thinking? Several of my colleagues at USF (Will Tyson, Kathy Borman, and others) have been involved in NSF-funded work studying recruitment to and success in undergraduate STEM education, including preparatory math and science work in high school. In lower grades, the National Math Advisory Panel made some suggestions about curriculum in primary and intermediate elementary grades that would be prerequisite for success in algebra, including work with fractions. (Speaking of which, check out this very cool Java Spirograph simulation. Yes, it's connected to fractions... or rather the nature of reciprocal relationships between frequency and wavelength.)

And somewhere along here, along with debates about the purposes of various proposed curricula, we generally get debates about which is more important, procedural fluidity or conceptual understanding. My answer: yes. They are. You need both "content" and "process" (and we'll get to the problem with those terms shortly), and I am generally sympathetic to arguments that getting to the point of automaticity with core skills is a part of getting ahead in conceptual understanding and also needs to be matched by teaching of concepts. (See my entry a few years ago on how to explain the more recent and reasonable NCTM curriculum framework materials.)

But there is something about the term automaticity that itches inside my head, because it sort of gets the idea right but is not entirely persuasive... and the places where it is not persuasive are troubling in a subtle but very important way. Let me explain why I can fluidly pull out material from my memory that looks remarkably like the standard definition of automaticity and yet really isn't like that at all. 

First, a digression: with apologies to Douglas Adams, the process of doing history is almost but not quite entirely unlike what Sam Wineburg describes in his research. Wineburg's writing is appealing to historians because it focuses on precisely the discipline-based processes that Bruner discussed 50 years ago in his book, and Wineburg's message is flattering: "academic historians, you have interesting ways of thinking, and here is what I see as a cognitive researcher and why high school history teachers need to pay much closer attention to what you do." And to be honest, there is some part of his work that has all sorts of interesting detail on the level of nuance and sophistication with which people try to commit history (such as the research on how people from different fields read primary sources about Abraham Lincoln and slavery). But Wineburg is enormously popular because his intended audience has a confirmation bias that leads them (us) to agree with someone who comes along and tells us we're special and intellectual. Wineburg weaves a story of historical thinking's exceptionalism... and there's the rub. As an historian, I'm supposed to be wary of anyone talking about American exceptionalism, and here comes this cognitive psychologist trying to seduce me with glorious tales of my discipline's exceptionalism, how difficult it is to be an historian, and so forth.

Pardon me, but I'll take the interesting cognitive questions without the side dish of (probably unintentional) pandering. A good bit of Wineburg's efforts have been to parse out how people read primary sources, and they generally focus on the level of ambiguity people read into primary sources: ambiguity about intent, background, effect, and so forth. And that's all fine and good except for two problems: Wineburg's work in this vein has generally been with adults, and they generally ignore the process participants use to put the primary source in context. The second is the part that troubles me most as a teacher, because the place where students in my undergrad history of education class first fall down is typically in putting a primary source in a broader context. It's not the most difficult task I put before students: usually the most difficult task in the semester is asking students to provide historical perspectives on a contemporary issue. But the difficulty of putting material in a broader context is a fundamental barrier to success in my class.

That sounds remarkably like students who are not yet at the level of history automaticity, whatever that might mean, and one would be tempted to refer to Checker Finn and Diane Ravitch's argument from the late 1980s, that American teenagers don't know enough history. But focusing on factual recall is begging the question: what does it mean to have sufficiently fluid mastery of history to put a primary document in context? Something about factual recall is helpful, but is that enough, and is that what successful students do? 

It might be helpful to explain the type of task that is not hard for students: confronting people whose glib brutality stands out of the page. That characterizes the very first primary source I use in my undergrad history class (printed in Jim Fraser's education history primary-source collection), instructions from the London Virginia Council to the colony's governor in 1636. It reads in part,

And if you find it convenient, we think it reasonable you first remove... [Native American children] from their ... priests by a surprise of them all and detain their prisoners... [and] we pronounce it not cruelty nor breach of charity to deal more sharply with [the priests] and to proceed even to dash with these murderers of souls and sacrificers of gods' images to the devil...

With 17th century texts, the first challenge is simply to understand what the source says, and that's a bit of skill in language, but the students usually figure out this passage soon enough, and their eyes open a bit wider: the official supervisors of the colony sitting in England were telling the colonial governor to kidnap Native American children and beat (or kill) the elders. That type of detail sticks with students, because it engages their emotions and sense of what a society is supposed to be doing (as well as what colonists did). It's not that any student is exactly surprised that English colonists in Virginia were patronizing and occasionally brutal, but there is something that takes them aback in the casual way which which colonists and English elite discussed their goals. 

I wish that all of history was that engaging, but that's just not true, and there is a good bit of background context that students need to pull out to put any primary source in context, and when you get to material whose explicit text is boring but is still important, students cannot rely on the immediately-engaging story to "get it." Instead, most primary sources require a student to identify at least one salient context that is not immediately apparent, and they need to be able to identify a relevant context (or more than one) without a huge amount of effort. If there is an "automaticity" to a professional historian's thinking, it is that: where does this primary source or other detail fit in a large scheme?

That larger scheme can start with "issues of the day," whatever the time and place. To be successful, you need to know what was happening at about the time of the primary source/event. You start with the year, go back and forth a few years, and think about possible connections. So when you look at the last of Horace Mann's annual reports on the state of education in Massachusetts (in 1848) and read the following passage, what pops out as contemporary and possibly relevant?

Now surely nothing but universal education can counterwork this tendency to the domination of capital and the servility of labor. If one class possesses all the wealth and the education, while the residue of society is ignorant and poor, it matters not by what name the relation between them may be called: the latter, in fact and in truth, will be the servile dependents and subjects of the former. But, if education be equally diffused, it will draw property after it by the strongest of all attractions; for such a thing never did happen, and never can happen, as that an intelligent and practical body of men should be permanently poor. Property and labor in different classes are essentially antagonistic; but property and labor in the same class are essentially fraternal.

That's from the middle of the 19th century in the U.S. So when I ask a class about the relevant context, some students look at servility and suggest slavery as an issue, point out that Mann was writing for an audience in the North, or ask whether Mann was anti-slavery. (No one in my classes has mentioned the compromise of 1850, but that would fit with this tentative reach for context.) Few of them would have heard of Eric Foner's book on free-labor ideology, but I can probe a bit: slavery's part of the picture, at least in rhetoric, but there's something else there. What were some of the concepts used in the North to discuss slavery? I wish that probe worked more frequently than it does, so I usually point out the "different classes" phrase and ask what else was happening in the U.S. in the 19th century. At least one student usually mentions industrialization. So what's Mann arguing, I follow up? More faces light up at that point.

Part of the problem here is that Mann's argument is too familiar, a little too close to a human-capital argument for students to realize how new this was. (Maris Vinovkis credits Mann with that early human-capital argument.) Part of it is also that students don't have a visceral sense of the simmering conflicts in Northern cities, even after hearing about the religious conflicts in Boston in 1836 or Philadelphia in 1844 (the latter so-called "Bible riot"). Because all of that was also related to social class, industrialization, and immigration, I can almost feel Mann's sense of urgency here in promoting mass education ("common schools") as a cure-all for social conflict. But most students usually can't. The prose is too prosaic and the context insufficiently emotional to engage students in the same way that happens in response to the "kidnap the kids and eliminate the elders" instructions from the 17th century.

There's an additional layer to this context, because 1848 is a signal year in European history: revolutions galore and the publication of the Communist Manifesto. To a literate, well-connected American, Europe was dissolving in chaos in 1847 and 1848. What could prevent the U.S. from doing the same? There is no evidence I am aware of that Mann was explicitly referring to European events, but it would have been in the air in the same way that natural disasters are "in the air" around the globe today after the earthquakes in Haiti and Chile. Even if he was not consciously constructing the passage above to respond to European events, it would have resonated more for someone concerned about social stability in 1848. 

There is nothing special about what I do in class: I take a simple question of context to push students about the importance of something Horace Mann wrote. And there is nothing particularly hard about asking what else was happening at the time. But while it's an easy task for me, this task flummoxes a lot of students. That task of pulling relevant context out of one's memory is the closest thing I can think of for the historian's automaticity, and looking for contemporary events and issues is the most obvious (but not the only) way to cut the issue. One might want to call this type of context affinity in time. I can think of other affinities which I might explore in other entries, but the key thing here is that this task is extraordinarily difficult for students. 

Why this is difficult is an interesting, substantive question beyond the usual "fact-process" dualism. You need a mastery of chronology to pull context out of your head, but to build that mastery you need a way to put the details into your head in a way that's not "one damned thing after another"--i.e., a mental scheme. And while I wish I could look inside my head to see what my internal schemes are, I suspect any attempt at reflection is going to fall far short. I suppose one metaphor might be a "thick" timeline of issues and events and trends inside my head, so that when someone says, "1848," I can think of a bunch of things (as described above). Or if someone tells you that the Little Rock crisis was in fall of 1957, you just might think of Sputnik and ask whether there might be a Cold War context to Eisenhower's decision to nationalize the Arkansas National Guard and send in the 101st Airborne.

In addition, you need to be able to filter out nonsalient issues. What else was going on in 1957? Let's see: the Ford Thunderbird that year was a particularly popular "muscle" car. And the Dodgers were planning to move away from Brooklyn. The Communist party won elections in the Indian state of Kerala. ABC started national broadcast distribution of American Bandstand. On the Road was published. You can find more details at the 1957 Wikipedia page, but going to an almanac-style "here's what happened" listing is an incredibly inefficient way to put something in context. But to be honest, I wish I had the problem of students who found too many potential contexts where I had to suggest filtering. Usually the problem is a lack of candidate hypotheses about context.

March 4, 2010

PolitiFact Erratum

The St. Pete Times's PolitiFact comes down today with the same ruling that I would on Governor Charlie Crist's statement that the high school graduation rate in the cohort just graduated last year is the "highest it's ever been." They rate the claim as Mostly True, and I agree.

And their reporting of my remarks when called by the reporter on the story is similarly Mostly True. (For the record, that's the way I'd rate most good reporting.) The ruling says in part, "Dorn says the state should not count students who received a diploma even after failing the FCAT three times." It is true that I pointed out that the number of students who receive an academic diploma using the SAT/ACT exemption path has ballooned in the last five years and corresponds very neatly to the rise in high school graduation over that time. However, I never said that the state should not count those graduates, and if I remember correctly reporter Lee Logan never asked me that directly in the phone interview.

On Tuesday evening, Logan e-mailed me, and after I replied with my cell phone, I pulled up the spreadsheet I'd downloaded from the FLDOE site in the fall. The state reported three different measures: the official Florida graduation rate it's used for a decade, the measure used for NCLB purposes, and a measure defined by the National Governors Association in 2005. The last addresses the concerns I and others raised 4-5 years ago about the exclusion of the dropout-to-GED path from the cohort base and the inclusion of GEDs with regular diplomas.

The SAT/ACT exemption is different. On the one hand, the idea of an SAT/ACT exemption flies in the face of the point of a graduation exam, since college admissions exams do not test what a student has learned from the high school curriculum. On the other hand, it's a political and practical safety valve since it gives students more opportunities to qualify for an academic diploma. I wish that the state had chosen other options because of the SAT/ACT-curriculum disconnect, but when faced with education policy problems legislators tend to reach for tests, some test, any test.

Trying to look at the NGA rate with/without the exemption category (WFT) is also trickier than with the GED data, since there could be a number of reasons why the use of that exemption has ballooned. Maybe there are now 9,000 high school students each year who are directed towards the SAT/ACT who really would not have graduated without the exemption, and if so the rise in the NGA represents students who would have been on the margin of receiving a standard diploma without that option. But maybe the rise is a consequence of more Florida districts paying for students to take the SAT, where students would have taken the FCAT but didn't because they had qualified through the SAT. From a student perspective, if you've failed to pass the diploma threshold in prior FCAT tries and suddenly you have an SAT score that qualifies, why take the FCAT again in your senior year? Or why try hard at it when you do take it? 

Then there's the more important question: where should we be with high school graduation? If you agree that we should include the students who qualified with an SAT or ACT score rather than a curriculum-based test, about three quarters of Florida ninth graders are graduating within four years. Using the NGA rate, of the African American students entering ninth grade in the fall of 2005, about 65% of them had graduated with a standard academic diploma by the summer of 2009. Even if you are skeptical about the inherent value of a credential, high school diplomas do serve as credentials for the job market and colleges, and someone without that credential faces significant institutional barriers to doing well as an adult. 

Update: The PolitiFact page has changed to reflect what I said more accurately. Thanks!

February 28, 2010

Larry Cuban has a blog

I have a bunch of reading to catch up on, more than I thought I did a few minutes ago: Larry Cuban has a blog! (Hat tip.)

February 26, 2010

More TFA in Miami-Dade: where's the money?

The Miami Herald is reporting today that Teach for America is going to send 350 recruits to the Miami-Dade school system, supported by a $6 million grant from the James L. Knight Foundation (hat tip). Thanks to the federal stimulus, the Miami school system avoided laying off hundreds of teachers this year, but it's not as if there are large numbers of paid positions that are going unfilled. So the TFA positions are going to supplement, thanks to the Knight Foundation? It might sound good, but do the math: about $17,000 in donations per TFA recruit. This just doesn't add up.

February 24, 2010

Title II proposal: TNTP, meet Florida's "Ippy-Dippy"

The New Teacher Project has a new advocacy brief out proposing changes in ESEA's Title II, which is supposed to focus on personnel development. Some of the observations and proposals make sense (let's stop paying money for 90-minute drive-by "professional development"). Some are essentially using Title II as a vehicle for pushing other agendas (teacher evaluation and differential pay), though only some of it fits easily within Title II (here, training administrators and peers to evaluate teachers makes sense in Title II).

And some ideas are proposed as brand-new but have been tried before, including the suggestion that professional development be tied explicitly to the needs of students that teachers have at the moment. That seems to me to be remarkably like the Florida mandate for an "Individual Professional Development Plan," or IPDP. I've heard the complaints of too many teachers about the IPDP, which is usually pronounced Ippy-Dippy: it's another few hours of paperwork to complete each year with no real individualization of professional development. In other words, in Florida it results too often in paper compliance only.

But I'm only an historian listening to teachers in one state. If you live outside Florida, does your state mandate anything like our "Ippy-Dippy" form? What happens outside the paperwork?

February 22, 2010

The cliff, layoffs, and another stimulus/state rescue

Andy Smarick's analysis is correct: Arne Duncan and Barack Obama are talking this week about pending teacher layoffs to lay the groundwork for more stimulus/state-budget-rescue discussions. I suspect I have one analytical and one policy disagreement with Smarck, though. First, it's not a "second stimulus," because we're moving into a period where there will be a lot of smaller spending packages, so this is going to be the fourth or maybe fifth stimulus proposal in this Congress. And this is going to be harder than an extension of unemployment benefits or anything else that costs under $25 billion, because states and local governments are still in horrid fiscal shape. I suspect Smarick would oppose another large federal rescue of state budgets, but I think it's absolutely necessary or we face another 1937. Teachers and other civil servants don't spend as much of their income immediately as those receiving unemployment benefits, but it's still a better emergency economic policy to keep most public employees at work than any tax cut except the publicly-invisible withholding reductions implemented last year. And as happened last year, there may be considerable inconsistency in the behavior of state-government politicians, many of whom may publicly be horrified about any additional federal spending but need it in reality. (The public conflation of TARP and ARRA doesn't help here.)

For anyone still arguing for budget cuts as public-policy colonic, all I can say is that I hope that you can still argue that point in a year or two without risking pitchforks from the public, if only because you didn't win the argument.

February 21, 2010

A closer look at HB 1009 (proposed expansion of Florida corporate tax-credit vouchers)

On Wednesday, I discussed my first reaction to news of Florida House Bill 1009, which I thought had some eye-popping proposals, and I posted Jon East's response. I've had a chance to look at the text of the bill, and there are some details hidden in there that are interesting.

  • The reporting of test scores for schools with 30 voucher students isn't for 30 vouchers students in any year but for those with at least 30 students who continue from year to year. That dramatically shrinks the number of schools that would have scores reported, and they would only be reported for continuing students, in contrast with public schools that report status test scores (which are part of the Florida system of labeling schools) as well as Florida's jerry-built "learning gains" measure.
  • The financial reporting requirements in the bill is only for schools taking vouchers worth a total of $250,000 per year. Let's assume that at some point FEFP funding per weighted student is $8000, and the 80% voucher is $6400. That would be about 39 students as the threshhold for the financial requirements. At the current voucher level the threshhold is 64 students. I suspect the financial reporting requirement would affect a tiny fraction of the schools accepting vouchers.

One other thought: if this bill passes, then the other large voucher program (for students with disabilities) will remain without any accountability for student outcomes. That's a huge question mark in terms not only of constitutionality but also state compliance with federal special education law. How are state assurances on providing a free appropriate public education affected when state general revenues flow through vouchers, either directly (as in the case of the disability-related voucher program) or indirectly (through the corporate tax-credit voucher program)?

Jon East responds on corporate tax-credit voucher expansion (HB 1009)

Jon East, a former reporter for the St. Petersburg Times and now Research and Communications Director at the corporate tax-credit voucher non-profit Step Up for Students, was unable to put the following long response in comments to my discussion of HB 1009, and when he e-mailed them to me, I agreed to post them here:

As always, my friend, you offer a provocative commentary and I appreciate your recognition that this bill adds two new mechanisms of accountability even if they don't go far enough for you. Please allow me to probe two different claims you made here, though, because I don't think they present the clearest picture.

First, the assertion that the bill "expands the dollar amount per voucher from the state beyond what the state gives local school districts" feels a little like it was intended to obfuscate. Any general assessment of cost should probably begin with the fact that this option is now and would continue to be the lowest-cost education option in Florida -- lower than traditional public schools, charter schools, McKay scholarships, even virtual instruction programs. That aside, the amount the state "gives local school districts" is controlled by a 1973 law that essentially puts all the money for the base funding formula, called the Florida Education Finance Program, into one pot. As we know, the state has been putting less money into the formula while increasing the amount of "required local effort" for property taxpayers. But the larger point is that the FEFP is intended to make funding more equitable, with the breakdown between state and local portions varying county by county according to the size of their property tax base. To then turn around and compare the state portion as though it is some isolated variable in school funding seems contrived. The bill would place the scholarship at a 20 percent discount on this FEFP formula, and the FEFP is only part of the overall revenue picture for schools. By way of comparison, the per-student FEFP in 2007-08 was $7,143. The total revenues per student, including state, local, federal and capital, that same year (this is the most recent one DOE has published) was $11,017. So back to your point: The bill would indeed increase the scholarship amount up to 80 percent of FEFP. But if it took effect all in one year instead of four and it took effect today, the scholarship would translate to $5,490 -- which is almost precisely half the total per-student spending in public schools.

Second, your concerns about the "elimination of the cap" take a little liberty with the wording of the bill and a little more license with the current marketplace. The bill absolutely eases the process of increasing the cap, but one fact worth noting is that this would still be the only major education option with any cap at all. There are no caps for charter schools, McKay scholarships, virtual schools. And the McKay scholarship, as one example, grew only about 3 percent last year. The controlling factor for any school option is ultimately the students. If students and families aren't interested then the program doesn't grow.

In e-mail correspondence, East made clear that when he was discussing the Florida Education Finance Program, he was combining the state, local, and federal sources, and that HB 1009 also was discussing a voucher of eventually 80% of the combined state and local (but not federal) funding. From one perspective East is correct: the legislature sets the amount of funding that comes in total from the state's general revenues, from state trust funds, and from "required local effort" property taxes at the local level, and then a complex formula determines what the required local effort is from each county. Counties have a certain amount of additional property taxes they can levy on a discretionary basis, but FEFP is a unitary mandate in the sense that the legislature determines the base funding for students, and that legislative mandate is met jointly by state revenue and local property taxes.

On the other hand, I think that legislative history will be cold comfort to local school board members and county commissioners who have seen the state shift school funding in the past decade away from state revenues and towards local property taxes. That use of FEFP to shift taxes allowed former Governor Jeb Bush and the Bush-era legislative leadership to claim that they were lowering taxes when a good part of that was a clever shell game. (This is not a particularly partisan flaw: the Democratically-controlled legislature played a similar game in proposing a state lottery in the late 1980s which legislators claimed would boost education funding without raising taxes.) Many school board members will see HB 1009 as a drain of state revenues that will contribute to the shifting of education funding away from the state and towards local property taxes. When the legislature created FEFP in the early 1970s, they quickly ramped up per-pupil state support of education, effectively shifting the revenues from local property taxes to the state's general revenue pot. HB 1009 looks like it will continue a reversal of that original intent.

February 17, 2010

Zombie idea: short-cut high schools

I guess I was wrong yesterday in labeling Utah State Senator Chris Buttars as obviously out of the mainstream in terms of practices. The New York Times reports today on something that slipped under my radar: two-year high school degree options based on an idiosyncratic constellation of what the article calls "board exams." Not too surprisingly, this is the brainchild of Marc Tucker, who's been peddling some type of policy proposal for changing the structure of high schools for more than twenty years as head of the National Center on Education and the Economy. I think this primarily shows that he's been more effective at lobbying than the higher-profile Partnership for 21st Century Skills, because it look like the various testing options essentially would require students to succeed at 12th-grade somewhat-more-intensive coursework, and my reasoning yesterday on three-year programs holds here (students likely to succeed at these tests are going to continue through four years). Given the original argument of NCEE for a much more vo-tech focus, I'm not sure what to make of this, except that it looks like it will have minimal impact on actual enrollments. 

February 16, 2010

Heady, headwinds, or just headstrong?

Two education stories caught my eye this evening: the attempt by a Rhode Island superintendent to use state law on school interventions to trump collective bargaining and fire all teachers at one high school, and the trial balloon floated by a Utah state senator to end 12th grade in the state, or make it optional. 

Utah State Senator Chris Buttars is just being headstrong, or maybe "grasping at straws" might be the better term, since the idea of skipping a year of a four-year program isn't attractive to many students even when it's possible. Florida created a three-year high school structure some years ago, and it's almost entirely unused, for obvious reasons: students who are doing well-enough in high school to finish in three years are also eligible to attend a number of colleges, and will be told by high school advisors, their parents, and others that they darned well are going to spend a fourth year in high school so they can attend a college of their choice. And in colleges, while many students could use AP credits to graduate in three years, that's not a common pattern. An old friend of mine was able to finish high school one year early in the 1970s and enter a UC campus, but she was an extreme outlier.

Central Falls Superintendent Frances Gallo is facing legal headwinds but is less obviously foolish than Buttars. Gallo is relying on a set of state options for addressing low-performing schools, and when teachers in the district would not agree to a longer workday without substantially increased pay, Gallo said she'd forgo bargaining a solution and use the option to fire all the teachers in the school instead (and let them reapply for their jobs). There are two general questions here on the legality of Gallo's move: is the state corrective-action structure outside the scope of bargaining for public employees in Rhode Island, and does her move constitute an unfair labor practice by construction (i.e., even if the state corrective-action structure is not bargainable, is her action retaliation in the context of the moment)? This is very far from my experience, but if this were in the state of Florida for most of the time we've had public-employee bargaining, I suspect the outcome of a similar legal battle would not be easily predictable. 

The rule of thumb with economic crises is that people innovate through desperation rather than through careful planning. We're seeing that in the case of Utah. And nerves are generally raw throughout the country, so situations that might otherwise be resolvable often head into conflict when that might have been avoided in better times (I don't know if that would have been true in Central Falls).

Books are not going to disappear from libraries

Student views of libraries are apparently all over the map when asked by the New York Times whether libraries need books anymore... but I think that's always been the case. Libraries serve multiple constituencies, and if you had phrased the question differently in different eras and for different media -- for example, "do libraries still need to stock cassette books-on-tape?" -- you'd have very different responses. Yes, public libraries still carry and loan audiobooks, though they're no longer on cassette tapes. 

Patrons like what they like, and librarians always have to figure out the right mix. Fundamentally, "books or ebooks" is the wrong question from the standpoint of library administration. It's going to be a mix of books, periodicals, audiobooks, video material, computer access, reference services, public space use, and outreach for public libraries and a different mix (but still a mix) for academic libraries.

From a public standpoint, too, "books or ebooks" is the wrong question. Funding public access to information is one of the best investments in the future I can think of. Yes, I think of libraries as part of the "constellation" of educational institutions (to borrow from Larry Cremin), and no matter how I may cringe when certain ones are used by students, it's better to nourish more sources of information than to be stingy.

February 13, 2010

The message of opening access to AP courses: "Take chances, make mistakes, get messy!"

Someone reminded me that because I was crazy-busy in the fall, I didn't comment on one local controversy a few months ago, so this is a catch-up post of sorts, in part justified by the February 4 story in USA Today and Leslie Postal's story earlier this week in the Orlando Sentinel. The St. Pete Times editorial board published two pieces on December 15 and December 16 criticizing local schools in central Florida for having low and widely varying "passing rates" on Advanced Placement exam subjects. By "passing rate," the Times is using the common threshhold score, 3 out of 5, for earning college credit. Never mind for the moment that the reasons why students take AP courses are not necessarily about college credit, nor do all colleges set 3 as the threshhold. One can assume that the wide variations in proportions earning a 3, 4, or 5 reflect an underlying variation in achievement in the classes. The editorials argue that the achievement in AP courses is too low and that variations are a direct reflection of teacher quality. In the first editorial, the board wrote,

The passing rates on the AP exam are often pathetic. It is a scandalous situation that fails students, misleads parents and wastes public money.

In the second, the board wrote,

District superintendents and school principals should hold teachers accountable for dismal passing rates.

These are conclusions from a superficial analysis. I know of situations where it is obvious that low scores are probably reflections of teacher quality, but I know of several classrooms where either students are scoring well despite low teacher quality or where students are scoring low though they have a fabulous teacher.

For example, I know of one school with two teachers in a particular AP subject taught in 12th grade. Students of both teachers have approximately equal proportions earning a 3, 4, or 5. Yet I know from talking with students that one of the teachers is engaging, providing both materials and an environment more reflective of a college class than the other, where discussion is squelched and theoretical frameworks are presented as narrowly as one could imagine (and this is in a topic where college classes would commonly revolve around discussion). That's right: two teachers, one school, different assignments in the year (how did that happen??), very different reception by students, similar AP scores on the measure the Times published and the editorial board cares about. Most obvious explanation: credit goes to these students' prior teacher(s) in the subject for getting them ready for the AP class they take in 12th grade. Did that possibility appear to the editorial board? Apparently not.

More broadly, members of the editorial board of the Times (and a number of people around the country) have the exactly wrong approach to challenging classes in high school, as evidenced by another point in the December 15 editorial: "There is some merit to the argument that passing rates are low because too many unprepared students are being steered into AP classes." This statement, which I've generally heard from middle-class or wealthy parents, assumes either a zero-sum game for schools (for some parents, that their children aren't getting enough exclusivity in AP credentials for college admissions offices) or that students in AP classes are worse off being in the AP classes than not. The latter is speculation by the editorial board without a clear research consensus (see below for a longer discussion). The former is not acceptable to me as a basis for making policy decisions.

On a philosophical basis, I am disturbed by the assumption that we always need to withhold content based on prior achievement. Why does the fictional Ms. Frizzle tell students to "take chances, make mistakes, get messy!" in elementary school, but we have to shake in our boots at the possibility that we might be challenging older students? There are plenty of classes (e.g., U.S. history) where keeping students out makes little sense, and the Times is dead wrong in their approach on this issue.

The alternative to opening up opportunity: a situation such as at Berkeley High School, where the breakup of the school into mini-schools several years ago has segregated students, left a disproportionate number of white students in AP science classes, and led to a real zero-sum game of reduced science instruction as some parents and educators propose redirecting effort away from science labs to smaller class sizes elsewhere in the school. That becomes the dynamic when science labs for advanced courses are the arena for a privileged few instead of a more common expectation that teenagers should be encouraged to take challenging classes.

Addendum: Paul Cottle wrote a blog entry this morning on the same subject, and he refers to Kristin Klopfenstein's research, including an article she published last year with Kathleen Thomas (earlier version PDF). The research is mixed; for a different view with recent research, see a 2008 Northeastern Educational Research Association paper by Xinhui Xiong, K. D. Matterny, and E. J. Shaw. I don't think that schools should respond to the mixed research either by shutting down access to challenging material or by making the teaching of AP courses a higher-risk assignment for teachers than other assignments.

February 11, 2010

Additional thoughts on performance pay politics

An addendum to my entry earlier this morning: I think that there is a politically-robust rationale for performance-pay policies, but it's not at the level of incentives usually used as the justification. The more plausible rationale for performance-pay policies is at the level of public-sector accountability: most people with jobs do not expect identical salaries or salaries based on a formula, and small variations based on something other than seniority and educational credentials might boost the facial validity of public-sector HR practices.

Note that this is not an argument that business practices are always incentives based (or should be: witness AIG as a disaster stemming from short-term incentives) or even widely varying. In some cases--large law firms, for example--entry-level professionals receive step pay increases in their first few years akin to teachers' step increases. But if I were to ask the head of the Florida Council of 100, Susan Story, whether she'd stop advocating performance pay even if the research consensus in a few years were solidly against its doing anything for student achievement, my guess is that she'd still push for some form of performance pay.

The discourse around this is somewhat similar to other comparisons people make between their lives and public policy: when policies look like you're pushing the cart and someone else paid by public funds isn't, you're less likely to maintain support for it. A friend of mine visited a newspaper columnist some years ago to complain about an article the columnist had written regarding AFDC (the federal welfare program before 1996). Don't you understand the factual errors with all of the myths about welfare? my friend asked. Sure, said the columnist, but you don't understand why public attitudes have changed: as the majority of mothers now have to find their own child-care arrangements while they're working, they're going to be far less sympathetic towards women who aren't willing to work or perceived as not willing to work.

I don't agree with the columnist's thumbnail history of public attitudes towards federal welfare policies or on assumptions that women on welfare have not historically wanted to work. But there is a significant grain of truth there that when the majority of mothers work when their children are young, and they have to find and pay for child care and wrestle with the stress involved in that, those mothers are not going to want to see that they're pushing the cart and others aren't. For similar reasons, those who oppose any performance pay have an uphill road telling people who work in environments with non-step-like pay arrangements that somehow public schools should be arranged differently.

Why the Teacher Incentive Fund and Race to the Top are long-term dead ends for merit-pay advocates

The apparent push in the proposed 2011 Obama budget for an enlarged Teacher Incentive Fund on the heels of Race to the Top makes me think that merit-pay/performance-pay advocates may be spreading their political capital very thin on teacher evaluation. Most advocates of paying teachers in part based on test scores are also advocates of using test scores in part to evaluate teachers more broadly, especially dividing probationary teachers from teachers with a right to due process before dismissal. And they're trying to do both. Smart or stupid? I think it's counterproductive for several reasons:

  1. The research on benefits of individual-teacher performance pay is limited. Very limited and quite mixed. Putting all your chips on a huge expansion of experimental performance-pay schemes? You may not get what you want, and public evaluations may doom the politics. (Think Reading First, though the allegations of corruption set the stage in that case for death-by-evaluation.)
  2. Grant programs end. If the expansion of performance-pay programs relies on temporary revenue, then the program may well die along with the extra revenue. Denver's teachers union and district worked together on a long-term political deal: performance pay that teachers helped develop tied to a long-term boost in revenue. That's not the structure of RTTT, TIF, or the Gates Foundation grants.
  3. Real-life performance-pay bonus budgets are stingy. The best example of that reality is here in Florida, where the state budget for the school-based rewards for test scores has been no greater than $100/student (for a school) since the late 1990s, and while my undergraduate students sometimes enter my classes thinking that a huge amount of school budgets are based on test scores, in reality that's no more than about 1.5-2% of per-pupil expenditures in Florida (and that's for the schools that receive the money). When this money is distributed to staff (sometime it is, sometimes it isn't), it's in the form of bonuses, not additions to base salaries. The fiscal and political reality is that the only way to permanently boost base salaries substantially based on test scores is to give the money to a tiny fraction of teachers, and that's a recipe for political disaster (and legal problems).

The last point is one I am surprised opponents of performance pay have not raised sufficiently, and here's how I thought someone would have put it by now: Okay, so you want to pay teachers well if their students learn a great deal? Wonderful. So if students perform at a very high level, you're willing to raise taxes to reward teachers for that accomplishment? Liberal advocates of performance pay would probably answer yes if. I don't think fiscal conservatives who are performance-pay advocates have thought through the dilemma on that point very clearly; either the answer is that you're willing to raise taxes or that you have low expectations for schoolchildren. 

Eventually, I suspect that advocates of performance pay will have to decide whether they want to put all of their political capital into pay schemes that are fragile or into hiring and retention issues. The proposed ballooning of TIF is a sign that no one in Washington is thinking about the political balance of these issues in the long run. 

Disclosure: I'm a member of a higher ed union that has long had a contract with merit pay and considerable differences in pay by rank and discipline. K-12 is a very different world in this regard. 

Note: I started this entry on Tuesday, and because I forgot to change the "publish date" (which Movable Type usually sets at the time you started an entry, not published it), it first appeared as if it were published Tuesday. My editing fault, not your faulty memory. Now, your forgetting to read all of my books and articles? That's a different story.

February 6, 2010

Another stupid article on "the dating scene" in college

Some of the clues that the latest article on the "dating scence" in colleges with 60% female enrollment was written by a reporter with an axe to grind and a preset angle at which to grind:

  • The featured photograph from a university with 60% female enrollment (a) is of college seniors (or I hope they're seniors) in a bar, (b) is of an all-white group of students, (c) has six women and one man, (d) has no older students.
  • Every photograph features white students.
  • All the women interviewed for the story appear to be members of sororities.
  • One of the interviewees is a former student who happens to be hanging out in a bar near campus. (So why is he representative? Why didn't the reporter step a few minutes away from a bar?)
  • The focus is entirely at a flagship public university.
  • There are no older students interviewed for the story.

Since the primary world of colleges is at the regional state university and community-college level, maybe we should skip the flagship campuses and look at the statistics of an institution such as Miami-Dade College. MDC has more than 150,000 students enrolled, and while 60% of them are women, only about 35% are right out of high school (under 21). About two-thirds are attending MDC on a part-time basis, and while MDC is now a four-year institution, I don't think there are any dorms, so every one of those students are commuters and live somewhere in the Miami area. In other words, the dating scene for straight, gay, or bisexual students is where they live as well as on campus. That's the reality for the majority of college students in the United States, not the preppy picture that the New York Times reporter and photographer portrayed.

But if you want to look at residential colleges and universities, maybe a little reality should intrude: the average age at civil marriage for women in the United States has moved back up to the mid-20s, where it has been historically for well over a century, with the exception of the immediate postwar years. College students' meeting and marrying in college is common enough but not dominant. 

And the history of colleges is not one filled with demographic "balance" in some hypothetical way. For many years, the ranks of elite residential institutions were filled with single-sex colleges and universities with single-sex undergraduate colleges, and the students in those colleges and universities had to go off-campus for a hetereosexual dating scene. And in the first decade after World War II, the GI Bill pushed enrollment in public universities in the other direction, towards majority male enrollment. If you can find more than a decade or two when the dominant demographic profiles of residential colleges, community colleges, and public universities were all fairly evenly split by gender, I'd be surprised. My guess is that maybe a decade or two will fit with the peak of the Baby Boom through the mid-1980s... when people worried about the social consequences of the sexual revolution. As one of Gilda Radner's characters would say, if it's not one thing, it's another... so let's stop obsessing with the on-campus dating opportunities of college students.

February 1, 2010

Sloppy journo skewered; readers await fix

Reporting is a hard job. These days, reporters are being asked to cover more subjects in less time with an even smaller news hole for newspapers that are losing money, laying off colleagues, and may be out of business within a matter of months. Even in good times, reporters knew that errors were going to be read by thousands of subscribers and that even if they worked twice as many hours in a day (usually impossible), they'd never catch all factual goofs or grammatical mistakes, or never quote enough interviewees to satisfy all readers. Great beat reporters are inherently improv artists.

Having said that, I know it should not be too much of a surprise that even reporters with solid reputations such as Ed Week's Debra Viadero sometimes get caught taking shortcuts. Thus far, no response from Viadero, but it's another part of journalism (and a reflection of the craft) to print corrections publicly. So let's wait and see how Ed Week acknowledges error.

Grading the "Grades" reports

I'm back from Toronto today--had a great time talking with Canadian faculty, had my head chewed off in a thoroughly polite, Canadian way for one bone-headed error I made in discussion, survived subzero temperatures for a few mornings, and completely failed to enter the Hockey Hall of Fame building--and back in Florida the temperatures are a bit lower-than-average for this time of year but discussion of the Ed Week Quality Counts "grades" given Florida is apparently heating up. So maybe I need to revisit my idea from last summer of grading the grading reports. Last June, I pointed out that professional grading practices generally provide scoring criteria in advance, so that those who are being evaluated will have a chance to... you know... meet the standards. Let me list all of the facets on which I think one can grade such "grade reports" of states and the like:

  • Purpose. Is there a clear public rationale for issuing such a report? How broad or narrow is the public purpose?
  • Scoring criteriaDescribed in June.
  • Description of sources and analysis. How systematic is the collection of source material (as opposed to anecdotal or convenience sampling)? Is there a clear chain described from collection of data to the application of labels? Is there a discussion of relevant caveats/alternatives?
  • Robustness of sources. Are the sources publicly verifiable or replicable? Are they subject to gaming, falsifiability, or manipulation?
  • Relevance of sources. Is the material relevant to the criteria, and does the "grade report" use the most relevant obtainable information? Is the source information analyzed appropriately to warrant the application of the grade labels?
  • Sponsorship. Are funding sources and potential related interests stated clearly? Is there a separation between the real or likely perceived material interests of sponsors, on the one hand, and editorial control of the project?

It strikes me on impression that different types of periodic "grading" exercises have different types of weaknesses. An advocacy organization whose reports rely on anecdotal evidence and give higher grades to states that are more extreme towards its position might receive lower grades on description of sources and analyses and sponsorship than in other categories. A news organization that makes millions of dollars by selling a volume ranking colleges and universities using reputational surveys of institution heads and data on institutional wealth is likely to receive low grades on public purpose, robustness of sources, and relevance of sources. A news organization that ranks states on categories that change every year using no apparent criteria that also change every year is likely to receive its lowest grades in the area of scoring criteria and description of sources and analyses.

As a faculty member who has assigned thousands of grades to students, where the grades affect student progress towards degrees and financial-aid eligibility, I know from experience that the process of grading is imperfect and in my field depends on judgment rather than objective cut-and-dried methods. That's why I state criteria as early as I can, display model work from prior semesters if possible (with the permission of their creators), answer questions about assignments, look at drafts, structure revision opportunities into a number of courses, and always let students correct me when they document that I have recorded individual assignment grades incorrectly.  I know from student complaints about grading in general that they hate being judged on criteria they feel the evaluator keeps secret, or that is designed to make the evaluator look good, or that serves some other purpose that isn't for the general purposes grading is accepted by at least some to serve. In other words, if you're going to assign grades, especially if the clear intent is to shame certain entities into changing, you need to take at least a few minutes of care to address common-sense ethical expectations. I'd have far more patience with these publicity-seeking exercises if there were more care evident in the process.

January 27, 2010

Why the "college hunt" genre is unrepresentative, and the shame of the College Board Profile

This morning's blog entry by Valerie Strauss is typical of the genre: a perspective on what it's like to apply to a number of selective colleges and universities and hunt for financial aid. And it's all wrong, both from a policy perspective and (I'd argue) even a hypercompetitive parents' perspective.

Policy perspective: the colleges most students attend are not very selective. Even for the ones that don't accept all applicants, most accept the majority of applicants (including most public universities). And even in the world of "very" selective institutions, you might be surprised. Sure, both Harvard and Stanford will reject more than 90% of their applicants this year, but most of the "very" selective private liberal arts colleges accept 25% or more of applicants... and we're at the peak of the baby boom echo, so it's only going to head up from here. (Math problem: If you're a high school senior and apply to colleges where you have a 50% probability of being accepted, and the decisions of each college are random and independent, how many do you need to apply to to have at least a 98% chance of being accepted into at least one?)

So the problem is generally not getting accepted into one college but being able to pay for it and being able to take all the classes you need and succeed at them. My daughter is applying to a few places where the tuition/board combination is high enough where some institutional aid would be very nice, and last night we completed the FAFSA, which is one half of the financial-aid paperwork for one of her desirable colleges. (I'll have more to say about the other half later.) The administration's promise on a simplified FAFSA has been fulfilled, at least from my experience: you don't need a CPA to fill it out, especially for families who are eligible for Pell grants and state assistance. The administration's proposal for a 10% cap on income you owe on college loans would be another step, and a definite improvement on the new income-based repayment option. Given the gap between Pell grants and tuition at a number of public universities, pushing on income-based repayment may be more valuable in the long run than expanding Pell grants.

Where Strauss is correct from a public perspective is the gap between the time high school counselors can spend shepherding students through the admissions process and the reality of the need. I'm thinking here primarily of high school students who would be first-generation college students. There aren't too many guidelines for a ninth-grader to keep in mind, but they're probably not repeated often enough: get your act together now to make sure your first semester grades are at least a mix of Cs and Bs, and they need to head up from there; read more than what's required; go as far in math as you can; take SATs or ACTs in your junior year; tell your parents to put their financial information in one place starting early fall of senior year; expand your college possibilities in one dimension from what you're being told by those around you. I suppose there are others that high school counselors use, but for the barebones, students whose parents never attended college can get into a fine public university following this.

If there's something that worries me apart from the high school curriculum and funding for poor students, it's the narrow way most high school students think about where and how to look for colleges, and the way that adults encourage that narrowness in part from their experiences or perceptions or because of tacit knowledge. There are sometimes circumstances that restrict students--those who need state assistance will be staying in-state, and often first-generation college students (especially young women) live at home while attending classes at a public university, at least for a year or two. (I know of one very large community college where faculty get the benefit of teaching incredibly talented first-generation students because their parents wouldn't let the students move away for a few years.) High school students can be creative in working with family preferences--Orlando high school students often prefer the University of South Florida (here in the Tampa area) and Tampa area students often prefer the University of Central Florida (Orlando) as a "far enough away from home so I'm not visited by my mom twice a week, but close enough to drive home on weekends" solution. But that's like chain migration: if you hear about an option from someone you know, you can use it.

What about the options you don't personally know? I've had some conversations with teenagers and parents in the past year or two where presumptions have become stereotypes and blinders. One parent completely dismissed a nationally-known public liberal-arts college because she knew some students with learning disabilities who saw that as a friendly place to attend... so it must not be good enough (i.e., prestigious). A student who is one of the most hard-working teenagers I have ever known and interested in engineering schools didn't know the difference between tuition-dependent private schools and those with endowments and substantial institutional aid. She was thinking very hopefully on an engineering school within driving distance that is tuition-dependent and where there was no way that she could get aid (and thus attend). She hadn't thought of CalTech at all, though it's well off and where she might get a boost because of the dominance of men in their undergraduate enrollment. Another student who moved to the U.S. four years ago was disappointed in her board scores and thought colleges wouldn't want her. She's another incredibly hard-working student, one who admissions officers would drool over in reality. For the students in these cases, I'm not worried because it didn't take much to persuade them or their parents to think a bit more broadly (and optimistically). For the millions of talented high school students I can't persuade personally to think a little more broadly about colleges, I worry about the mental shortcuts we take when looking for colleges. It's an understandable but sad statement about our country when some of the most effective recruitment of college students is done through Saturday television broadcasts in the fall.

Private perspective: As I wrote above, the FAFSA is one of the pieces for institutional aid for a college my daughter is keenly interested in. The other is the College Board Profile. Last night, I printed out their 19-page worksheet and filled in answers for the several-hundred questions about parental income and assets so my daughter can enter the data this afternoon. I'll just say this to the admissions officers for the private institutions using the College Board Profile: you've just demonstrated to me why your efforts at recruiting a diverse population of students is often a facade. When your chosen tool (which you don't have to pay for) is several orders of magnitude more difficult to complete than the old, more complicated FAFSA, it's clear that you don't have a clue about how to get poor students to apply for financial aid. And College Board? Shame on you for requiring poor families to pay for the privilege of having one more barrier to receiving financial aid.

My daughter will do fine, and unlike other college seniors, she hasn't panicked. Several years ago, when it was clear she was interested in Type X college, her mother and I talked about the financial feasibility of that. (I'm a public-university professor in a relatively low-paid field. Well-off? Definitely with respect to human history! Able to send my daughter to Type X college on my and my wife's income alone? .... uh, what type of cat food tastes good?) We figured we could expand her horizons, but given that her spine is stiffer than mine, I expected it would be in one direction.  Let's see: ask her to consider Type Y college? Not going to happen. Z? Not a chance. Type X-public? Hmmn... that worked. In the fall of her sophomore year, I told her that if she could find a Type X college that would let her visit classes, either public or private, I'd take her. And she found such a place, so we went. As a result, we spread out college visits over a few years, not a few weeks. That first college is still on her "very interested" list, and overall she liked (and applied to) roughly half of the places we visited, most of which were Type X colleges. Her interests have changed a bit, but she'll do fine in any of the places she's applying to, and it's her life, not mine. Yes, she's been accepted to at least one. As I stated above, if you've worked hard in high school and you're not set on getting into the One True Place for You, you'll get in somewhere you can learn a great deal in.

January 24, 2010

Florida legislative session education preview

Former Miami Herald reporter and current free-lancer Gary Fineout has a solid legislative session preview on education policy in this morning's Sarasota Herald-Tribune (hat tip). I may disagree with his predictions on the margins, but on the whole I think he's on the money in identifying the obvious issues. Fineout was starting his analysis from the Florida Chamber of Commerce report released a few weeks ago, which had a combination of noncontroversial suggestions as well as a few ideological throwaways (such as the resurrection of the failing-schools voucher program). Fineout is probably correct that budget woes will kill or maim any suggestion with a large price tag (though I would love the suggested large boost to higher education). So let's divide the policy ideas into the noncontroversial and the controversial and then the elephant in the room.

Noncontroversial: end of course (EOC) exams, especially since Rep. John Legg said his bill would have non-biology science EOC exams as non-high-stakes tests. I've been watching that issue in semi-despair for several months after the U.S. Department of Education confirmed the Florida DOE's view that Race to the Top grants could not be used for assessment development. Legg's promise is a good compromise, if it happens as he stated.

Noncontroversial: continuing to use federal stimulus dollars to boost local district budgets. The decline of property-tax collections is the giant sword hanging over schools this year, and the balance of state-local K-12 funding is one of the giant budget issues this year along with Medicaid and the lack of any trust funds to raid for 2010-11. 

Controversial: modifying the constitutional class-size mandate. There might be a compromise here involving statutory changes to the implementing laws. Legislative leaders might have to choose between spending political capital on this issue and on the next two.

Controversial: legislative attempts to end K-12 teacher tenure. The legislature has mandated the end of tenure before, sort of like the way the legislature has mandated merit pay before (next issue below). If the legislature overplays its hand, an extreme bill might turn out to be a short-term nightmare for teachers and a long-term Pyrrhic victory for tenure critics. I can think of at least two ways that FEA can fight more extreme laws in the long term with reasonable chances of winning.

Controversial: merit pay, or rather legislatively-mandated mechanisms. This would be the fourth or fifth go-round on this issue in the past decade in terms of state mandates. Someday there will be a set of legislative leaders who want to work a deal with the FEA on performance pay at a time when FEA leadership is interested in a deal; until then, the heads will continue to butt. (For Mike Antonucci and other union critics, you need to work harder to understand how a teachers union in a state with weak collective bargaining laws can successfully resist state-level mandates when the political branches are often actively hostile to the state affiliate; your usual explanations flounder in Florida.) 

The elephant in the room: money. Legislative leaders seem disinclined at the moment to do anything that could be called raising taxes. While state revenue collection appears to be on a slight upward trend, that is more than counterbalanced by a decline in tax collection at the county level and increasing demands for Medicaid. Last year's budget politics was set by two contexts: growing legislative disillusionment with Charlie Crist and the chaotic aftermath of Ray Sansom's speakership on the House budget committee. This year, Crist is a lame duck who is viewed in the legislature as somewhere between a powerful fool and an opportunistic sell-out, and that's within his own party. Speaker Larry Cretul has reset his caucus's leadership according to his own preferences, which now include one rather than two budget chairs. And Senate President Jeff Atwater may be inclined to burnish his conservative fiscal credentials for his political future. As a result, Senate budget chief J.D. Alexander will have several quanta less influence and will probably be picking one or two battles on large issues. I have no idea whether this presages a slow-moving train wreck on the budget or opportunities for quick-thinkers in April. But a budget wreck on the scale of Pennsylvania (if not California) is possible.

January 16, 2010

Weingarten, teacher evaluation, and the long haul

I started this entry Tuesday, before the Haiti earthquake, thinking about long-term policy changes. There are a number of students and staff at USF, as well as Tampa-area residents, who are worried about relatives or who are now in mourning. Sometimes the long term pauses while you take care of immediate needs. Well, it doesn't really pause, but most people have a limit on how many things they can focus on at one time. We can multitask, but not hypertask.

As I hope most of us have become aware, our woes are often small in comparison with world events. I've had a few bumps in the road this month, one quite literal: the driver of the gray van behind me Monday morning didn't stop when I did for a yellow light. Thankfully, crumple zones, air bags, seat belts, headrests, and other bits of technology did their job, and two uniformed officers of the Tampa police guided traffic around the immobile vehicles until the wrecker could take them away. My laptop, sitting in the trunk at the time, appears unhurt. I'm unhurt; or, rather, if you tell me I need to see a doctor about my head, I'll tell you to stand in line, and the previous suggestions have not been after an accident. Surviving rear-enders without a scratch makes me grateful for government regulation, for technology, and for people such as the two witnesses who helped steer traffic until the police arrived. After news of the earthquake, I think I can handle the small bumps in life. Look around you and you realize you can and often should suck it up.

One thing I haven't been able to do this week is look closely at the Randi Weingarten speech or much reaction to it. There's been a semi-understandable "hey, she's given a Good Speech before; where are our flying cars?" reaction. But for those who are jaded by a speech, I'd agree a little more if I didn't see so much immediate score-keeping kept about who won on which issue in which city. You either care about and focus on long-term structural changes, or you don't. We're in the middle of an era in which many policymakers believe that a few derived measures from tests are good enough for high-stakes decisions and extending that to personnel decisions. There are going to be districts that make disastrous decisions on how to use student outcome data, in different directions, and districts where both the structure and the practice is uses information appropriately (and yes, does use the information). For the short term, I care a great deal about the disasters. For the long term, I know they'll exist and hope there's enough nudging of things in the right direction. For that long term, Weingarten's speech is right and consistent with AFT national support of local bargaining.

For those who keep scorecards, the battle over the Detroit collective bargaining agreement is important for counting coup. To those who think about the human impact of change, you have to worry about the attempted (and possibly successful) coup inside the Detroit Federation of Teachers. (To those who thought the new Detroit contract was too little, too late, I told you so on the internal union politics.) For those who focus on the long term, Detroit is a blip either way, losing students consistently over the years and one district out of hundreds in Michigan. Quick question: how many so-called "suburban" school districts have more students than Detroit's?

If there is a big picture on teacher personnel issues, there are several issues to pay attention to:

  • Teacher preparation and professional development evaluation. I think Louisiana's approach to evaluating preparatory programs is about at the right scale: uses test data cautiously, and I think appropriately at the program rather than graduate (i.e., teacher) level. Florida is starting something that it claims is similar, but it's on a jerry-built measure (or, rather, Gerry-built measure since I know the person who is at least partly responsible for the measures of growth used here), and because it's incautiously done, I suspect it'll take several years to straighten out the kinds. I am relatively optimistic here.
  • Teacher preparation and professional development structures/curriculum. Here, Arne Duncan, Arthur Levine, NCATE, and TFA/other alt.-entry routes are going to push things in one productive and one disastrous direction. The productive direction may be more time earlier in classrooms with appropriate (scaffolded) support. The unproductive direction is the denigration of psychology and other disciplinary knowledge as "theory." Incidentally, that denigration has been a common pattern within schools and colleges of education for the past 30 years--the same people who are being criticized for their ineffectiveness. My college of education is a sample of 1, but it's the educational psychologists on my floor who are the ones most adamant in the college that there is no research support for learning styles (and Michelle Rhee's district that requires teachers to use something it calls learning styles). And as far as sociologists and historians of education controlling teacher education programs? Ha! Please point to one. If there is pushing of social theory inside programs, my firm prediction (which can be empirically tested!) is that there is either no relationship between respect of disciplinary-based faculty and puffery in the teacher-ed curriculum or a negative relationship. I am cynical here.
  • Teacher preparation and English language learners. There are major problems here. Unfortunately, teaching teachers about the history of immigration is both necessary and insufficient, but the social history (or a watered-down version of it repeated ad nauseam) tends to be the focus of many professional development structures that attempt to address ELL problems. Linguistic psychology takes a back seat, and there is too little research on both methods and appropriate assessment. I am in despair right now on this area.
  • It's the baby-boom echo, stupid.  All the cries about teacher shortages with the retirement of baby boomers is ignoring the baby-boom echo, the peak of which is right now passing through college. In a few short years, they'll be the bulge of early workforce participants, and you won't need a high proportion of them to be teachers to fill the empty seats. Oh, yeah, and there are the people in their late 30s and early 40s who can also do so. Apart from spot needs by geography and specialization (esp. science and special education teachers), I don't think that there is going to be a significant teacher shortage. I am optimistic.
  • The mix of evaluation sources. I've written about this before: we have no clue as a society how to mix different sources of evaluating teachers together when each source is incomplete and sometimes severely flawed. For ideological reasons, there are advocates of different varieties of sticking one's head in the sand, either ignoring student outcomes or treating them as infinitely-accurate and -valid measures. The major Gates initiative here might be an oasis or buffer of experimentation in the RTTT era. I am cynical but hope to see something of value, eventually, maybe, filtered through a lot of political spin.
  • Incentives vs. protocols. As John Thompson has pointed out, Atul Gawande's advocacy of protocols (checklists) is an uphill battle in some areas of health care.We see similar resistance in education, sometimes for good reasons (there are some awful protocols in education) and sometimes for bad reasons (see Lisa Delpit's discussion in Other People's Children on the disingenuous criticism of DISTAR for alleged abuse of power relationships). But few have pointed out that there is a conflict between the advocacy of incentives, which assumes that teachers can deliberately choose to act in a way that increases test scores, and the pushing of protocols, which assumes that no matter how well-trained and professional, teachers could use reminders to act in a way that increases student achievement.

January 9, 2010

Spot temperature:Climate::Test score:____________

I fully expect that within a week (if not yet already) some climate-change skeptic will use the cold wave currently freezing much of the country as an argument that climate changing really isn't happening. And every time there's a vicious cold snap in winter or a cooler-than-average summer we get the argument. And some reporter and editor decides to devote part of the ever-shrinking news hole to bad coverage of the issue, while a relative handful of reporters use the question as an opportunity to educate readers about the difference between weather and climate.

Today, I'm sitting in central Florida with more layers on than I usually need in early January. It's colder weather than usual. But we're in a warming climate, because in the long run of decades (or centuries) the current cold wave is just noise, and the trend is towards a warmer atmosphere. "Just noise," you may be thinking through chattering teeth, "tell my heating bill that it's just noise." The current cold wave is nasty for individuals today (and a few days more), but it's temporary.

The variability of weather makes sense to most people because we have enough experience to distinguish between spot temperatures and broader patterns. We know that temperatures have daily and seasonal cycles. But the cyclical nature of weather does not give us enough background to grasp climate change. For that, you need data. A lot of data. A lot of data from a lot of places and times, of different sorts, with a number of experts sifting through it.

And even then you get climate-change conspiracy theorists, including someone who's evidently a hacker.

You can probably guess the logical analogue here: we do not have anywhere near the same density of data on student achievement that we have on climate, and yet we draw bold conclusions about the underlying achievement from a relative paucity of noisy data. As I wrote in August, we need to learn how to make decisions with noisy data. But in terms of broad trends in achievement, it is a bad habit of Americans to equate the latest test scores with long trends. 

And that doesn't even touch the question of whether test scores are like temperature readings. Ah, but they are, if you're talking about your and my outside thermometers: placed at different heights, in different conditions (sheltered, out in the open, shade v. sun), different ages of the thermometers (and thus consistency of the readings across the years). I am sure that background thermometers in these varied conditions are highly correlated in the sense that when it's colder, they're all colder, and when it's warmer, they're all warmer, and so the correlations across time are likely to be very high. But I wouldn't use them in any scientific research.

Stay warm, and have whatever hot beverage you like!

January 1, 2010

Wannabe education reformers in the U.S. need to use English

Confession: I do not have a professional editor review these blog entries before they become publicly available. As a result, there is the odd grammatical error that I notice only after publication.

And yet, I do not abuse the English language deliberately. In contrast, one of the least attractive stylistic tendencies of wannabe reformers, reformists, reformistas, or whatever term you wish to use, is the blatant word abuse, and unfortunately we see that in Tom Vander Ark's blog entry December 26, which had impact and leverage (ab)used as transitive verbs. They are not quite as chalkboard-scraping as incent (which I have heard and read from Arne Duncan and Mike Petrilli), because they do exist as nouns (and impact does not hurt my inner ear when used as an intransitive verb). But good grief, friends: do not add business jargon monoxide to the conversation, or you have no ... hmmn... leverage with which to criticize others for the same sin.

December 31, 2009

Education stories of 2009 (U.S.)

The end of the year is the traditional time for journalists and laypeople to look back and identify major issues in a year. As Phil Graham (or maybe Ben Bradlee) said, journalism is a first rough draft of history, and you know what a first rough draft looks like. Nonetheless, as an historian I'll take a stab at what I think will be seen in retrospect as key developments in education in the U.S. They may even have been key issues this year!

  1. The Great Recession and students' lives. More children are homeless, hungry, or displaced in some way because the adults in their family have lost jobs or their homes. We won't know the exact extent of the effects on children's lives for a few years, but the news stories of the recession's effects on children are first indicators of a quantum leap in child poverty. And there is also an effect on the lives of college students, though the effects are more complicated. People are returning to school at a rapid clip, but because financial resources are lower, there is also a greater demand for financial aid at college.
  2. The Great Recession and the education stimulus packages (plural). In late 2008 it became obvious that for several years Florida had been leading the country again... in declining state and local revenues. Around the country in early 2009, school-system budgets for 2009-10 looked like they were going to collapse, resulting in catastrophic layoffs that would affect not only schools but the whole economy. Federal spending kept hundreds of school systems afloat and is a good part of what saved the economy from a much worse decline in aggregate demand. The early-2009 stimulus package (aka ARRA) is the major part of the story but not all of it. If you didn't hear about the mid-December shifting of $23 billion from TARP into an account school systems could use to save jobs, you missed a substantial increase in the stimulus that should be considered part of December's second stimulus, along with an extension of unemployment benefits and federal subsidies for COBRA payments.
  3. College financial aid reform. The Obama administration is combining administrative changes to simplify the Free Application for Federal Student Aid (FAFSA) with a push to eliminate the federally-subsidized private lending program and shift resources into direct lending. While it is not politically possible (and probably not legally possible) in many states to require that all students complete the FAFSA, it is possible to make it much easier to complete, encouraging more students with real financial need to take advantage of financial aid.
  4. The growing role of community colleges... and erstwhile or soon-to-be-erstwhile community colleges.The July plan to give $12 billion to community colleges is a relatively small part of the overall policy emphasis of the administration on community colleges, from the appointment of a community-college president as the chief administrator of higher education policies to the greater scrutiny of proprietary training institutions (where do you think students who would otherwise go to proprietary job-training programs will be headed instead?). Ironically, two of the largest states are headed in a different direction, with Florida's community-college system disintegrating or morphing into a "state college" sort-of-system, and some voices in California voicing a similar idea with new caps on Cal State enrollments. (DC is headed in the other direction, with UDC splitting into two- and four-year institutions.)
  5. Race to the Top. Some of you may wonder why this isn't #1, but I'll defend my judgment that it's important but whether you like RTTT or not, it's not nearly as important a change as the issues I've put above this. But don't fret if you disagree: see #8.
  6. Common core standards effort. The halting, awkward, adolescent-like steps towards creating at least some vague national-level standards developed, and while Alaska and Texas may not be involved, and other states may opt out later, this is the curriculum equivalent of the 1989 Charlottesville summit, in that it is a national rather than a federal effort.  (See Maris Vinovskis's recent book for that story.)
  7. City school control battles. From the renewal of mayoral control in NYC (and Bloomberg's relection) to an emergency manager in Detroit and the apparent devolution of Los Angeles Unified, governance is once again front and center in urban school politics. Well, maybe it never left as an issue, which is a cynical historian's perspective. But if you think I'm cynical, wait until Diane Ravitch's new book comes out in a few months. No, I haven't read the manuscript. But you don't have to before you can take a good guess at what Ravitch will say about New York City. (Recent developments in Detroit and Los Angeles came after she must have submitted her manuscript.)
  8. Teacher evaluation in local bargaining. Collective bargaining agreements put the AFT in the center of teacher evaluation debates through its support of new arrangements in New Haven, St. Louis, and even Detroit. And both teacher evaluations and collective bargaining more generally are at the heart of disagreements between the Minnesota and Florida teachers union state affiliates, on the one hand, and state departments that would like teacher union signoffs on RTTT applications, on the other. Disclosure: I am a member of the Florida Education Association and was on the governance board for a two-year term that ended this past summer. I haven't had time to learn much more than what's available publicly on the Florida disagreement, but I'll give you one idea in the back of my mind that's also in yesterday's Ed Week story (requires subscription) by Stephen Sawchuk: both affiliates are merged (i.e., in both the NEA and AFT).
  9. Sexting as a news topic. This is the latest object of our perennial concern about youth behavior, made  highly visible with the suicides this year by Jesse Logan and Hope Witsell. The main difference between teens' sending racy photos of themselves by cell and other foolish teenage behavior is that cell-phone technology enables a social chain-reaction from an MMSed photo that other (and more fundamentally stupid/dangerous) behavior does not. Not that any of these is a good choice, but if you knew that your teenager was either going to get addicted to a drug, become pregnant/impregnate someone, or send or receive a sext message, which would be the least inherently dangerous behavior?  Fortunately, Mike Petrilli is correct about the state of American teenagers: the trends on seriously dangerous adolescent behavior is headed in the right direction... not that any reporters covering the sexting issue noted that fact.
  10. Textbook affordability. Arnold Schwartznegger's midyear ramblings about ebooks aside, there has been movement in several areas to address the rent-seeking behavior of both textbook publishers and college bookstores. This includes public and private ventures to create online textbooks with inexpensive print-on-demand options and textbook rentals, and Florida is probably not going to be the last state where public colleges and universities need to list textbooks for all courses at least a few weeks before a term starts, to allow competition. There are some logistical problems with the last, such as with brand-new courses or new sections opened up to serve demand, but some tweaking will probably result in an institutionalized arrangement allowing students to search for books they can find anywhere.

So what have I missed? Any errors in judgment on the ordering? What do you think the issues for 2010 will be? Time to kibitz!

December 21, 2009

I agree with Paul Cottle: set a date for science

In response to Florida Commissioner Eric Smith's weekend op-ed in the Tallahassee Democrat (responding to the December 13 op-ed by FSU physics professor Paul Cottle), Cottle says, roughly, okay, maybe not this year, but set a firm date for setting up EOC exams in all central science areas. I'll go further: Florida needs to set a firm date for increasing the lab-based courses required for a standard diploma, including chemistry, physics, and an earth/space science as well as biology. Even if it's for the class of 2015 or 2016, we need a deadline.

There are several reasons to do so:

  • The challenges of this century require a citizenry that has a much better understanding of science. I don't care if you don't work in a field related to science; if you vote, I want you to understand some basics of science. So do my children. So should you.
  • Minimum requirements can reduce the inequalities of course enrollments. Gender differences in high school math and science enrollments shrank dramatically between the early 1980s and this decade. Part of the change was the fact that many states, especially larger states such as California and Texas, increased their core-academic graduation requirements. Right now, there are dramatic inequalities in who takes advanced science courses. My 17-year-old daughter and most of her friends who are seniors are taking a second year of some science (in her case, physics), partly because of their interests but also because her school offers those courses. Set the requirements higher, and school districts will have to figure out how to make more seats available in science classes.
  • Minimum requirements set the floor for what the next generation of elementary teachers knows. About a century ago, W.E.B. DuBois argued that African American activists had to care about the academic training of future teachers, and he is still right, and his point is correct for teachers of all students. Except that while DuBois talked about the "Talented Tenth" in the early 20th century because he knew that teaching was the most attractive profession for college-educated African Americans, we can't assume that any more. We no longer have a world where teaching is the best professional opportunity for women or for all members of marginalized cultures and races. That's a tremendous advance for the basic fairness of our society, and that means that the realistic pool of teachers is comprised of all adults with baccalaureate degrees. So we need to think about the pool of elementary teachers coming from the Talented Third or the Talented Half (which is a statement of relative attainment, not inherent ability). Want to attract new teachers from elite colleges? Go right ahead, but that still won't get you more than a fraction of the likely set of teachers in the future. Want to increase the content expertise of teachers? Great, but for the most part those requirements will focus on subject specialists in secondary schools, not elementary-school teachers. The majority of elementary-school teachers will still come from public university graduates, and many of them will have had their second-to-last lab science course in high school. (In many universities, students can satisfy the general-education requirements in science with one class.) The central question here is how much lab science do you think elementary-school teachers should have experienced? 
  • Technological breakthroughs 10-25 years from today will be crafted by the hands of those in high school now and in the near future. The real work of science and engineering research stands on the shoulders of senior researchers in any field and also relatively new graduates from college (whether grad students at a university or new employees in the private sector). Much of bench sciences these days is a team enterprise, not the work of a brilliant faculty member working alone. While this is not as persuasive an argument for me as the ones above, because only a limited number of high school graduates become members of such laboratories, it's a good thing to have a larger pool of people who are qualified to think about this work.

So I agree with Paul Cottle: the state legislature should look at its dance card and put down science for the near future. It doesn't have to be the first dance coming up, but it should be listed there somewhere.

December 13, 2009

Turnaround or abandonment in NYC?

The extent of school closings in New York City is becoming evident, and after JD2718's posts on the subject over the past half-week, UFT's Leo Casey provides an overview and alleges an ulterior motive (to create available space for other purposes, not to improve education).

I'm far from NYC and can't speak from close knowledge of the city schools, and I'm still grading student work so I have no time to read extensively. But this is an important story and rolling conflict, and there are a few predictions I'll hazard:

  • At least one conservative will commit rampant inconsistency by simultaneously (or nearly simultaneously) weeping over the demise of the DC voucher program and applaud Klein for his bold moves, repeating the double standard on the issue I have described before
  • A small handful of schools may be preserved through fairly heroic efforts, but most of the closures will stand.
  • There will be no effective way to hold Tweed responsible for consistency and rationality in its school opening/closing decisions. 

In truth, many administrators engage in maneuvers that appear as arbitrary as Klein's closures do, but rarely is it on such a scale or so visible beyond the locality.

December 9, 2009

Online grandiosity failed, so get back to work

So U21 Global looks like it's failing, after the dumping of U of I Global and the morphing of Western Governors from the "we're going to conquer the world through online enrollments" stage into the "we'll settle for 10,000 students based on a Netflix model of tuition with half of our students in teacher ed" stage.

This is not the death of online education, which exists at virtually every institution of some size. Nor is it the death of scaled-up online education, since there are several outfits, notably the K-12 Florida Virtual School, which appear to have done just fine at a large size. So what's made the difference between the thriving programs and the dying programs?

  • Thriving programs serve specific purposes. Florida Virtual School is not trying to conquer the world. It addresses a few specific needs, notably providing catchup classes, a few basic requirements that many students would like to "get out of the way" to take other classes they prefer face-to-face, and some opportunities unavailable in smaller districts. The fact that thousands of students in Florida find those valuable is related to the size of Florida, not a lack of specific planning on the part of the Florida Virtual School's administration.
  • Thriving programs have stable (and nurtured) feeder relationships. An online program within a university can develop constituencies much more easily than Vague Global Program, and the Florida Virtual School has cultivated or taken advantage of a number of ways that students find out about its strengths (as far as I can tell, from other students and from counselors).
  • Thriving programs have staff and teachers in a relationship modeled on bricks-and-mortar schooling. Florida Virtual School has made a point of explaining its acceptability in part because it has a dedicated staff and faculty "just like" the local public school down the street. As far as I can tell, thriving online programs within universities tend to treat faculty teaching online like other faculty, largely because they are faculty in regular departments and because the hiring patterns for full-time faculty normatively follow departmental patterns. How many ads in the Chronicle have job positions in an "online" department as opposed to a position in anthropology, economics, marketing, etc.?

What appears to have died is online grandiosity, and that's a good thing.

December 7, 2009

"The gap is gone"

If Aaron Pallas's report is correct, and Roland Fryer did tell Anderson Cooper bluntly in reference to the Harlem Children's Zone and Promise Academy, "The gap is gone," Fryer committed an understandable but all too common sin of education reformers across the centuries: overpromising. I've been in the room as one or more program directors and the like have promised the sky, the moon, and a few thousand stars to stakeholders and potential funders. Every time it's happened I've winced, because I've seen the storyline play out many times before: do something good, overpromise, and then see the program never be able to fulfill the more grandiose claims.

To me as an education historian, this is not an issue of whether we're adjusting for social class and other variables. Nor is it whether Geoffrey Canada is a good person (go read Paul Tough's book if you doubt that). Or whether Canada himself is overpromising: "it's worth about an hour of celebration" is his comment about the test score reports. It's about a persistent dynamic in education reform of being so desperate for something that works that you see more than is there.

I don't get that sense from Canada, who strikes me as driven and gritty and tied to what is happening to the kids in the area he's working. I'm worried about the talk around Canada and the HCZ, of taking Fryer and Dobbie's recent paper on the Promise Academy (which strikes me as fine work, but just one paper) and seeing that one paper as definitive. I've read Paul Tough's work (assigned it to my summer class), and I want HCZ to do everything Canada wants it to.

But I also want someone to look at it judiciously. And here's the irony: while it's common for a program head to be enthusiastic and a professional evaluator to be jaundiced, what is clear in the 60 minutes segment (and everything else I've read about Canada) is that the roles are reversed here. Canada's driven enough to be skeptical, to have changed school and program leaders when he doesn't see the progress he wants. Fryer? Well, check the CBS video of the segment between minute 10 and minute 11 (while watching the whole 14-minute segment). He said "the gap is gone" as baldly as Aaron Pallas claimed.

Yes, you're hearing me wince.

December 5, 2009

Are central Florida schools flouting Florida law limiting test-prep?

I have heard from teachers and students in three area districts (Hillsborough, Pinellas, and Hernando counties) that secondary teachers in some subjects are being ordered to spend the first 10 minutes of class suspending the curriculum and teaching material from another class. In the case of two counties (Pinellas and Hernando), I have heard stories that math teachers are being asked to teach 10 minutes of reading--not include word problems in math, which is certainly appropriate, but teach reading (a subject very few of them would have certification in). In one county (Hillsborough), I have heard a report from a student that a high-school anatomy teacher has been asked to spend 10 minutes reviewing other science subjects (and the emphasis appears to be in chemistry), probably to prepare students for the 11th grade FCAT science comprehension exam.

In 2008, the Florida legislature added a section to the existing law on assessment (F.S. 1008.22(4), if you're curious), specifying limits to what schools can do to prepare for tests, specifically

STATEWIDE ASSESSMENT PREPARATION; PROHIBITED ACTIVITIES.--Beginning with the 2008-2009 school year, a district school board shall prohibit each public school from suspending a regular program of curricula for purposes of administering practice tests or engaging in other test-preparation activities for a statewide assessment.

There are a number of exceptions to this prohibition--school districts can distribute sample test books, teach test-taking skills in limited quantities, etc.--but the spirit is clear: schools are not supposed to be engaging in test-prep that is a substitute for instruction. And taking time away from math class to teach reading, or away from anatomy to teach chemistry, looks like it's clearly prohibited.

It's also counterproductive from an administrative standpoint: if you wanted to add reading instruction, why would you ask a math teacher to do it? I should be clear: these are unconfirmed reports rather than documented examples. But if these reports are true, this clearly looks to be an end-run around ordinary curriculum policies requiring a certain amount of instruction in the classes to get more instruction or more test-prep in for high-stakes subjects.

There is one additional legal problem with this practice: there are both state and federal policies about teacher qualifications. I bet it's illegal in a number of respects to assign math teachers to teach reading and then report that everyone instructing in a subject is properly certified.

I have contacted the three districts in question to ask where the policies required by the law are. If you are aware of any specific examples (and I would need the school, date, class, period, and witness for sufficient documentation), please contact me by e-mail (sherman dottish-thingie dorn at-symbol-stuff gmail.com).

November 12, 2009

Race to the Top: review, revise, redux

I am in California this weekend for the Social Science History Association annual meeting, where we get to talk about Maris Vinovskis's book on the last quarter century of school reform, and since one of my copanelists Saturday morning is Jennifer Jennings, I finally get to meet the sociologist-formerly-known-as-Eduwonkette in person, face to face. Because several family members live in Costa Mesa, I also get to enjoy Kean Coffee about 20 miles south of the conference hotel/cruise ship (when the heck did the SSHA officers decide to book the Queen Mary??!).

While the focus of the book panel will be ... well, Maris's book, I'm sure we'll be talking about Obama education policy at some point, including Race to the Top. I was rushing around last night not getting enough done, so I didn't have a chance to do more than casually skim the stuff that's now available on the revised final guidelines. A few initial thoughts:

  • Bottom line? No idea. I traveled west and had coffee (see above), so I don't have a bad case of jet lag, but I've been on planes for 7 hours today. 
  • I very much like the competitive priority on STEM fields. That uses a standard device for focusing grant-writers' minds in USDOE competitions (the bonus points for meeting a competitive priority). (Disclosure: it looks like my state's department of education is following the push a bunch of us have been making about using Race to the Top funds for end of course exams, especially in science.)
  • From the list of changes made, it looks like there have been a lot of political calculations made on what changes had to be made to keep stakeholders in the game and what had to stay the same to satisfy policy goals.
  • Duncan is not anal retentive enough to make the points add up to a "nice round number." I have a suspicion this is deliberate, and if so I think I know the reason why.
  • People who focus on the total potential range of points for each section are missing an important feature of point distributions in scoring systems: it's the actual range and not the potential range that matters on rankings. If the potential range is 58 points from top to bottom on one component but the scoring leaves a real-life range of 10 points, it doesn't matter that the total number of points is 58. It could have been anything from 10 to 58. So what matters is how the reviewing panel looks at everything.

If we have time, I'll try to persuade Jennings to put on her Eduwonkette cape and save the state where I grew up. But I think California's problems are beyond what even a brilliant sociologist can solve. At least I get to see family members, which is worth the jet lag I'll be fighting in the next week.

Methodoxology

My graduate students are reading Jeff Henig right now, and it appears that few editorial boards or other advocates have taken his argument in Spin Cycle seriously, at least from reactions to the latest sets of charter-school reports issued by think tanks. Ritualistic incantations at the publication of the Brand New Latest Report Showing That Your Deepest Beliefs Are True should be tempered by the possibility that Sean Reardon might soon write a Think Tank Report focusing on the study's methods. Hoxby is a respected economist, and the key point of Reardon's report should be to remind us that one study does not a literature make. As Henig argued, individual studies are drops on the mill's paddles and are very rarely the whole stream. Or as Colorado's Kevin Welner said in response to Reardon's review of the 2009 Hoxby report, even the most enthusiastic reader of a study on one city might wish to "explore the causes rather than to jump to broad conclusions." I will hereby jump to the broad conclusion that this is wise advice.

November 10, 2009

Incentives for high school curriculum change

Leslie Maxwell writes a short and solid blog entry (and maybe a story later this week) about the politics of college admissions at San Diego State University. Specifically, SDSU's move to eliminate a preferential admissions policy for high school students from San Diego has sparked a debate about perceived obligations to serve the local community. I am of multiple minds here about the consequences of excluding potential students who are unlikely to move to go to college outside their home county, but I don't know if the potential SDSU students outside the county are more or less advantaged on the whole, and what would happen with college completion.

On the other hand, I think see where the dynamics are heading... towards setting up one of the local districts (Sweetwater Union's school district) as a model because of its existing compact with SDSU. I recall Peter Sacks reporting on a certain high school teacher in Oceanside, and I'm curious how he'd see this. Calling Peter Sacks...

November 6, 2009

Issues in electronic grade reports

This morning's article in USA Today on electronic grade reports is a reminder of a few important facts in evaluating technology use in schools:

  • Ease of use (in jargon, "usability") is critical to adoption. The systems that existed a few years ago were (and many still are) clunky and hard to use for both teachers and parents. New systems are becoming easier for parents to use, creating different accounts for students and parents (so students are aware of what parents can access but not interfere with that access), e-mailing notices of new grade uploads, and so forth. Larry Cuban's dicta about hybridization still hold true for anything living on a server.
  • The digital divide is especially important to pay attention to when private records are involved. Many poor parents and children use public libraries for internet access. With libraries' reducing hours, and with the public nature of computer-use rooms in libraries, parents without at-home internet access face significant barriers to accessing information that is online. That doesn't mean that districts should not build on-line systems, but there needs to be careful thought about how parents might access the information when they do not have private internet access, in the same way that there is a need to plan for parents with disabilities, parents who do not speak English, etc. 
  • Districts should begin to figure out how to bring data together for parents. I'm not talking about a giant data warehouse--that becomes cumbersome (as well as security-fraught) if anyone can have access to databases--but a slim addition to the type of stuff that is showing up in the online grade report systems. I've proposed that for high school students there could be something akin to a look-at-everything-your-student-is-doing "dashboard" (if you'll forgive that term). Grades, extracurricular activities, jobs, etc. That will take some careful thought, but maybe an economic crunch is the right time to do it, when districts will think about the tradeoff in use v. design/maintenance costs.

My children's high schools are both using Edline this year, which is a dramatic improvement from attempts at online assignment and grade access a few years ago. There are still significant issues: some teachers find the interface hard, the school district took several weeks before realizing that maybe it might want to send the private authorization codes to parents in the mail rather than entrust them to students, and the school district still has not yet addressed the divorced-parents issue with regard to access (at least from the report of one co-custodial parent frustrated that the other parent has the authorization code and sole access but isn't using it). This is still significant improvement from my perspective. 

Now, if only the school district will get new online systems for high school counselors to schedule classes, for special educators to work on IEPs, and teachers to sign up for professional development. At least in Hillsborough, those are legacies from when the district incompetently tried the low-bid strategy with vendors who didn't demonstrate capacity to fulfill the contracts, and so everyone is stuck with systems that still (expletive verb) (colorful adverbial expression). 

November 4, 2009

Election results -- eh.

Andy Rotherham has a tempting interpretation of election results (and their effect on federal education politics), but I'm guessing he's just suffering from living in Virginia this morning. Normally, it's a very nice state, but I've seen some pretty-well-expected "darned my state is going down the tubes" messages from Va. acquaintances over the past 12 hours. 

The more fundamental questions for any domestic initiative are whether health-insurance reform passes this year and what happens with employment in the next 4-5 months. My best guess is that health-insurance reform will pass and employment will start to nudge up but not by leaps and bounds. The result is that the potential for "oh my gosh I have to protect my seat" paranoia by majority Congresscritters will abate as a result of a health-insurance law but that pressure on the employment front will keep members of Congress nervous (regardless of party). 

And, in any case, since the action in education politics is usually at the state level, that's where the import of yesterday's elections lies:

  • The death of two more TABOR referenda means that education funding is imperiled only by a horrid economy and state revenues. Yippee?
  • An unpopular Democratic governor in NJ is replaced by a Republican governor who may well enter office nearly as unpopular, facing a legislature that tends to protect wealthy communities at the expense of poor communities when it comes to education. 
  • A popular Democratic governor in VA is replaced by a conservative Republican governor who promised to focus on education (among other service-oriented campaign promises), with a legislature dominated by Republicans. 
  • In the sick state of New York, a billionaire buys a third term and a probable minor scandal about his elbows as well. In the meantime, an ineffectual governor will increasingly be overshadowed by state-level politics over education. 
  • The sick state of California loses its often-running lieutenant governor to Congress. 

November 1, 2009

Ready-made dissertation topic on local school politics

Anyone looking for a dissertation topic on school policy or politics can now rest easy: read the Palm Beach Post's description of a local reform effort that blew up in the face of a superintendent. You've got everything in there from the data-driven mantra to parental backlash to odd bedfellows with the teachers union and coalition politics. I have been watching the story unfold for a few months and suspecting that there's been a lot more beyond the headlines. I want to read the book on this, so get cracking, somebody!

October 30, 2009

Do Times reporters know the difference between percentages and raw numbers?

I suspect the following is an unfortunate placement by the reporter on a story about record high percentages of young adults in college (with an emphasis on percentages):

"What's behind this," Mr. [Richard] Fry added, "is that we have the biggest pool of young adults we've ever had who've finished high school."

I suspect that this is in reference to the growth of enrollment in two-year colleges, not total college-going. That distinction was not clear in the article.

October 29, 2009

Channeling Jerry Bracey on "proficiency": it's political, not scientific

One of the late Jerry Bracey's hobbyhorses was the pretense that the NAEP achievement level labels were scientific, as he argued in 1999: "The standards have generally been the object of scorn and derision from the psychometric community." He was fond of quoting the 1999 report on NAEP proficiency levels, esp. from p. 162: " Standards-based reporting is intended to be useful in communicating student results, but the current process for setting NAEP achievement levels is fundamentally flawed." So when NCES issues a report comparing the implied theta-values of cut-scores for proficiency on state assessments to the theta-values of cut scores for proficiency on NAEP and both Ed Week and the Christian Science Monitor report on the paper with a straight face, we're obviously seeing one place where Bracey's voice is already missing.

I think Jerry perseverated on this issue, to the detriment of a sensible argument about political judgments. The larger point which is inescapable is that cut scores are set arbitrarily, and there is no way to avoid that fact. Those who support setting achievement levels hope and pray that they're arbitrary in the sense of arbitration and careful judgment, not by being capricious. But they are arbitrary, and even moreso the labels assigned them. What we know is that someone who scores at a "proficient" level on NAEP is scoring higher than someone in the "basic" band. That's all we know from those labels: ordinality. Moses did not come down from Mount Sinai with NAEP scores carved in tablets. 

So what do we do with the inherently political nature of those labels? As I have argued in Accountability Frankenstein, the caution with which we use the judgments on cut scores should depend on the stakes of their use. If they're used to target resources, that's one thing (resources are going to be targeted in some manner), but the more that someone's job depends on them, the more wary we should be of how we set thresholds. 

Today, however, NAEP labels and cut-scores are serving a purely performative act, to stigmatize states for their political response to NCLB. I hereby propose that we have the following new labels for NAEP achievement levels: 

NAEP-achievement-levels.jpg

I think that's in the spirit of the day's report...

Correction: I assumed that NCES was using detailed data from the state assessments to estimate IRT parameters. Silly me. They were using distributional data for linkage. Oops... for me for forgetting the methods from the last such report. I'll let the measurement folks argue about the methods used here. 

October 25, 2009

Ted Sizer's push

It had instant credibility to the vast majority of readers who all probably shifted uncomfortably while reading certain passages, recognizing themselves. And the terms that came out of that project...

Classroom treaties.Tell me if you don't remember an entire class wheedling a teacher or two to change an assignment, to lower expectations a smidgen, and also reduce the teacher's workload. 

The anonymity of the high school student. Tell me if you don't remember the bright classmate hiding in the back of the class, never called on, never pushed to think hard, never affected personally by a teacher.

The shopping-mall high school. That was the title of one of the other books that came out of the same project, and while it had a bit more of an edge, it had the same subtext: we can expect more. 

Exhibitions. Most people call them portfolios, but he wanted them to be exhibitions in a more public sense, to get adolescents to be proud of their work, even if they were works in progress themselves (as are we all). 

I know that I'm going to read laudatory eulogies of Ted Sizer in the next month, and I hope they don't forget his strategic choices in the 1980s, as he put together the project that became Horace's Compromise, The Shopping Mall High School, and The Last Little Citadel. I suspect that while his own books will be emphasized, along with his Essential Schools project, there was a subtle and clever point about his focus on the plurality experience in suburban high schools after World War 2: "I'm talking about you. Not Other People who don't have your advantages. You. Your children. How we're not expecting what we can from teenagers in your life."

His underlying ethic was one of pushing teenagers in healthy directions. It's close to Deborah Meier's point about a small high school: adults are supposed to be "in your face" in the right ways, so adolescents don't disappear into the woodwork. It's a structure to encourage pushing without having to be pushy. "I love you and expect more from you." "No, you can't get away with that." "I know you can do more." It's not without choices, by any means, but the choices have consequences and need to be deliberate, not the first thought off the top of a teenager's head. "That's interesting. How else could you do that?" "How did that affect the people in your lives? What else did you think about doing?" It's about pushing teenagers into thoughtful independence. "Here's the end goal. How would you get there? What would be your first step?"

I'm at the History of Education Society meeting this week, and there are so many here who knew or worked with Ted Sizer, including Bob Hampel (who wrote The Last Little Citadel). Many of the historians of ed who knew Sizer closely have retired, and many of us (including me) are young enough and unlucky enough that we never met him. But we know both his scholarly contributions (the first serious historical work on the high school) and his contribution to serious reform discussions over the past quarter-century.

In lieu of sending flowers, don't let an adolescent get away with sloppy thinking this week. Push.

October 22, 2009

Duncan's talk at Teachers College: first impressions

Some quick impressions of the text of Arne Duncan's speech at Teachers College today: 

Historical quibble: Duncan said he was speaking at a place where "giants like John Dewey played such a formative role." No, he didn't, or at least not at Teachers College. When Dewey moved from Chicago to Columbia, he moved from education to philosophy, which is south of 120th Street. At Teachers College at the time, Edward Thorndike was far more influential. And after Dewey left Chicago, Charles Judd ruled the roost there. Correction to the quibble: In comments, Aaron Pallas points out that Duncan's speech was sponsored by Teachers College but held in a lecture hall south of 120th St. (i.e., on the Columbia side of the Academic Gorge of the Upper West Side). I stand corrected.  Or I blog corrected.

Right: Duncan is correct that teacher education in the U.S. is currently inadequate. Duncan is correct that colleges of education do not teach everything that teachers need, and the reports he hears (about the inadequacy of preparation for classroom management and use of student performance information to improve instruction) is consistent with plenty of other information.

Wrong: Duncan wrongly implies that teacher education can easily fill the holes that teachers see from the classroom. Many years ago, I remember seeing the surveys for one absolutely solid program that taught about behavior management and using student performance data in a rigorous manner, and the primary complaints of alumni/ae was ... that the program didn't prepare them adequately in classroom management. On some things there is no substitute for experience, I suspect. 

Right: Duncan argues that teacher education programs (and states) have not looked sufficiently to what happens with their graduates and the students of their graduates. He points in contrast to Louisiana's longitudinal analysis of teacher preparation programs, and he is right to do so. In contrast with all sorts of self-aggrandizing projects, George Noell has built a team whose reporting is relatively careful with methods and conclusions.

Wrong: Duncan baldly claims that he knows what good teacher education looks like.  Dear Secretary Duncan: don't you remember the other part of the speech where you said that we don't look sufficiently at outcomes? Either we need to look at data carefully to figure out what works and what doesn't, or we know everything right now. I suspect that we know plenty of stuff that does not work, but that doesn't say much about the inevitable tradeoffs--whether it's more important to put resources into giving teachers detailed assessment classes or putting principal and specialist candidates through those classes, whether it's more important to make teacher-ed students spend their entire last year in schools (as happens with one of the programs Duncan praises), or make them spend more time learning content. By highlighting and praising a few current fads in teacher education, Duncan is falling into the same pattern he criticizes schools of education for. 

Right: Duncan did not try to point fingers in politically-convenient directions. He did not try to claim that all teacher-ed programs are alike in content or structure. In contrast to Arthur Levine's semi-ahistorical report, Duncan did not claim that a major problem somehow lies with those of us on the margins of teacher education (as if all colleges of education are run by philosophers and historians). He correctly pointed to the institutional environment within which teacher-education programs operate:

It is far too simple to blame colleges of education for the slow pace of reform. In fact, universities, states, and the federal government have all impeded reform in a variety of ways.

Minor quibble here: One could legitimately claim that colleges of education have been on the forefront of reform plenty of times in the past century, sometimes but not always on the side of  improving education. See my note above about Dewey, Thorndike, and Judd. And Diane Ravitch is correct about Teachers College in one very important way: William Bagley was on the right side in the early 20th century, against the conventional-wisdom of the day about reform. 

But the reasons why elite schools of education headed in the wrong direction at the time fits with Duncan's institutional context: for universities, the easiest money in the early 20th century was in collecting school administrators and administrator wannabes into graduate programs, at the beginning of a trend that no one who reads Duncan's speech text should be surprised about: for decades, education and chemistry regularly vied for the highest number of doctorates granted in the country. 

I teach at a college of education, one of the larger ones in the country. At first blush, Duncan's criticism strikes me on the whole as reasonable, and far more reasonable than the more venomous attacks I've seen before. I would love to trade the double standards and incredible micromanagement of programs we currently experience in our state (and I could tell tales of some of the idiocies we experienced in our last joint state-NCATE review--and this comes from one of the faculty members who had relatively little time sucked away for this) for a requirement to pay attention to what happens to our graduates and their students after they leave us. 

October 21, 2009

A gadfly remembered: Jerry Bracey

An e-mail from Kevin Welner yesterday announced Jerry Bracey's death Monday night. I only met him a handful of times in the past 20 years of his persistent, indefatigable efforts to poke holes in every public report or news story he saw as an effort to demonize public schooling. His Huffington Post column from September 25 is representative of both the topics that he addressed year in and year out and the disdain he felt towards those who he thought libeled and slandered public-school students and educators.

According to one online biography, he was an early-childhood psychologist at the Educational Testing Service and Indiana University before becoming a testing director for the state of Virginia in the late 1970s and then taking a similar position in a small school Colorado district in the mid-80s. At about the same time he moved to Colorado, he began writing columns on education research for Kappan magazine, and in 1991 he wrote a long article excoriating critics of public schools, primarily the authors of the 1983 A Nation at Risk report and anyone who repeated the claims in that report.

He has spent the last 18 years writing detailed critiques of whatever target happened to catch his eye. I first met him when he visited the University of Delaware in 1992-93 as he was beginning his second career as a mythbuster. My impression at the time was that he was smart, detail-oriented, and tilting at a windmill. I think my judgment at the time has been borne out by his writings since then. For more than a decade, the Kappan magazine published his annual "Bracey Report on the Condition of Public Education," which usually praised a handful of individuals and dished out acidic criticism to those Bracey thought were fools or worse. For a few years, Kappan published his "Rotten Apple" awards with Bracey's annual report and then thought better of it once the first lawsuit threat appeared (when Bracey handed Willard Daggett the "No, you're not a ham, ham can be cured" Rotten Apple Award in 2000). Thereafter, every year at about the same time as his rotten-appleless report appeared in Kappan, Bracey would e-mail his annual Rotten Apple nominations to the world (or at least a long list of recipients), eventually publishing them and the annual report manuscripts online. Bracey was the Pauline Kael of education research.

Bracey was a true gadfly, a semi-retired professional who did his best to discomfit those who he thought were abusing their positions. He held no White House post, no political appointments in the U.S. Department of Education, no leadership spot in a well-funded think tank.

It is often the case that gadflies are ill-appreciated during their lifetimes, and often they pick the wrong windmills, or they tilt at windmills when they could be digging out the foundation instead. But Bracey was always there to respond to what he thought was poor reporting and sloppy thinking. There is probably no national reporter on the education beat in the past 20 years who didn't hear at one point or another from Jerry Bracey about Simpson's paradox or why NAEP's achievement levels were more political than scientific. Debra Viadero's blog entry today is very much in the vein I've read from reporters on occasion over the years: "He was, to put it bluntly, a thorn in our side. Once in a while, though, he had a point and I was awed by his tireless persistence and his willingness to heap criticism on government leaders from both sides of the political aisle, from Margaret Spellings to Arne Duncan."

October 17, 2009

An historian reads the business section (with apologies to John Allen Paulos)

I do not generally comment on economic matters, but I think historians of education can say something productive about the current myths plodding around the internet about the stimulus and the non-bank sector of the banking industry. First, some of the current discourse:

  • Sean Snaith, an economist at the University of Central Florida, is unimpressed with stimulus dollars being spent in Florida, arguing that to do much good, the money should have come in and been spent much faster.
  • John Quiggen is upset over at Crooked Timber over Goldman Sachs's profiting from risky ventures, or maybe upset that they're getting significant leverage over financial firms that have taken federal recapitalization and sat on the money, or repaid it to avoid additional regulation. I am not exactly sure how close Quiggen is to Krugman's being upset that we're not moving fast enough to regulate the unregulated (non-bank) part of banking.

These appear to be fairly standard concerns with economists. And I sort of understand that, except for a few perspectives from the history of stodgy institutions (schools):

  • Sometimes moving slowly is what's needed for longer-term needs. As other economists have pointed out, White House economist-in-chief Christine Romer's broader concern has consistently been with the general output gap over several years. In contrast with a mild recession where the output gap really is short-term, we're going to have problems with output for more than 8-12 months. So spending over more than 8-12 months is not a bad idea. This is about saving the entire country's economy, not just Florida or any single state.
  • Lots of institutionalized changes are hidden, and that's as true for the stimulus as it often is with education. For political purposes, the White House is now starting to highlight the jobs created and to a lesser extent the jobs saved by the stimulus. To my mind, it's the thousands of public-service jobs saved that are evidence of effective policy, but that's hidden because people have kept jobs (and it's hard to see non-change as a success). Similarly, part of the stimulus is the reduction in federal income-tax withholding. If I understand things correctly, that's more effective than a tax rebate precisely because it's not that visible, and people of low and moderate means are likely to take that extra money every paycheck and spend it on things they desperately need to pay for... and that keeps demand up. (Giving people a tax rebate may be perfectly justifiable public policy for other purposes, but I'm not convinced that it's effective for stimulus.)
  • Instead of hoping that we can fix those buggers so they can't game the system anymore (a common dream in accountability policy), maybe we should assume that the attempt to game the system is as much of a permanent feature of financial institutions as it is in schools. And maybe we should take a long-term perspective that we always assume there will be attempts to game the system and a need to adjust public policy on a cyclical basis to respond to such gaming. As many have pointed out, even if the bank-in-name side of banking has recovered and started to lend again (and I think it has), there is a huge hole where the non-banking side used to leverage itself out the wazoo to give out subprime loans, liars' loans, and the like. Yes, there needs to be better regulation of the finance industry, but we should assume it is always incomplete and never done. An example of where the evolution of financial regulation worked is in so-called peer-to-peer lending, where propser.com and lendingclub.com popped up in the wake of Kiva's charity microlending on a social platform. The difference between charity social-networked lending and social-network lending with interest is disclosure and risk. In Kiva, you're not expecting interest, and you know that your money loses value every day it's out there in a loan. But that's not a problem since your goal isn't making money. In 2008, the Securities and Exchange Commission ruled (properly, I think) that the Prosper and LendingClub operations were essentially securities and needed to be run as such. And both sites have now been approved and reopened as SEC-approved securities operations. This is where regulation works well to keep things transparent. This doesn't mean that P2P lending serves the functional role of putting money to its most productive uses, but I don't think subprime lending did, either, and at least the risks exist and are stated up front, while individuals have the power to make both wise and foolish investment decisions. 

And now, I'll crawl back into my HistoryCave, waiting for the next Little Red Schoolhouse silhouette to show up on the underside of my metropolis's clouds to signal another emergency requiring an Historian of Education.

October 14, 2009

Don't exercise: you'll destroy the world

If you had asked me this morning what I expected from the latest round of NAEP math scores and what was going on in DCPS, I would have told you to expect NAEP math scores to increase at a snail's pace with loads of arguments about what that meant, that Michelle Rhee seemed to have decided at long last that working out a deal with Randi Weingarten was more important than a charismatic image, and that maybe we should focus on long-term issues more than evanescent news stories.

After I exercised midday ... and got dizzy and fainted slighty (I'm fine, don't worry, it's only a flesh wound) ... only one of those statements turned out to be true. Fortunately, it's the most important one. I wouldn't make too much of the NAEP trends from a single cycle, nor of the apparent resurgence of the image of Michelle Rhee the Warrior/Tyrant (depending on your POV).

But I've got to say I'm a little worried here. I partially lose consciousness, and a little bit of the universe's fabric frays. I've learned my lesson: I'll never exercise again, to keep the world and reality safe.

For those who are curious: probably a combination of too-little a/c in the gym and my body trying to fight off a virus. My daughter had a fever last night, and while I don't have a fever, I've been exhausted for the last 3-4 evenings. And the most embarrassing detail? It all happened at the leg-press machine. I mean, if you've ever looked at me, you'd say, "If that guy ever faints while working out, it'll be on upper-body work. No real biceps, and don't even try to identify triceps on the man. But the thighs and below? Not a problem." Apparently the large muscles in my body had a larger appetite for blood and oxygen than was healthy.

The comparability fly in the Ouchi/principal-autonomy ointment

Yesterday from a "stakeholders" meeting (I think at the USDOE), Charlie Barone tweets,

Richard Laine of Wallace Foundation: forthcoming Rand study will show [principal] autonomy in hiring a key factor in student achievement.

I've been expecting something like this for a while, not because I'm connected to a RAND insider (I'm not) but because this is the obvious new version of decentralization form that would marry the 1980s-90s site-based management fad with new managerial fads in education.

To some extent I am attracted to Bill Ouchi's argument about principal autonomy leading to lower total student load. Ouchi's claims about total student load is essentially one of Ted Sizer's central arguments from Horace's Compromise, that the number of students a teacher sees is a key factor in the ability to push student achievement. But... and here's a fairly important but... Ouchi's work is tantalizing rather than definitive (because it has not be replicated substantially in terms of total student load), and the temptation to manage large urban districts as "portfolios" with quasi-independent school-level management may push a single form of decentralization at the cost of comparability in expenses and access to great teachers.

What the heck do I mean by that? In a sentence, we may not want principals to have complete autonomy in a task where they have relatively weak skills: knowing which novice teachers are going to be great teachers.


Everyone and her or his grandmother is focusing on the problem of where senior teachers work. This is an intellectual sleight of hand if you simultaneously argue that teachers with seniority are taking advantage of contracts with seniority privileges on transfer to avoid schools who need them and also insinuate that experience means nothing. Let me get this straight: we need to prevent experienced teachers from exploiting labor-market choice to move to schools with more comfortable teaching situations because... they're not inherently any better than teachers with only a few years of experience? This is an inconsistency ripe for Jon Stewart-like treatment.

More important than the intellectual sleight of hand is the way that this argument ignores an opportunity for a simple but politically sensitive intervention we could make that could simultaneously improve the lives of poor children and new teachers: create regional new-teacher clearinghouses and matching services. Here's the thought experiment: Far from decentralizing, I think it would be a healthy system for schools to require new teachers go into a large regional market where vacancies for relatively new teachers (e.g., those with fewer than three years of experience) would be balanced with a matching process akin to matching of med-school graduates to residencies. This would require collective bargaining and regional agreements between districts (or changes to statute), but here's the idea:

Brand-new teacher's perspective: A new teacher registers with the regional teaching market clearinghouse, with all of the stuff you'd want applicants to provide. The clearinghouse is directly tied to vacancies in the region, and that would probably include multiple districts in most parts of the country. The clearinghouse matches teachers to jobs for the first year. The teachers and administrators are told, explicitly, "This is a one-year arrangement. In the second year, the teacher is headed to a new school, and the administrator provides an evaluation knowing that the teacher is not coming back to that school until at least two years down the road." And that's what happens. At the end of the first year, the clearinghouse matches jobs to teachers who want to continue teaching and whom the first-year administrators recommend continue. Same with the end of the second year. And the clearinghouse's job is to make sure that by the end of a new teacher's third year, that teacher has worked in multiple settings, with different characteristics of students (at least within the range of the region), in areas of the teacher's documented expertise (i.e., no out-of-field matches). 

At the end of Year 3? Open market in the spring, in most places, and administrators wanting to hire on the open market must hire teachers with at least three years' experience -- in other words, teachers for whom there is a record of evaluations from different administrators and for whom there is a record of performance for students in different settings (within the range of the region's student population). Schools are allowed to hire teachers who worked in their schools before... if the now-third-year teacher wants to work there again.

Benefit to teachers: first-year teachers stuck with horrible administrators (or generally toxic environments) know that they'll be moving on if they survive. They'll get experience with multiple settings where they'll be able to demonstrate their chops. At the end of their third year, they'll have some variation in experience with administration to be able to judge people better when applying in an open-market situation. Disadvantage to teachers: if you happen to get lucky and get a great job in Year One, you have to move on.... and let another new teacher get the benefit of that experience.

Benefit to administrators: because new teachers are forced to move on after a year, honest evaluations are less likely to result in social backlashes. When you hire on the open market, you'll know you'll have evaluations and (where this is gathered) other performance data that is from school settings with a range of student populations. Disadvantage: you don't get to hire absolutely new teachers; you get whom you get, and if you were great spotters of talent, or you think you're better than the average principal at spotting good talent, you'll be upset.

(Personally, I think I would prefer this as an administrator: if you've read Moneyball, you know the sabremetricians' rule of thumb: you can predict a baseball player's professional performance from college experience, but someone straight out of high school is just a raw bet without college experience. Why would you want the authority to make hires in a situation where you're almost guaranteed to be a worse judge of talent/skill than any other personnel situation? Then again, I'm sure many principals think of themselves like the [very poorly-predicting] old scouts of baseball, making seat-of-the-pants judgments.)

Advantages for systems: See advantages for administrators above. In addition, you have lower risk with variation in administrators' skills in talent judgment, while principals would still have the autonomy to pick more experienced teachers, after they pick up enough of a record for administrators to see who has more talent. You could also get development of evaluation skills in a regional context without diseconomies of scale. If clearinghouses have to track teachers, they could also be tasked with additional evaluation responsibilities across a region. Advantage for relatively poor systems: you know that wealthier districts will not be able to be as much of a magnet for new teachers, because of regional rotation, and you could push administrators to do what is necessary to convince teachers that they want to return to your district after their initial three-year rotation is done. Disadvantages: there would need to be legal agreements to cover this, and there would be some logistical challenges in identifying vacancies (and making sure those vacancies are reported accurately and promptly) as well as the operation of a clearinghouse. School districts would have to delegate hiring authority for some of their jobs to a regional body, and if school systems really thought that they were hot stuff in terms of talent scouting, that might be hard to swallow. (See above and Moneyball on the egos of baseball scouts and possibly school administrators.) Disadvantage for wealthy districts: poof goes your advantage in recruiting brand-new and relatively-new teachers, because they'll spend some time in your districts but also some time in poorer districts.

Now, the payoff in terms of debates about comparability: a regional new-teacher clearinghouse/matching process would instantly equalize a significant part of the teaching staff across a region, because of rotation among jobs and districts. Yes, there would still be an advantage of wealthier districts in attracting teachers with three or more years of experience, but poorer districts would know that they at least have a shot of persuading new teachers that they can make a good career inside a district... if the relatively new teachers have an experience that is supportive. 

Remember that this is a thought experiment: I don't know of any places with regional new-teacher clearinghouses/matching services, and I dreamed it up out of whole cloth (plus some inspiration from what happens with med-school students). But I think it points out a structural problem with giving principals entire autonomy: with complete autonomy, there is no balancing out of regional needs. Equality of opportunity would depend entirely on the skills of individual principals, and while principals are extraordinarily important, that's putting a heck of a lot of eggs in a single basket. If you care about making sure that a broad range of students have access to great teachers, there are serious dangers in the Ouchi principal-autonomy approach.

Why you don't always need a statewide charter authorizer

I don't understand the obsession some people have with multiple charter-school authorizers. In Florida, it has always been the county school board since the charter-school law was first approved in the mid-90s. A few years ago in Florida, the legislature decided for some reason that not enough proposals were being approved and created (and spent gobs of money) on a commission that would be an uber-authorizer. To me, it looked like a giant loophole for low-quality applications and politics. Fortunately, the state court system struck down the law as an infringement on the constitutional authority of school boards. 

While there may be an apparent conflict of interest between an elected board and charter-school authorizer, in fact there has not been. And there is a need for people with at least some experience looking at schools to vet the proposals. In Manatee County, for example, staff are going to recommend that the school board reject 8 applicants to open charter schools, and at least from a Bradenton Herald article, the rejections would come for some fairly good reasons. There is no charter cap in Florida, and a number of school boards have no problems approving well-planned charter schools. In addition (and this is fairly important), we have a public-records and open-meeting statute that is rigorous, and the administrative rules in place in Florida make it difficult for a public agency to be arbitrary without being held accountable on appeal.

Disclosure: I have been associated with two organizations that have started charter schools... and then stopped running them. In one case (my university), it was planned in my first year and I would not have had the chance to participate in planning. The USF charter school was essentially turned over to the county public schools (and became a local public school within the system) some months ago. In the other (a non-profit organization), I expressed my concerns about organizational capacity from the inside, the charter school started operations, and I was no longer a member at the time that the school closed.

October 12, 2009

News item: Boy Scout suspended for being prepared

The suspension of Zacharie Christie is the latest tomfoolery in zero-sense discipline policies, because the tyke decided to bring his Boy Scout cutlery to school. Next: Fox News special on the Evil Spork. And this brings to mind a parody of medieval-fandom-society "weapons at the door" policies:

A Bard was next whose goodly Voice has entertained us all
but he, too, was prevented from entering the Hall
and told he could not carry deadly weapons on the floor
he left his Voice and Harp among the weapons at the door
    -- Joe Bethancourt, "Weapons at the Door" (1974)

October 10, 2009

One Blog Schoolhouse: the PDF

Should've been done a few months ago, but if you want to read the entire text of One Blog Schoolhouse, it's now available as a nonprinting PDF. (I recommend that you click the "PDF" link in brackets, since I don't know if scribd will convert a nonprinting PDF.) The entire thing. Absolutely free to read.

October 8, 2009

First, find me a box of cereal that squirms and drips snot in winter

Congratulations to former Florida Governor Jeb Bush, who knows a critical rule of politics: declare victory whenever you can, no matter whether you were right. I am quite serious about his political acumen: his push of a system that assigned letter grades to schools was ingenious politics. And Bush deserves credit for supporting a research technical assistance center in Florida as well as funding for reading coaches. But Jeb Bush's comments to the Jeb Bush Celebration Conference this week had an interesting quip:

Frankly, if Walmart can track a box of cereal from the manufacturer to the check-out line, schools should be able to track the academic growth of a student from the time they step in the classroom until they graduate.

I am firmly in favor of using longitudinal data, but this comment is cheerleading and not serious discussion. There are significant challenges in the creation, maintenance, and use of longitudinal data systems, and Walmart-style tracking logistics don't touch the greater ones.

October 6, 2009

Dozens of Veblens, a handful of Heckmans, not one Keynes

Skimming the Ryan Lizza portrait of Larry Summers, reading Paul Krugman's focus on the size of the stimulus, and listening to Krugman's The Return of Depression Economics while driving around Tampa this week makes it clear to me that outside the bubble that is Fox News and talk radio, Krugman, Romer, and a number of other liberal economists are at heart technocrats: when they looked at the Great Recession, they saw an output gap that government could and should fill. This isn't socialism; this is Keynes. (As Krugman notes in passing in Return, Keynes was someone who believed in capitalism, in contrast to plenty of others who thought differently in the 1930s.) When you think there's a technical fix, you're not a revolutionary. And Krugman definitely thinks there's a technical fix to be had here.

Despite whatever else one might say about the dominance of economic policies by anti-empirical rationalists, freshwater Austrians, and other odd critters, Gertrude Stein's quip doesn't have a foothold: there is a there there somewhere. But we are far from there in education. There are a number of erudite, smart commentators on education, and while I try to learn from all of them, there is no Keynes. There are a number of technical savants in different corners of education, the education equivalents of James Heckman (and, heck, Heckman's helped out with small heaps of his talent focused on education), but there is no Keynes. And there are people who would like to or pretend to be systemic analysts like Keynes, but there is no Keynes. 

In part as a result, when people debate education policy, it quickly slides into attacks on politics and posturing and whatnot. Now, of course people criticize Krugman for his politics and accuse him of posturing, but it's easy to see his technocratic leanings and from whence they come. There is no equivalent in education. Maybe there shouldn't be, but at least as a consequence, maybe we should stop pretending that there is a macro technocratic field in education. There just isn't.

October 3, 2009

Child murder, Chicago style

Chicago teacher Deborah Lynch pointed out in a Sun-Times opinion piece yesterday that one of the Chicago schools' "turnaround targets" this fall has been Fenger High School, near the gang fight that led to Derrion Albert's death and the school where she implies many of the combatants attend. (Hat tip/alternative source.)

I am not saying that knowing the kids better could have averted the melee and tragic death of last week, obviously. But trouble had been brewing at the school even before last week. Staff reported a riot the previous week inside the building, involving teachers being hit, and that two different police stations had to be called in to quell the disturbance. Those are the times when the staff members draw on their relationships with kids to urge restraint, to urge calm and peace, to try to talk things out rather than fight things out. Those are the times when a seasoned staff can identify strategies and resources to address and prevent further problems.

Lynch's argument is interesting and plausible. I'd be cautious of taking it at face value, but don't toss it out the window. As far as I am aware, there is nothing either to contradict or to support the claim that the length of time a staff (as a whole) has spent in a school is predictive of the general school environment. I suspect it depends on the staff; experienced good teachers and staff are going to have the types of relationships with students that Lynch describes.

But there is another important limit to Lynch's argument, and I'm thinking about the debate that's usually focused on academics rather than violence: the relationship between schools and the rest of students' lives. I suspect that if George Schmidt is correct, that the police congregated around Fenger rather than following potential combatants, any immediate investigation needs to focus largely on the tactical decisions of the police. It's possible that no matter what happened in the school, the gang fight would have occurred unless police decisions had been different.

The murder of Derrion Albert is representative of one fact: in violent neighborhoods students are usually safer in school than out of school. A skilled set of professionals can make it so kids are safe in school, safe enough to focus on school. And it's much harder to bring peace to a violent neighborhood without involving schools. What happens inside the classroom can change the conversations that happen outside school boundaries, but there are no guarantees. What if Fenger had not been the target of a turnaround effort: would Albert still be alive? I don't know. 

Update (October 7): More on MSNBC, and more focused, on the rearrangement of enrollment patterns.

October 2, 2009

Whitmire and Rotherham fall prey to faux-trend fallacy

If I were a union activist without an historical perspective, I'd say ouch with Richard Whitmire and Andy Rotherham's WSJ opinion piece proclaiming a trend in news reporting on education and teachers unions. Or, to put it another way, there's an op-ed in Rupert Murdoch's new plaything proclaiming that a dying industry is giving birth to another trendlet about something, and we should therefore pay enormous attention?

Pardon my skepticism, but since I cut my teeth on documenting how one issue became a visible, recognized social problem in the 1960s (and the broader historical picture thereof), I have at least a little background to comment. I'd be cautious of making much of a handful of stories in the New York Times, the New Yorker, and the Washington Post. Most of the issues Whitmire and Rotherham mention have had small blips of attention over the past few decades, and you would expect there to be such blips in any year simply by random chance, with the only question being what issues pop up on the collective radar screen of journalists. 

The bigger issues here are ... well, the bigger issues, or long-term trends in coverage. I'd wait a year or two to see if there is still a trend that anyone thinks is starting now.

September 24, 2009

Students can study more than one subject at a time

The future of our nation and world depend on our citizens' understanding of both how they interact with each other and how they interact with the natural world [emphasis added].
--FSU physicist Paul Cottle, responding to a critic who thinks we should be more worried about civics knowledge

As an historian, I agree with Cottle. I want my fellow citizens to have some grounding in more than one subject. I want my neighbors to know that a cookie contains more energy than an equivalent mass of TNT. And I want my neighbors to understand that Jehovah's Witnesses were the plaintiffs in the landmark 1943 case striking down a West Virginia law that mandated students say the pledge of allegiance. And I want them to have those small bits as part of a larger context from each discipline. I think it's possible to hold ideas in one's head from more than one discipline. More than possible: necessary.

Unfortunately, no one has yet taught high school students -- or college professors -- how to answer 80 e-mails in five minutes, which is why I now need to turn to my inbox instead of blogging about anything in depth. 134 e-mails. Yikes.

September 17, 2009

Serious science toy for the International Year of Astronomy: the Galileoscope

Galileoscope
Galileoscope (not the ones in my house)

I wasn't going to mention it here until I got my hands on it, but the two Galileoscopes I ordered in the summer finally made their way to my house yesterday, so I can now tell you all: they're shipping! They're real! One is for us and the other is for a young man we hope to surprise with it (and not upset his parents by giving him an excuse to stay up far, far too late, as well as insisting that they drive him somewhere without light pollution).

These 25x telescopes were designed for the International Year of Astronomy (2009) to have much better optics than telescopes of similar cost ($20 per for small orders), and while I cannot vouch for their qualities (yet!), I am definitely looking forward to putting this together, putting it on a tripod, and looking up at the sky this winter (when Floridians can stargaze with some reliability). The people who are behind this project are dearly hoping that this will give kids all over the world an experience that helps teach them science and inspires some to go into science. I hope they're right!

The following is an image of the moon through a prototype of the Galileoscope:

A great site for astronomy photos if you can't stargaze today: APOD, or the Astronomy Picture of the Day. Today's is absolutely amazing (and that's going to be true no matter when you read this entry).

September 16, 2009

Pork-barrel airplane hangars or science?

Florida State University physics professor Paul Cottle has a wonderful way of comparing the state's science-education needs with what the state legislature's pork-barrel politics has produced in the recent past: think about the $6 million cost of the airplane hangar former House Speaker Ray Sansom stuffed into the budget a few years ago pretending to be a classroom building for Okaloosa-Walton Community College, and then figure out how many "Sansom Airplane Hangars" would be the equivalent of different options for science education. Cottle's list includes the following:

  • 4 Sansom Airplane Hangers = a comprehensive set of end-of-course science exams for high school
  • 5 Sansom Airplane Hangars = comprehensive professional development for science teachers across Florida

I'm biased: I have two teenagers in high school, I hate when politics distorts higher-ed governance, and want the state's economy to rely on more than tourism, cattle, and real-estate bubbles.

September 15, 2009

And now, Harvard digs deep in the public interest--NOT!

Is there anyone else who read of Harvard's new tuition-free doctorate in ed leadership supported by the Wallace Foundation and first thought, "Oh, that's in competition with the Broad leadership indoctrination inbreeding mutual backscratching society training"? I know what faculty and administrators thought: if there's a (reputational) market for a tuition-free, glitzy finishing school for superintendents, why shouldn't Harvard get in the game? 

Reality-check request

I have what should be a more long-lasting podcast that I'm starting for both of my classes this semester. It'll be a set of historical perspectives on education news, and it should be public-access, though it's hosted on a walled-garden CMS. Right now there are three podcasts, and I'd appreciate if someone could try to add it to your podcast aggregator (whether iTunes or something else) and let me know if you can grab the episodes.

September 11, 2009

Health insurance reform and college completion

I have yet to see anyone discuss the obvious relationship I see between health-insurance reform and education policy: health-related financial crises and college completion problems. There are many reasons why students leave or switch colleges, but one of them is the financial fallout from health crises of students and their immediate family members. Over the years, I have known about a number of students who have either cut back from full-time to part-time status or left college entirely because someone got sick (either them or a relative) and that left a huge hole in the family budget. Especially for first-generation, older students, many of whom have children, many of whom are going to college to escape the dead-end, no/low-benefits jobs they're currently in, this is a nasty catch-22.

I do not know the extent of the problem, but the discussion within higher ed often runs something like this:

  • We know some students drop out because of health problems, either directly or indirectly from the financial fallout.
  • We have college-sponsored insurance plans, but either the premium or the coverage is lousy because the only people in the pool are the people most at risk of having problems.
  • Let's recommend mandatory health insurance for all students!
  • Oh, shoot -- the legislature is telling us we can't, in part because we're already in a financial hole and can't subsidize poor students.

That's what happened in Florida: one university started discussions about mandatory insurance, another (Florida State) took the lead and mandated insurance, a statewide group at the university level continued discussions, and the legislature (this year) banned any university but the first mover (FSU) from mandating insurance.

I don't know the exact extent of the problem, but this is one of the reasons why I am bewildered that major business groups continue to oppose health-insurance reform that would create nearly-universal coverage. With assurance of coverage, people can go out on a limb and start new businesses, something that business groups always claim they want to promote. With the dramatic reduction of health-induced bankruptcy and financial crises, more people would complete college, something business groups say they want.

September 9, 2009

Bowling for senators, dominoes falling, or some other inapt metaphor

Alyson Klein is reporting that Chris Dodd is probably staying with the banking committee, which leaves the Health, Education, Labor, and Pensions (HELP) chair to either Tom Harkin or (if he doesn't take it) Barbara Mikulski. While the rest of the world (rightly) worries about Max Baucus's control of/chokehold on health insurance reform, the HELP committee is where Ted Kennedy held sway. For those who doubt the influence of the Kennedy family on federal education policy, ask yourself who started substantial federal funding in both education for students with disabilities and research into special education, who created the Special Olympics, and who insisted on annual tests in ESEA originally, as well as who helped shepherd many education bills through Congress in the 1990s and in this decade. The most influential single politician in education policy in the 20th century may have been Lyndon Johnson, but the most influential family was definitely the Kennedy clan.

So there are some mighty big shoes to fill, and as important as those shoes, some interesting relationships to negotiate. As many people have said for months regarding health care policy, Kennedy had enormous negotiating credibility, and whoever replaces him will have to work very differently, in addition to having different priorities and ideas. I am going to assume that some version of health insurance reform will pass this fall, and the primary question about education policy mid-fall will be whether higher-ed policy or K-12 policy is more urgent, and how those issues play out vis-a-vis other legislative priorities.

If Harkin decides to drop the leadership of Agriculture for HELP, he is going to be a tough negotiator for the White House on NCLB/ESEA. Part of that comes from what I gather is a close relationship with Iowa's teachers, and part is probably from his status as a rural-state legislator (and wanting to protect federal funding that would normally go to Iowa in formula-driven processes but might be diverted in a discretionary program). In the end, I suspect he'll stick with Agriculture because he is from Iowa, but even if he is not the HELP chair, he'll still exert influence because of his seniority and the way the Senate works.

I don't know much about Mikulski and education policy, except that from her senate issues site, her focus is on higher education (and that is consistent with whatever vague impression I've had). If she were HELP chair, she'd probably push the higher-ed priorities, probably leading to K-12 issues (NCLB/ESEA) being left for late in 2010 or even 2011.

This just in: the Baltimore Sun's Paul West is reporting on the paper's political blog that Harkin is hopping from Agriculture to HELP. Wow. If that report is true, the USDOE's legislative liaison job just got about three times more interesting.

September 8, 2009

Rebuttal to the president and the extension of the silly season

One of my Chicago friends, author and professional smart aleck Adam Selzer, has the best rebuttal to the President's school comments today. The theme? "Responsibility is for squares. Slack off, buster!" (taken from one of Adam's FB notes--not in his rebuttal, but it captures the essence)

In the nutty wing of the loyal opposition, the next criticisms of the president:

  • Republican Party of Florida chair Jim Greer will criticize Obama for "socialist breathing." Then, when he is presented a tape of what Obama actually sounds like when asleep, will proclaim, "Oh, it's okay breathing, but he changed his respiration to acceptable, American breathing after my criticism!"
  • Radio/tv show host Glenn Beck will attack Bo as "the most racist dog in America, or at least the most vicious Portguese water dog in America, and probably an illegal alien as well. Maybe a Muslim terrorist illegal alien Portguese water dog who is waiting for us all to fall asleep."

The most serious (and legitimate) criticism I expect of the speech: too long. He was lucky he was in front of high school students, though fourth-graders would also have sat for the entire speech. But kindergarten students or seventh graders? He'd have had no chance.

September 5, 2009

Alterknit educashun polticks

In the alternate reality where pundits and talk-show hosts live, Barack Obama is not going to say Tuesday that students have a personal responsibility to work hard in school. Somehow this is indoctrination, or maybe unseemly politicking for all the 7-year-olds who will be voting in 2012. Though Richard Whitmire and many others are scratching their heads on this one, putative social conservatives evidently don't want to echo the fundamentally conservative point Obama will make: put yourselves to work and get a stake in society. 

Do I really need to explain why the paranoid style of education politics is supremely nutty?

September 2, 2009

"Lake Wobegon" Klein

From pp. 68-69 of Accountability Frankenstein:

The complexity of an accountability system can also help muffle opposition to accountability if it gives a reasonable chance for students or schools to be successful in the system's labeling... the political potential to muffle opposition within a system may be more important than the technical qualities of a system, for schools typically trumpet any positive label on any website, pamphlet, or streetside marquis. All three of these states provide evidence of the capacity for complex systems to muffle dissent. In North Carolina, the majority of schools have received some recognition award in every single year of its accountability system's history. In Florida's system, 13% earned recognition in its first year, 1999, but that proportion rapidly grew, and a majority of schools received recognition awards in each of the years from 2003 to 2006. In California, 47% of California's schools earned statewide recognition in 2002, and two thirds of the schools in the Los Angeles Unified School District earned recognition.

I don't know why anyone would suspect that there is any political convenience involved in having the single letter grades assigned to a whole slew of NYC schools jump to A, but it's not isolated to New York. It's just that New York has overtaken Lake Wobegon as a symbol of overestimation of results. Then again, since Garrison Keillor spends several months a year in New York, maybe it's highly appropriate.

August 31, 2009

Structure, choice, or just good books to read?

Several years ago on the prompting of then-Governor Jeb Bush, Florida's legislature mandated reading classes for middle-school students and ninth graders. On the first day of seventh grade a few years ago, a teenager I know discovered that his reading teacher had assigned Stephen Covey's The Seven Habits of Highly Effective Teens. Yes, this was a stellar choice as mandated reading for the hard-bitten cynical set: time-management skills. You can imagine the snide comments from students ("great work of fiction"), and it was truly a double facepalm moment in the annals of reading instruction.

So when debate erupts over the New York Times "we have no news to report so we will make up a trend" story on letting students pick their own books, I hope you will pardon my bewilderment. I am not too old to remember the structure of middle-school English classes: you read some books together as a class, and then you read others on your own and wrote a book report. In my day, the "let's try to engage the students" ploy was letting students create dioramas. Today, I guess it's keeping a reading journal. Those are both fine as long as teachers understand that no trick-du-jour works with all students. 

There is a role both for mandated reading and choice, and maybe 36 weeks of school allow both, as long as there is a reasonable chance that the mandated books will not be treacly. Less Stephen Covey, more Gordon Korman: "Go to the library and pick out a book with an award sticker and a dog on the cover. Trust me, that dog is going down."

August 30, 2009

Race to the Top comment sausage

A friend of mine from Chicago introduced me to the term link sausage as a blog entry that is not much more than a set of links. Here are links to various comments on Race to the Top (a tiny slice of the well-over-thousand comments submitted):

As I expected, others have started to chime in on the NEA comments. The New York Times took the comments as a sign of obstinance. Former Park Ridge Education Association president Fred Klonsky wrote,

While it seems to me that it is late in coming, the letter from Brilliant is well deserved, and [Sherman] Dorn's comments notwithstanding, I think it reflects the views of the NEA membership. At least among those who have been following the debate.

I think that was my point: the comments reflected the views of a large slice of the NEA membership, but not in a productive fashion, and I fear that on balance it will harm the concrete interests of teachers (both in and out of the NEA) no matter how you want to define those interests. 

Note: As Klosnky points out in comments, he's not an ex-president (yet). The error is all mine in sloppy reading of his about page.

August 28, 2009

I'm commenting on Race to the Top, and I want a pony, too!

Impressions of a quick skim of 20 or so comments on the draft Race to the Top regs:

  • I couldn't find the national AFT comments anywhere.
  • Thus far, the two sets of technical comments by the Learning Disabilities Association of America and the group of academics with Kane, Staiger, and several others (uploaded by Thomas Kane), respectively, earn my "okay, you guys read the regulations and targeted your comments" award. Whether you agree with them or not, the comments were shrewd and focused. (I happen to like most of the comments, which are practical and sensible on the whole.)
  • The New Teacher Project signed onto the multi-organization letter that was essentially a vague "okay, we agree with this" note (with the advice for the USDOE to be selective in the first round), and then submitted comments that were, ahem, not nearly as far in the opposite direction as NEA but bewildering in its unbridled confidence in the suggestions made. TNTP staff, please read the comments written by Kane et al. You're smart, and they're smart, and they're much closer to the mark than you were this week. At least you don't come close to winning the second "I'm commenting on Race to the Top, and I want a pony, too!" award (first was to the NEA). 
  • I think that the California Teachers Association (the NEA affiliate in California) avoided the factual blunder in the NEA comments of asserting that Race to the Top is a mandate. Instead, they asked what states would have to give up in return for the money. In this case, they were deeply, deeply concerned with the threat to federalism embedded in asking that a state be able to link teacher and student records. That would be more plausible if TNTP's comments were enacted, but either the draft regs or the Kane et al.'s suggestions are reasonable in an imperfect world.
  • One state department of education accidentally sent the USDOE its cover letter to a national organization telling the national organization it was sharing its reg comments, in the place where it was supposed to upload comments. No signs of actual comments on the regs (thus far today). Ouch! I suspect there are similar technical glitches in other places.

I didn't comment. This is the first week of classes, and I'm a firm believer in the biggest bang for my buck (or hour).

Greg Mankiw provides the laugh of the day

Economist Greg Mankiw provides the unintentional humor of the day: "Smart parents make more money and pass those good genes on to their offspring."

Smart parents years ago miraculously picked employers who survived the Great Recession without laying them off?

Smart parents are dumb enough to spend money on expensive private schools and expensive private test-prep services when according to Mankiw's claim their kids would do well anyway?

Smart parents who choose public schools are dumb enough to spend far too much for houses in wealthy areas because it's really not necessary for their kids to have a decent education?

Historical perspective: I think that Greg Mankiw is living in the past, a time when wealthy people would accept the argument that they're wealthy because they're smart rather than the argument that they're wealthy because they work their tails off. The second that we became a workaholic society, the arguments of Charles Murray, Greg Mankiw, and the like became dinoideologies. Wealthy people no longer need to argue that their wealth derives from their being smarter than other people in the sense of algorithmic cognition. And it's been years since I've heard any of that crap from actual wealthy people who don't fancy themselves as part of the chattering class. They and their close admirers will talk about their being "whip-smart," sure, but also working very, very hard and having the luck to have good mentors, the right opportunity at the right time, and so forth. 

August 23, 2009

NEA's comments: righteousness over responsibility to members?

I'm an NEA member, through my membership in the United Faculty of Florida. I'm a skeptic and critic of high-stakes accountability. Wrote a book and a few articles on the topic. And I am astounded at the NEA's comments on the Race to the Top draft regulations. (Hat tip.)

It is one thing to submit a righteous objection to the entire program if you are an individual with no responsibilities but to your conscience and your personal judgment of posterity. It is an entirely different thing when you represent several million teachers and you submit a document that for all intents and purposes appears to have an internal audience inside the NEA. That's nice, in the worst sense of the word "nice," because NEA staff had a responsibility to protect and advance their members' interests, not indulge any of our fantasies. To put it bluntly, on what planet would this regulatory comment have any effect on the final regs?

Let me be clear on my perspective as an NEA member and as an observer of political processes: There are lots of reasonable individual passages within the document, but you don't submit a manifesto when you comment on regs as an organization. You don't submit a manifesto that covers up any potential for effectiveness with what amounts to political poison. And you don't submit a manifesto that undermines your credibility. 

Two examples will have to suffice, because there's only so much I can wince at publicly: "we cannot support yet another layer of federal mandates" (from p. 2), or with regard to the creation of statewide longitudinal data systems, opposition to "[i]gnoring states' rights to enact their own laws and constitutions" (p. 24). The problem with these claims (and attendant tone of outrage) is that Race to the Top is not a mandate. Love it or hate it, it's something states must apply for. 

There were certainly alternatives available to the NEA, including the following choices:

  • Realpolitik: nudge the regs a bit to help state and local affiliates.
  • Legal: set up a legal challenge after final publication.
  • Abstinence: if you need to make a statement of conscience, declare that "we have serious doubts that this program will substantially help schools and will not participate in the regulatory comment process." 

I may be dead wrong about this, and there may be some uber-secret strategy behind this comment, but from where I sit at the end of the summer, it looks like one of my national affiliates' new president's first major move has been a bunch of wasted electrons.

August 17, 2009

The principal as eunuch?

I will confess to tremendous confusion about the actual skill level of principals, especially after reading a bunch of stuff about teacher evaluation and teacher labor markets (and whether principals should be able to hire teachers directly). Many of those who criticize the current state of teacher evaluation note that principals usually are poorly trained in evaluation, engage in drive-by observations, and give satisfactory ratings to almost every teacher. Yet the same people who criticize principals' evaluation of teachers are often in favor of dramatic autonomy for principals in selecting teachers. I am confused: principals are incompetent at evaluating teachers, so we should give as much authority to them as possible in hiring? 

In the comments on my entry about teacher evaluation policy debate ground rules, New York math teacher Jonathan writes, "Our current evaluation does not work well because administrators implement it poorly, not because it is inherently flawed. Why should teachers be punished (for that's what the refomers' schemes would do) for something that people other than teachers have messed up." And then he proposes a vague transition to holistic review of teachers. Again, I'm confused: principals have screwed up massively, so let's move from a checklist to an unstructured process where they'll be important partners?

The common phrasing here sounds remarkably like rhetoric that frames the principal as simultaneously a doofus and an entrepreneur, someone in a 1980s suburban-teen flick or a turnaround artist, a petty tyrant or a serious partner in collaboration, or, let's face it, a cipher moldable into whatever you think is necessary at the moment. In the early 1990s, Lynn Beck and Joe Murphy wrote a book about the changing metaphors used for principals, and I'm wondering how to think about the inconsistencies in how people write about principals. They should be functionaries in a giant bureaucracy aimed at achievement, I guess, and they can rise as far as their talents take them, but not to the top, which should be reserved for visionaries, and they can be trusted with all sorts of tasks.

The principal as eunuch, in other words.

It's probably a saner goal to see principals as human beings with a certain set of skills, and like all of us, that set of skills can be impressive but is always finite. The concerns I always have about systems and proposals that rely heavily on a single role within a school is that people are variable, including principals. Setting up the principal as a hero is no better than setting up the teacher as a hero.

August 16, 2009

What "multiple measures" looks like in reality

Friday's Sun-Sentinel article on the new evaluation scale for Florida high schools shows what happens when a state moves away from general-assessment test scores as the end-all and be-all of accountability. In this case, Florida's new scale for high schools rewards schools for graduating more students, especially those who have problems with the state assessments, for enrolling students in challenging courses, for students who succeed in the challenging courses, and for student success in voc-ed certification programs.

How are Broward County schools responding?

At South Broward High School in Hollywood, students will get the chance to take additional AP classes, such as human geography, world history, music theory and macroeconomics, in addition to more traditional offerings such as AP English and biology, said principal Alan Strauss.

They're also ready to better monitor performance of at-risk students and ensure the entire senior class is ready to graduate, Strauss said. "I say overall I would hold myself accountable for grad rate and preparing my kids for college," Strauss said. "I don't find a problem with that. I think that's what my job should be."

Surprise, surprise! A more balanced accountability mechanism leads to planning a more balanced set of programs for students. I can quibble with loads of details on the new scale, but the direction is the right one, and I think we'll know in a few years how this is going. I'll stick my neck out and predict the evidence will be reasonably good (in terms of outcomes). A small step for a single state, a giant step for accountability options.

August 13, 2009

How can we use bad measures in decisionmaking?

I had about 20 minutes of between-events time this morning and used it to catch up on two interesting papers on value-added assessment and teacher evaluation--the Jesse Rothstein piece using North Carolina data and the Koedel-Betts replication-and-more with San Diego data. 

Speaking very roughly, Rothstein used a clever falsification test: if the assignment of students to fifth grade is random, then you shouldn't be able to use fifth-grade teachers to predict test-score gains in fourth grade. At least with the set of data he used in North Carolina, you could predict a good chunk of the variation in fourth-grade test gains knowing who the fifth grade teachers were, which means that a central assumption of many value-added models is problematic.

Cory Koedel and Julian Betts's paper replicated and extended the analysis using data from San Diego. They were able to confirm with different data that using a single year's worth of data led to severe problems with the assumption of close-to-random assignment. They also claimed that using more than one year's worth of data smoothed out the problems.

Apart from the specifics of this new aspect of the value-added measure debate, it pushed my nose once again into the fact that any accountability system has to address the fact of messy data.


Let's face it: we will never have data that are so accurate that we can worry about whether the basis for a measure is cesium or ytterbium. Generally, the rhetoric around accountability systems has been either "well, they're good enough and better than not acting" or "toss out anything with flaws," though we're getting some new approaches, or rather older approaches introduced into national debate, as with the June Broader, Bolder Approach paper and this morning's paper on accountability from the Education Equality Project.

Now that we have the response by the Education Equality Project to the Broader, Approach on accountability more specifically, we can see the nature of the debate taking shape. Broader, Bolder is pushing testing-and-inspections, while Education Equality is pushing value-added measures. Incidentally, or perhaps not, the EEP report mentioned Diane Ravitch in four paragraphs (the same number of paragraphs I spotted with references to President Obama) while including this backhanded, unfootnoted reference to the Broader, Bolder Approach:

While many of these same advocates criticize both the quality and utility of current math and reading assessments in state accountability systems, they are curiously blithe about the ability of states and districts to create a multi-billion dollar system of trained inspectors--who would be responsible for equitably assessing the nation's 95,000 schools on a regular basis on nearly every dimension of school performance imaginable, no matter how ill-defined.

I find it telling that the Education Equality Project folks couldn't bring themselves to acknowledge the Broader, Bolder Approach openly or the work of others on inspection systems (such as Thomas Wilson). Listen up, EEP folks: Acknowledging the work of others is essentially a requirement for debate these days. Ignoring the work of your intellectual opponents is not the best way to maintain your own credibility. I understand the politics: the references to Ravitch indicate that EEP (and Klein) see her as a much bigger threat than Broader, Bolder. This is a perfect setup for Ravitch's new book, whose title is modeled after Jane Jacobs's fight with Robert Moses. So I don't think in the end that the EEP gang is doing themselves much of a favor by ignoring BBA.

Let's return to the substance: is there a way to think coherently about using mediocre data that exist while acknowledging we need better systems and working towards them? I think the answer is yes, especially if you divide the messiness of test data into separate problems (which are not exhaustive categories but are my first stab at this): problems when data cover a too-small part of what's important in schooling, and problems when the data are of questionable trustworthiness.

Data that cover too little

As Daniel Koretz explains, no test currently in existence can measure everything in the curriculum. The circumscribed nature of any assessment may be tied to the format of a test (a paper and pencil test cannot assess the ability to look through a microscope and identify what's on a slide), to test specifications (which limits what a test measures within a subject), or to subjects covered by a testing system. Some of the options:

  • Don't worry. Don't worry about or dismiss the possibility of a narrowed curriculum. Advantage: simple. Easy to spin in a political context. Disadvantage: does not comport with the concerns of millions of parents concerned about a narrowed curriculum.
  • Toss. Decide that the negative consequences of accountability outweigh any use of limited-purpose testing. Advantage: simple. Easy to spin in a political context. Disadvantage: does not comport with the concerns of millions of parents concerned about the quality of their children's schooling.
  • Supplement. Add more information, either by expanding the testing or by expanding the sources of information. Advantage: easy to justify in the abstract. Disadvantages: requires more spending for assessment purposes, either for testing or for the type of inspection system Wilson and BBA advocate (though inspections are not nearly as expensive as the EEP report claims without a shred of evidence). If the supplementation proposal is for more testing, this will concern some proportion of parents who do not like the extent of testing as it currently exists.

Data that are of questionable trustworthiness

I'm using the term trustworthiness instead of reliability because the latter is a term of art in measurement, and I mean the category to address how accurately a particular measure tells us something about student outcomes or any plausible causal connection to programs or personnel. There are a number of reasons why we would not trust a particular measure to be an accurate picture of what happens in a school, ranging from test conditions or technical problems to test-specification predictability (i.e., teaching to the test over several years) and the global questions of causality.

The debate about value-added measures is part of a longer discussion about the trustworthiness of test scores as an indication of teacher quality and a response to arguments that status indicators are neither a fair nor accurate way to judge teachers who may have very different types of students. What we're learning is a confirmation of what I wrote almost 4 years ago: as Harvey Goldstein would say, growth models are not the Holy Grail of assessment. Since there is no Holy Grail of measurement, how do we use data that we know are of limited trustworthiness (even if we don't know in advance exactly what those limits are)?

  • Don't worry. Don't worry about or dismiss the possibility of making the wrong decision from untrustworthy data. Advantage: simple. Easy to spin in a political context. Disadvantage: does not comport with the credibility problems of historical error in testing and the considerable research on the limits of test scores.
  • Toss. Decide that the flaws of testing outweigh any use of messy data. Advantage: simple in concept. Easy to spin in a political context. Easy to argue if it's a partial toss justified for technical reasons (e.g., small numbers of students tested). Disadvantage: does not comport with the concerns of millions of parents concerned about the quality of their children's schooling. More difficult in practice if it's a partial toss (i.e., if you toss some data because a student is an English language learner, because of small numbers tested, or for other reasons).
  • Make a new model. Growth (value-added) models are the prime example of changing a formula in response to concerns about trustworthiness (in this case, global issues about achievement status measures). Advantage: makes sense in the abstract. Disadvantage: more complicated models can undermine both transparency and understanding, and claims about superiority of different models become more difficult to evaluate as the models become more complex. There ain't no such thing* as a perfect model specification.
  • Retest, recalculate, or continue to accumulate data until you have trustworthy data. Treat testing as the equivalent of a blood-pressure measurement: if you suspect that a measurement is not to be trusted, take the blood pressure test the student again in a few minutes months/another year. Advantage: can wave hands broadly and talk about "multiple years of data" and refer to some research on multiple years of data. Disadvantage: Retesting/reassessment works best with a certain density of data points, and the critical density will depend on context. This works with some versions of formative assessment, where one questionable datum can be balanced out by longer trends. It's more problematic with annual testing, for a variety of reasons, though that can reduce uncertainties. 
  • Model the trustworthiness as a formal uncertainty. Decide that information is usable if there is a way to accommodate the mess. Advantage: makes sense in the abstract. Disadvantage: The choices are not easy, and there are consequences of the way of modeling uncertainty you choose: adjusting cut scores/data presentation by measurement/standard errors, using fuzzy-set algorithms, Bayesian reasoning, or political mechanisms to reduce the influence of a specific measure when trustworthiness decreases.

Even if you haven't read Accountability Frankenstein or other entries on this blog, you have probably already sussed out my view that both "don't worry" and "toss" are poor choices in addressing messy data. All other options should be on the table, usable for different circumstances and in different ways. Least explored? The last idea, modeling trustworthiness problems as formal uncertainty. I'm going to part from measurement researchers and say that the modeling should go beyond standard errors and measurement errors, or rather head in a different direction. There is no way to use standard errors or measurement errors to address issues of trustworthiness that go beyond sampling and reliability issues, or to structure a process to balance the inherently value-laden and political issues involved here. 

The difficulty in looking coldly at messy and mediocre data generally revolves around the human tendency to prefer impressions of confidence and certainty over uncertainty, even when a rational examination and background knowledge should lead one to recognize the problems in trusting a set of data. One side of that coin is an emphasis on point estimates and firmly-drawn classification lines. The other side is to decide that one should entirely ignore messy and mediocre data because of the flaws. Neither is an appropriate response to the problem.

* A literary reference, not an illiteracism.

August 12, 2009

Belated kudos to Broader, Bolder and to Fordham

In the whirlwind of my obligations this year, my reading has lagged, and I am late in recommending and praising two reports published in the first half of 2009:

  • The Broader, Bolder Approach's accountability report, published in late June. This report suggests combining the use of achievement test data and on-site school inspections for school-level accountability. For those who have read Accountability Frankenstein, you'll know that I agree with those ideas. This report addresses the central gap in the original Broader, Bolder manifesto, and I am delighted to have read the proposal.
  • In March, the Fordham Institute published a report recommending a scaled approach to accountability when private schools take public dollars. Their proposal is roughly that the more dependent a private school is on public funding, the more the school has to provide data and be accountable in a way similar or parallel to local public schools.

Both are thoughtful, well-reasoned brief arguments, and they move each debate in interesting directions. Whether or not you agree with the conclusions, you'll have things to think about.

Updated: Aaaaargh! Six days later, I realize I've been calling the group the Bolder, Broader Approach instead of the other way around. Dear readers: when I make a stupid error, please point it out as soon as you see it.

Proposed ground rules on teacher evaluation and test discussion

Seeing how too many writers about Race to the Top, tests, and teacher evaluation would have taken actions in the Cuban Missile Crisis that would have led to nuclear war--i.e., seeing the worst in opponents, or maybe seeing posturing as the best path forward for themselves personally or for their positions (sound like the health-care debate-cum-food-fight?)--I am hereby proposing the following ground rules/stipulations:

  1. The modal forms of teacher evaluation used in K-12 schools are not useful.
  2. Some aspect of student performance (abstracted from all measurement questions and concerns about flawed tests) should matter in teacher evaluation.
  3. At least one problem of including student performance in teacher evaluation is how to use messy and flawed data. This comes from the fact that current tests are flawed. Heck, all tests are going to be imperfect and create the dilemma that Diane Ravitch referred to this morning. But plenty of today's tests should embarrass anyone who approved their use.
  4. Yes, people who disagree with you have used inane arguments, and some of them might even have gotten some provisions through a legislature by logrolling. I know I can say the same about your putative allies. Let's call each other out on those moves, and then move on to the substantive issues. Doing more than calling people out on that at the time (i.e., holding grudges) is playing the game of "your side is dirtier than mine," and you will inevitably lose that game, especially if there's an historian in the room (and in addition to me, there's also Diane Ravitch, Larry Cuban, Maris Vinovskis, and others who can quickly point out where folks have played dirty political pool for decades, though many of us will just call it the standard operating procedure in education politics). See reference above to Cuban Missile Crisis. If Reagan make an arms-control treaty with Gorbachev, we can all be a little more mature in disagreements.
Anyone who has broken these ground rules or is going to break the ground rules in the near future is currently in a grace period thanks to my staying away from blogging much in the past few weeks. But if I have time in the fall, I'll write a weekly entry on who's doing the best and worst jobs of fighting fairly on this issue.

August 4, 2009

Your personal, homemade commission on tenure and test scores

Sick of finger-pointing in the absence of a New York state commission to study how to use test scores in teacher evaluation (including tenure) decisions? Look no further! In this space, we will be conducting our own homegrown commission over the next three months. No need for the New York Assembly and Senate to act! We'll do it ourselves.

What? you say. You're in Florida. Well, yes, but everyone knows that Florida is just the Southern branch of New York. My father grew up on Flatbush Avenue and graduated from Lincoln High School. He was in New York City for his residency in pediatrics (with an office in Bellevue, but that's another story). The Yankees' spring training home? Eight miles from my house. 

And if that doesn't convince you, you should know that Alexander Russo runs his blog on Chicago schools from Long Island. If he can do that, I can run a citizens' commission for New York from here (and then someone in Chicago can run something in Florida).

Apply in comments: name, role in New York education, what you'll bring to the table.

August 2, 2009

The liberal arts and narratives of declension

There is a teacher's voice in my head, asking the logical question of New York Times reporter Patricia Cohen who speculated whether the humanities are in decline (perhaps because of the Great Recession) and whether older history subdisciplines are also in decline: "where did she go to school, and who were her teachers?" Evidently, the Times is hiring reporters who either never had good history teachers, never paid attention to them, or forgot one of the basic lessons in a good college history class: beware narratives of climbing societies, falling societies, or any society-wide "rise and fall." The February article brought the expected number of letters to the editor to a newspaper that might just depend on readers who want to read (you know, that humanities-ish activity), Timothy Burke had some words, and Michael Berube had solid things to say in early June and late June. About the second article, again see Burke as well as Mary Dudziak, Mark Grimsley, Claire Potter, and David Silbey. I am months late on this, so I will do what I can.

First, before panicking it probably makes sense to divide what parts of the proportionate decline of humanities majors in the past few decades are attributable to different factors: the growth of undergraduate professional degrees, the growth of higher-education enrollments more generally, the decline of GI Bill-related enrollment as a proportion of undergraduates, and any leftover changes that just might be related to the nature of the disciplines themselves. In part because the expansion of higher education came side-by-side with the belief that a college degree's main utility was getting a job and growing credential requirements for jobs, enrollment grew faster in professional majors than in the humanities.

Maybe I should cry over the fact that a lower proportion of students are history majors than there used to be (though the percentages bounce up and down), or maybe I should celebrate the dramatic expansion of college attendance in the past 70 years and the fact that even if the proportion of history majors has dropped, there are still more graduates with history majors living in the country today than were living in 1950. Remember new "old saw" about the total population of China and India; apply as balm to humanities woes. Not only does the general expansion of college attendance make me less concerned than others are, but my guess is that they're more likely to be exposed to teaching that asks important historiographical questions and that uses primary sources. I didn't say immersed in: exposed. 

Those perspectives do not completely eliminate concern about the future of humanities teaching and humanities departments in colleges and universities. Though regionally accredited colleges and universities have some version of a distribution/breadth requirement or general-education program (depending on your regional accreditor), that fact does not mean that a department has to be anything more than a "service outlet," the higher-ed equivalent of the quick-lube shop tucked in between the strip malls of Finance and Psychology. "Shakespeare while u wait! Fulfill writing requirement in 30 mins or ur money back." 

On the other hand, while the standard choices of academe has been for greater adjunct use in all high-student areas (and that is true whether they're called adjuncts or graduate students), the reality is that humanities classes are cheap in comparison with science and math if one looks at course credit earned. High failures rates in algebra and the costs of maintaining labs add up in a pragmatic sense, and that's only looking at credit courses. What about community college remedial classes? As DeanDad has noted, developmental courses in math are a death march in comparison with other noncredit classes. Teaching-heavy institutions may short the humanities in individual places, but the combination of gen-ed/distribution requirements makes it virtually impossible for college students to graduate without some liberal-arts classes and thus virtually impossible for colleges to eliminate liberal-arts programs entirely.

And then, if you look at the costs of maintaining the research capacity of faculty, the humanities look even better: no lab animals to house, fewer research assistants to hire, and the primary need for many scholars is a computer, some travel funds for conferences or research trips, and time. The big difference is in universities with doctoral programs, where the expectation of support for doctoral students has both direct costs (tuition waivers, which are on top of the pitiful stipends for TAs and RAs) and also indirect costs (in terms of the classes that graduate faculty are not teaching while they are running seminars and advising students). What I'm seeing in Florida universities is a combination of closing small doctoral programs as well as some atrocious decisions about closing departments. 

The probable consequence of the first type of decisions--closing down small doctoral programs in the liberal arts and in other areas--is a change in the doctoral-education opportunities in those fields, somewhat different workloads for those faculty, and perhaps a bit of status shift back to traditionally-elite programs. It's not as though small-program closures is going to bump the publication trends in any significant manner, and Cohen's articles presume that the rolling crisis in academic publishing is in an entirely different universe from the mythical status decline she posits. In her February article, the world of publishing is entirely ignored, and the June article only discusses a presumed shift in journal publishing. In the real world where I live, as opposed to the make-believe world of the New York Times reporter, the long-term crisis in the liberal arts is in academic publishing and questions about the economics of monographs and the long-form argument.

(Among the atrocious departmental closure decisions, the University of Central Florida almost shut down its statistics department the same year it's opening up a new medical school, and Florida Atlantic University reorganized its engineering college into the Department of Tenured Faculty We Like, Department of Tenured Faculty We Hate, Department of Tenure-Track Faculty, and Department of Non-Tenurable Faculty Who Teach Boatloads of Undergraduates. Those weren't the official names of the reorganized units, but that's the central function of the reorganization. Guess which "department" was closed, with the tenured faculty told to leave by August 7?)

July 27, 2009

Talking turkey on "Race to the Top"

The hoopla surrounding the draft "Race to the Top" guidelines have obscured the long-game strategy involved here. If you think about the structure of the funds--more discretionary money than the U.S. Department of Education has ever had before, competitive grant system, and a set of priorities that the Duncan department has been signaling for six months--there are two guesses I have about the broader goals:

  1. The double-shot of grants over the next year is intended to be the first of two or three shots of large amounts of discretionary money for the department.
  2. Duncan's learned about vicarious reinforcement and intends to use it here.

The obvious initial "winners" will be states such as Florida which have a number of the required elements in place and are ready to go on a few payoff projects. But there will also be a few very large states left in the cold (and without that extra funding) after these first two rounds of awards. What if California is one of those states out in the cold? Or New York? There will be local pressure from school boards and administrators on members of Congress to continue feeding money to the department until their states land at least one award.

In the long game, the fact that Race to the Top can't bail California out is not really the issue, and I disagree with Mike Klonsky's assumption that this is an attempt to starve the states into submission. While I think a number of people would have preferred a larger ARRA stimulus fund, I don't think you can claim that the Obama administration has acted at all as if it wants thousands of teachers fired. Far more likely is the ordinary political dynamics of federal programs: no one wants to be without a slice of the pie. For these reasons, if it were legal to place a bet of this kind, I'd give rather interesting odds that California loses out big in the first two swats at Race to the Top money. 

And speaking of misdirected Mikes, Mike Antonucci is wrong about the teachers union dynamics in Race to the Top. While my higher-ed local has both the AFT and NEA as affiliates, I'm generally out of the loop on national headquarters stuff, but I can see the writing on the wall: one of the unions may well push in the regulatory process to increase the leverage of state affiliates, not to eliminate the requirement on linkability of teachers to student data. The best thing that the national affiliates can do is help state affiliates' negotiating position with their own state departments of education. If two states' applications are similar, but only one has a letter of support from their state affiliate's (or affiliates') elected officers, both the NEA and AFT need the state with union support in the application to have an advantage. (There are some interesting dynamics here vis-a-vis merged state affiliates, but the larger incentive at the national level is to help all state affiliates.)

July 25, 2009

Temporizing and teasing on tests and teacher evaluation

I still don't have time to expand at length on combining qualitative and quantitative sources of data for teaching evaluation, but given the hoopla surrounding the draft Race to the Top regulations, I should at least provide an update, or rather a bit of a tease for what's developing into a short paper-to-be. In addition to my fairly general understanding of some technical issues, I'm developing the argument that any point-based system for combining professional judgment and test scores needs to avoid fixed weights for the components of the system.

The explanation is not that technical, and I can sketch it here: the benefit of a truly Bayesian approach to using test scores to evaluate teachers is a reciprocal relationship between the decision-making authority of professional judgment and the power of other data (including test scores). A forceful judgment by professionals reduces the power of test scores in such a system, while tepid judgments increase the power of test scores. That is one possible solution to the thorny question of relative weights: if educators are willing to judge their own, test scores are less important (addressing the concerns of teachers unions and many administrators), but if educators are not willing to judge their own, test scores are more important (addressing the concerned of those criticizing the very low proportion of teachers given poor evaluations). 

In a point-based system with fixed weights (or fixed percentages of the total) assigned to individual components, you don't have a structure with a reciprocal relationship between the exercise of professional judgment and the authority of test-score data. But I think the dynamic benefits of a Bayesian approach can be created in a point system, as long as the weights are not fixed. I need to think through the potential approaches, but it's possible.

There: that's the tease.

July 22, 2009

On the proper state of being bothered

Are you bothered?

Seasonal bother: It's summer in Florida, and if you park a car anywhere outside a meat locker, touching a steering wheel earns you a second-degree burn.

Caffeinated bother: I started the day a little after 7 am at a local coffee shop, grading student papers. My brain fried about 210 minutes later, after a few cups of coffee and my getting to the point where two-thirds of the papers are now read (no, not two-thirds read this morning).

Unreasonable bother 1: I'm at a public library, where a children's program started in one of the library's rooms an hour ago, and one of my fellow (adult) patrons was bothered that there might possibly be a crying child anywhere in the building whom he could hear. (The child was taken out into the hallway reasonably quickly.)

Political/policy bother: Ezra Klein (along with Matthew Yglesias) seems to understand the long-term game of the Obama administration on health care (among other issues). Unfortunately, most reporters still don't get it, about health-care politics or, to pick another random topic except that it's my interest, education politics. It's too much fun to report the latest (wording-dependent) poll results or the latest pronouncements by the diva du jour

Unreasonable bother 2: TMI in the library. You really don't want to know (and neither did I). But in my head and heart, I know that I'd rather be bothered in the public library than not have a public library.

Intellectual bother: The popular philosopher's text by Howson and Urbach on Bayesian reasoning troubles me, less because of its style (which is fine, if dense for us nonphilosophers) or omissions (which I will trust statisticians can correct) than because of the disturbing but sensible point early in the book and that Steven Goodman has described as the p-value fallacy: statistical tests of significance say nothing about the probability of ruling in or out various hypotheses. If I understand Howson and Urbach's analogy between the standard discussion of medical tests and inferential statistics, the conditional probability of any hypothesis (after gathering data) depends not just on the inferential equivalent of false-positive rates (tied to statistical significance and p-values) or the equivalent of false-negative rates (power) but also on the underlying probability of the hypothesis being true. I pondered this last night while cleaning the kitchen, and the small point got under my skin. On what basis would a non-Bayesian (frequentist) respond? If I remember correctly, the easy response is to say, "Ah, a frequentist perspective is close to a Bayesian one with a non-informative prior." Except that the prior for categoricals, even with a non-informative assumption, depends on the number of bins, or hypotheses being tested. I think that the only way out for a frequentist is to either artificially restrict the number of hypotheses or to not care about the number of hypotheses being compared. To answer a question Gene Glass asked me a few years ago, it's just about at this point that my brain begins to dribble out my ears: historians are generally not theoretically minded. 

Unreasonable bother 3: I need to concentrate on an article that's already late, but rewinding to 7 am and having the whole day over again to work on the article as well as grading? Not going to happen.

Why bother: decaf nonfat latte with sugar-free flavoring, no whip.

July 15, 2009

The clinching argument for national curriculum standards

"Let's do it now, before total nuts from Texas take over!" To be truthful, there have also been nuts in New York, Florida, California, and other places (and of various flavors) where a state's size gives enormous temptations to warp the textbook approval process as leverage for controlling the entire country's text market. That's not a 100% clinching argument for some national standards, but it's very tempting to strike while the people holding the irons are definitely not hot.

Crazy idea on teaching induction...

One more idea while my attention wanders from writing the exam for tomorrow morning: why do large school systems rotate administrators on the principle that they "need experience in a range of settings" for leadership purposes and then keep new teachers in the same school year after year? Who needs more rotation through a range of settings? In which case would rotation provide better evidence that student outcomes are not the result of selection effects?

Full-credit answers require coherence, avoidance of tangents, and reference to relevant research. Oops. Sorry. Thinking about that exam, obviously.

Journalistic kudos on NCES NAEP-gap analysis

I agree with Alexander Russo: much of the news coverage of the new NAEP score releases has been sober and nuanced, the stuff that the New York Times can be proud of (as opposed to stuff that prods Brad DeLong). It looks like the Ed Week coverage came too late for Russo's entry.

And now that I have my regular laptop back, I should return to preparing the final for tomorrow morning...

Intellectual baubles

Jay Greene's comment about intellectual fetishes and think-tank cliques had me grinning from ear to ear this morning, in part because of the multiple layers I'm reading into the comment (and that I doubt Greene intended: I use Janice Radway as my excuse to poach) and in part because Heathers is one of my favorite teen movies:

Dismissing policies because they aren't on the agenda of the current majority is like the type of argument heard in the 1988 film, Heathers: "Grow up Heather, bulimia's so '87."

In this case, David Figlio's data-informed hunch aligns with mine: in the long run, the evidence will not show vouchers to improve the achievement of students who use them, and the asymptotic effect will gravitate towards zero. (The potential competitive effect of vouchers is a different research question, one that relatively few rigorous studies have touched, and the evidence is mixed: see Figlio & Rouse, 2006, for the only refereed study of Florida vouchers' competitive effects that I find to be sufficiently rigorous.) As Jeffrey Henig notes, in the long run research can matter, and I suspect only part of the reason why the Fordham Institute is shying away from voucher debates (Greene's instant target) is because it's not politically viable at the moment. If the evidence does not show that vouchers are a smash-bang-up success, it's going to be hard to justify them except on grounds of values, emotions, or political interest. The second and third may provide enough for current voucher programs to survive (see Florida's and DC's voucher politics for variations on a theme), but probably not to expand.

July 13, 2009

AFT QuEST presentation slides on performance pay

I am not in DC, but I do catch things online: the presentation slides for the AFT QuEST session on performance pay are available, and while Edward Tufte thinks Powerpoint is awful, a stack of straightforward, well-written slides provides a wonderful vicarious outline for those of us who Were Not There.

Hechinger Institute hypes the obvious -- this is a role model for reporters??

I received an e-mail advertisement for a "webinar" on "The Dropout Crisis" from the Hechinger Institute. This is an organization that claims it "exists to equip journalists with the knowledge and skills they need to produce fair, accurate and insightful reporting."

Both the e-mail and the webpage for the seminar claim that "new research shows that 17 states produce some 70 percent of the students who don't graduate." Is that a mundane claim that is being hyped to produce seminar enrollment, or is it truly interesting? A quick check of the relevant table from the 2007 Digest of Education Statistics reveals that --voila!-- the 17 states with the highest high school enrollment also contain about 70 percent of all 9-12 enrollment in 50 states and DC (70.4%, if you want a third significant digit). In other words, this fact would be entirely expected simply from the pattern of school enrollment across the states. 

So... is the Hechinger Institute modeling the type of "fair, accurate and insightful" publications that they wish reporters to produce, or are they trying to jack up enrollment with scary and misleading statistics? C'mon, folks: high school graduation and dropout patterns are of serious concern as it is, without modeling patently bad reporting.

July 10, 2009

Those evil union supporters who denigrate objective measures...

Quick: who said the following recently?

We do see the incredible power of setting stretch goals. But if you set a goal that's really not within reach, people will just give up on it and you really don't have a goal. We've seen this over and over. I think there's as much talking down of goals around here as there is of actually saying, "You're not thinking big enough."

Oh, this evil denigrator of the value of objective goals. From the text, you might conclude that this person is a teacher union supporter who will die before wanting to break down the firewall between teacher records and student test scores.

Except that the speaker was Wendy Kopp, head of Teacher for America and someone who said later in the interview that she is an advocate of using data and setting goals. But there's an important piece here about motivations and goals. No, I don't have answers for the K-12 world, but as I will continue to state until someone proves me wrong, there is something deeply wrong when an historian knows more about the relevant goals and motivation literature than most of the people who advocate setting extremely high goals in education.

Combining qualitative and quantitative evidence for teacher evaluation: What does "predominant" mean?

According to Gotham Schools, former NSVF and current USDOE official Joanne Weiss "said the Obama administration aims to reward states that use student achievement as a 'predominant' part of teacher evaluations with the extra stimulus funds" (emphasis added). I followed up with a USDOE representative, who emphasized after talking with Weiss that she meant a predominant part, not the predominant part of teacher evaluations, and that is how Walz reported the comment. The department representative added that department leaders "consider it illogical to remove student achievement from teacher evaluation, and we want states and districts to remove any existing barriers."

This came on the heels of TNTP's Widget Effect argument and Joan Baratz-Snowden's Fixing Tenure. I know that the political context of Weiss's remarks is to push the Duncan line that New York State's moratorium on the use of test scores in personnel decisions is wrong, and if necessary Weiss will bar New York from the Race to the Top funds if the legislature doesn't get its act in gear. Stand in line, please; I have a feeling a few million New Yorkers have the first dibs on dunking the entire state senate in the Hudson near Albany sometime in late November.

Back to policy, though: the word predominant perked up my ears because Florida legislature's language has evolved from language involving the dominance of student achievement to quantification. The current language on personnel evaluation is a legacy of language first written in 1999:

The assessment must primarily use data and indicators of improvement in student performance assessed annually as specified in s. 1008.22 and may consider results of peer reviews in evaluating the employee's performance. [emphasis added]

The current performance-pay language in Florida has the Merit Award Program which stipulates that for the purposes of merit pay, achievement data "shall be weighted at not less than 60 percent of the overall evaluation" (F.S. 1012.225(3)(c)).

I need to think about this in some depth, but it strikes me that the Florida legislature mandated one of several options to use in combining quantitative and qualitative judgments of teacher effectiveness, the point system. You can probably come up with other variations that meet the statutory language, but my guess is that any real-world implementation would almost all be linear combinations of different subscores, and I will use incredibly technical measurement language to call it the point system of combining different sources of information about teaching effectiveness. But that's not the only one, and I am always troubled when a clunky system is chosen as the default because it is the first option rather than a deliberate decision among options. I understand why a point system is in the bureaucratic and political gravity well, and it may well be that this particular clunky point system is the best option. However, it should be considered in comparison with what other clunky systems might be appropriate.

For example, there is also the holistic review of teacher effectiveness, such as exists in the new Green Dot-UFT collective bargaining agreement teacher evaluation system. There's no specific way that test scores inherently enter the judgment as such, though the implication is that teachers will have to show that they use assessment to shape instructional practices (what's called action research in the document, at the very least).

But those aren't all: a flow-chart is at least theoretically possible, though I do not have a real-life example. Yes, there are process flow-charts such as exists in Denver (and in the Green Dot system), but it's a flow-chart essentially describing when and how you schedule meetings, not how you make decisions in a meeting. (Step 1: Can you understand this chart? Yes: read the rest of it while walking to your secretary's desk; no: pretend to read it while walking to your secretary's desk. Step 2a [at secretary's desk]...)

Most theoretical: a Bayesian bump algorithm. I am guessing that there is a high probability that any subjective Bayesian statistician reading this blog will have thought of this idea already, but I'll adjust that guess after some data comes in. Since even well-trained evaluators are making subjective judgments about people, you could treat a principal's or peer's judgment as a prior judgment about the probability that a teacher should be retained/rewarded, given help, or fired. In the Bayesian world, that prior judgment can and should be shifted based on data, to form a posterior estimate of the probabilities of what should be done (you can play with a Bayesian calculator here, in a medical-test context). That adjustment is why I'm calling it a "bump" -- start with a professional assessment on various grounds and allow that to be bumped somewhat by test data, with the magnitude of the bumping depending on the data. Going down this path would involve some interesting studies, and it would probably be working with Bayesian posterior odds (which provide an interesting possible back door to a point system). This is a little out of my league in terms of specific characteristics, but the Bayesian perspective on statistics makes it possible to combine qualitative and quantitative data in a framework that already exists.

So we have four large categories of ways to combine essentially qualitative and quantitative data. While I am busy reading student work and doing other stuff in the next week, you all have a chance to dive in and describe what you think are strengths and weaknesses of each approach, as well as any additional categories (or disagreements with my classification entirely). After I have a weekend and get other tasks finished, I will return to explain (a) why a Bayesian approach is not only philosophically appropriate but serves the needs of unions, students, and anyone Alexander Russo describes as reformy; (b) why a Bayesian approach is not that different from a point system, at least in theory; and (c) what characteristics you would look for in a point system for teacher evaluation to meet the political interests described in (a).

July 8, 2009

A word to the wise on accountability

Dear fellow Americans who support equal education and are inclined to attack teachers unions when you get frustrated (e.g., Charles Barone and Citizens' Commission on Civil Rights):

  1. Borg-like rhetoric ("Those who resist the school reform movement are going to find they are on the wrong side of history. They may affect the pace of reform, but not its inexorable direction") is not likely to convince anyone that they're wrong and you're right. It's not even close to the level of Rod Paige's NEA = terrorist remark, but it's still intemperate. And I don't know about you, but the last degree I earned came with a beautiful, shiny rearview mirror, not a crystal ball.
  2. I'm persuadable that NEA staff and national leaders made some incredibly stupid/venal moves in trying to shift policy in the backrooms of power (which apparently are no longer smoke-filled), that the AFT may have made (fewer) such moves, and that locals and state affiliates of both national affiliates also make stupid/venal moves at varying rates depending on location and internal union politics. But a report that essentially treats policy concerns and backroom politics as identical? It strikes me as shoddy analysis, for several reasons. First, it's scattershot, which undermines the credibility of what probably would be stronger arguments on more narrow grounds. Second, it misunderstands the nature of organizations, assuming that unions have intentions rather than internal politics, agreed-upon positions, strategies, and tactics. Third, if you criticize both regular and backroom politics, you're implicitly committing yourself not to do much politicking on your own part.

Every few years you see a wavelet of attacks on teachers unions, and I am assuming that this is part of a new one. Sometimes it's just a coincidence, and I hope that's the case in the entries linked above... and here

Addendum: Charles Barone takes me to task on two items; in comments I say he's right on one and wrong on the other, but you'll have to read what he writes rather than my summary.

July 7, 2009

Not a love fest on mayoral control, but close

Something in the National Journal's website is swallowing spaces after periods when I submit comments, and I discovered that I could not edit remarks after submitting them, but those are not really the important matters. It turns out that at least for most of the respondents who have written thus far this week, no one believes that mayoral control is a cure-all for city school systems (or those who do have been chastened enough to admit the obvious). I think that of the comments thus far, Randi Weingarten's has been the most upbeat about mayoral control, at least in NYC. Go figure. But we have close to a consensus.

For my friend who asked me Sunday why I hadn't written a blog entry in five days, here's your entry and a pointer to another one, and another one in a few seconds. No promises about how long these will be, at least for the next week or two. I had a great weekend, but the fact that I spent about 12 hours this weekend reading student work tells you something about it.

June 30, 2009

Grading reports that grade states, which have schools that grade

It's now a PR cliche in education wonkery: grade states. Issue grades, and that's a hook for reporters to write stories about the reports, because the reporters at daily metros can say, "[Your state's name here] receives 'F' in think tank report on education." But beyond the PR value of grades, it's facile, which is why I'm surprised Education Sector gave into this particular venal sin in its report on states' higher-ed accountability policies. C'mon folks: can't you figure out a more substantive way of evaluating states? At the very least, this is so 1990s.

So I'm thinking about developing a report over the next year that grades think-tank reports that issue grades for states on some matter of education, where of course schools have teachers who grade students. Among the standards will be the following:

Clear standards for grades: a year before the report is issued, does the entity that issues the report publish grading standards or criteria?

A - Entity publishes grading standards with sufficient criterion specificity that an outside observer would not be surprised at the grade a state receives the next year. (Note: this is a low bar, not requiring agreement with grades.)

B - Entity publishes standards, but standards are too vague to provide benchmarks for policy progress.

C - Entity has previously published reports issuing grades to states, but changed the standards, or described the project and the areas where states would be grade, but no standards for those areas.

D - Entity has previously published the existence of the report project, but there is no previous publication of intent to grade states in this area of policy.

F - Report appears out of the blue with no publication of intent in this area.

Okay, folks: where does today's Education Sector report fit? How about Ed Week's annual Quality Counts phonebook? Fordham's reports that issue grades?

And, yes, if I'm serious about this, that implies I have to develop some more grading criteria. After all, it would be most interesting and ironic if I created a report that contained the mechanism by which the report itself could be torn apart. Hint, hint, ...

Find the typo! and other national-stage blogging

The National Journal unveiled its new education policy blog yesterday. My first response has an embarrassing writing goof; see if you can spot it!

No differences -> politics as usual?

While the DC vouchers debate swallowed more airtime, it's David Figlio's new study of Florida's voucher programs that will reveal the state of voucher politics. Several years ago, opponents of vouchers pointed out the lack of accountability for the programs, and in response supporters inserted a mandated study comparing achievement of students using vouchers to public-school students. Fortunately, they picked one of the best economists of education, who is careful and cautious and has done several studies of Florida's voucher programs in the past decade (including the best article on the topic, published in 2006).

Figlio's conclusion is roughly that given the data he had available, there is no evidence of differences in student achievement between those in the corporate tax-credit voucher programs and similar students in public schools. Further, the usually-cautious Figlio went out on a limb and said if additional data were available, he wouldn't expect the conclusions to change. This is not the only report I expect Figlio to produce on the corporate tax-credit voucher program, since the interesting questions for microeconomists are about how the shape of the market (the presence and size of a voucher program) changes its characteristics (esp. responses of public schools). 

But until that report is produced, and probably after it, the no-difference finding here mirrors a bunch of other studies. At this point, it looks like there is no solid evidence that students using vouchers perform better as a result, and in Florida at least, it also looks like students don't perform worse, either. So the voucher debate will not be settled by evidence of effectiveness, and we default back to questions of values embedded in public policy and the way that experiences shape the policy-relevant questions.

Those who support vouchers are spinning the no-difference findings as "vouchers do the job for less money, and choice is a positive value." Those who oppose vouchers are spinning the findings as "vouchers are no panacea, and choice can exist within the public system." And as voucher-receiving schools accumulate in the state, the ordinary politics of constituents make it hard for legislators to oppose eliminating the program. It is the last item that makes Florida (where a number of Democrats have voucher schools in their districts) different from DC (where the governing authority, Congress, has only one voting representative with constituents who use the vouchers). In the end, I think we'll see voucher programs generally stay in the states where they currently exist (primarily from the constituency-experience dynamic) but not expand much (because of the lack of evidence of great effects and because charter-school expansion in cities is an easier political sell).

June 29, 2009

Prevent backtalk: turn on the television!

I knew it years ago, and in two studies released earlier this month and this week, I think both in peer reviewed journals, we have it confirmed: the best way to prevent teenagers from talking back to you is to turn on the television years earlier so that they don't develop the ability to talk back. So that spring day in 1996 when my wife and I decided to sell our television before moving to Tampa? A big mistake.

And there we were, deciding that we were advancing our children's interests. No, that wasn't it at all: they were 4 and 1 at the time, and we decided that since we didn't like their arguments over the television, we'd see how long we could go without one in the house.Answer: 13 years and counting. And no matter what arguments we have in our household, it's not about the channel the television's tuned to. Instead, it's about who gets the computer...

Serious side: The article released this week is more about the relationship between adult caregiver and child than about television, and it highlights the importance of one-to-one interactions at early ages. I suspect this will be followed by other analyses from the same data set.

June 26, 2009

How to steer CYA-oriented bureaucracies, or why NCLB supporters need to think about libel law

Someone at USDOE sent me an invitation to listen to the June 14 phone conference where Arne Duncan explained how disappointed he was in Tennessee, Indiana, and other states with charter caps, let alone states such as Maine with no charter law, and how that disappointment might be reflected in the distribution (or lack of distribution) of "Race to the Top" funds (applications available in October, due in December, with the first round of funding out in February 2010). There are a few details that reporters didn't ask about (Duncan's somewhat surprising statement that a good state charter law would set some barriers for entry rather than establish a "Wild West of charter schools," and the way that small charter schools and charter schools with grade configurations outside state testing programs can stay off the radar for accountability purposes), but I was not surprised that two Tennessee reporters were called on for questions.

But apart from the selection of reporters for questions, the phone presser and other DOE moves made me think about the various uses of power in education-policy federalism. In limited ways, explicit mandates can be effective, if there is a sustained willingness within the USDOE (and esp. OCR) to make painful examples of the nastier school systems that try to evade those mandates. Offering technical assistance is another method, and despite the massive conflict-of-interest problems in Reading First, I agree with one of the researchers in the field who thinks that Reading First did improve primary-grade reading instruction, on balance. (Thumbnail version: hourslong scripts, ugh; explicit instruction in phonemic awareness and some other fluency components, obviously necessary.)


But neither heavyhanded mandates nor technical assistance can do everything, and neither works with the greatest motivation for both defensive and hubris-oriented bureaucracies: risk management. If you are a public school teacher or administrator, my guess is that you can identify some fairly silly action by your district that was motivated almost entirely by CYA motives, and if you can marry those CYA activities to pedagogy, you've been lucky or have a black belt in administrative maneuvering. (If you have such victories, please describe them in comments! Otherwise, we'll all wallow in the shared misery of observing defensive administering and the all-too-frequent ensuing train wreck.)

I think the federal government can shape bureaucratic behavior to the good by using that risk management and structuring accountability policies around that. And here's the lesson I take from my high-school journalism class in ninth grade 30 years ago: libel law in the U.S. generally recognizes the truth as a positive defense agaist libel allegations. That seems like a backwards way to frame the legal issue -- after all, isn't it common sense that a publication is libelous only if it's false? -- but the notion of a legal positive defense gives an individual or organization a way to organize behavior in a way that is both professionally appropriate and also make a legal defense aligned with professional expectations. Because the truth is a positive defense against libel claims, even an idiotic general counsel for a newspaper or publisher looks to the professionally-appropriate standard: is there documentation that the published work is true?

Sometimes a positive defense is not explicitly part of jurisprudence but evolves as a practical guidance for clinical legal work and internal advice for school systems. Observing procedural and professional niceties create exactly that type of positive defense in special education law. There is nothing in federal special education law to carve out an explicit positive defense for school system behavior, but many articles written by Mitchell Yell over the past few decades constitute a convincing case that school systems now have a de facto positive defense: professional documentation of decisionmaing and scrupulous adherence to procedural requirements are a positive defense against a broad range of allegations by parents of and advocates for students with disabilities.

Yell has argued (persuasively) that due-process hearing officers and judges use procedural adherence and professional documentation as a filter in special education cases. If a school district can document that it has paid attention to procedural mandates and has met professional standards for documenting decision-making, then hearing officers and judges are extremely reluctant to look at the substantive merits of those decisions. But if a school district has ignored standard procedural expectations that most districts meet, or if a school district has kept no or inadequate documentation of its decision-making rationale, then all bets are off and a hearing officer or judge will be much less likely to defer to the school district on professional judgments.

In essence, Yell implies, school districts can avoid adverse judgments if they pay attention to timelines and other procedural niceties and if they keep teachers and principals on their toes about current "best practices" as well as deadlines, notices, etc.  Not all districts are aware of this positive defense, or I suspect that some enterprising special education researchers could make a mint running seminars, "How never to get sued again." 

More broadly, I'm beginning to think that the construction of a positive defense against charges of incompetence would be healthy for school systems and state policies. The devil would definitely be in the details, but instead of being frustrated by a consistently observed school system behavior, maybe we should take advantage of that consistency.

The right kind of infection

The Powell et al. article on cultural complexity 90,00 years ago, published in the June 5 issue of Science, has some interesting consequences for education policy, though it's an archaeology article. The argument the authors make is that one needs a certain population density before one can find surviving signs of cultural complexity (archaeological evidence of more sophisticated used of symbolism and technology). Sub-Saharan Africa had both those population densities and archaeological evidence from 90,000 years ago, as did Eurasia 45,000 years ago.

Powell et al. are arguing that the development of the earliest human cultural skills may have depended on nothing other than density. This is an appealing story: get enough humans living in proximity, and whatever culture is developed will be maintained while the various subpopulations (clans, etc.) interact and teach each other, keeping the ideas floating around the population in a way that would not happen in a sparse population with little interaction between subgroups.


I suppose that as someone without an archaeology background, I have no insider knowledge of the contribution this paper makes to studies of human evolution. The authors are portraying the issue as an explanation of how human culture could appear suddenly (on the eon-scale) without resorting to changes in biology (esp. cognitive capacity). We'll see what other researchers of human evolution say about that, but there's something important there for education.

The article suggests that one can categorize various cultural characteristics by the extent of continuity across time. Isolated behaviors and skills may not survive unless they spread beyond the individuals who may exhibit/learn them for a time. With enough contact among people, knowledge, skills, and behaviors can become continuous; that continuity is the subject of the article. But one can look at knowledge, skills, and behaviors that are more than continuously existing. Are they common (maybe the experience of a minority but where everyone knows several people who have that experience)? Are they the normative expectation? Are they ubiqitous (universal or nearly so)? That's a five-category, ordinal variable for the extent of cultural behavior in a population: isolated, continuous, common, normative, ubiquitous.Okay, there's a sixth category, absent. 

Many of the debates over education policy are about shoving the national population from a common experience of X to a normative expectation of X, or from a normative expectation to the ubiquity of X. In the space of 70 years, high school graduation moved from a continuous population behavior to a normative experience; that's the story in my first book. But the rhetoric surrounding a national population's experiences often obscures variations. As Claudia Goldin has pointed out, high school graduation became normative in the midwest and northeast by 1940, while it moved much more slowly in the South (for Southerners of all races/ethnic backgrounds). And today, while approximately a quarter of teenagers leave high school without a standard academic diploma, there are many high schools where graduation is common but not the majority experience, and probably a few high schools where graduation exists every year but is not common.While the latter should be alarming to anyone, in reality the majority of high schools in deep crisis fall in the former category, schools where graduation is common but not the majority experience. 

There is an argument that the Powell et al. article suggests: if culture "spreads" once there is a sufficient number of "carriers," maybe we should look at education as akin to a disease process that we want to propagate. This is close to the contamination theory Geoffrey Canada has (or had when Paul Tough followed him around while writing his book about the Harlem Children's Zone). There are both ways in which that argument is interesting (esp. in communities where half or more teenagers drop out without a high school diploma) and others in which it is disturbing (assuming that students can be "carriers" of culture in way that adults can manipulate, though they can't shape adolescent experiences directly.. uh, no).

How do you move a behavior from a common-but-minority experience to a normative expectation? That's essentially the question we have in a large number of high schools in the country and with regard to baccalaureate degrees for the entire country. At least in my understanding, there are two requirements, involving both the spread of an idea and set of habits (habitus, in Bourdieau's language of cultural capital) and also institutional infrastructure. Attending high school became the normative experience for teenagers when they could no longer enter the full-time labor market with ease, when people began to think of high school as an experience that could be useful, and when there were enough high schools for majority attendance to be physically possible.

I do not think that there are exact parallels for all circumstances, just a combination of population behavior and institutional behavior. They go together. And, yes, there are cases where the extent of cultural experiences can reverse: working-class attendance at Shakespeare in the late 19th century, if you believe Lawrence Levine, or girls' primary education in Afghanistan from 1995 to 2001.

June 25, 2009

Nothing fuzzy about fuzzy math

George W. Bush is probably responsible for people calling constructivist math advocacy "fuzzy math". There is a field called fuzzy logic, and while I know very little about it, I'm irritated that the former president's maladroit use of English is messing up technical terminology. Fuzzy logic is a useful tool in engineering, and while Lotfi Zadeh's original term may not be perfectly descriptive, math teachers should be the last to misuse a term that's in their own discipline.

See-no-knowledge in education policy?

I seem to be reading several "we don't know anything so let's plow ahead" arguments in education think-tankery, from Mike Petrilli's argument that because we don't currently have a solid research base about how to turn schools around, we shouldn't try, to Kevin Carey's consistent argument in Education Sector's blog that because there is no research consensus about predictors of good teaching (and considerable research suggesting that there is not a link between effectiveness and countable items like years of experience beyond the first few or graduate degrees), it makes better sense to let people into teaching and then evaluate their effectiveness.

Fortunately, that's not the approach of the Institute of Education Sciences under John Easton, which has just announced a large research initiative on turning around schools. I suspect that both Petrilli and Carey would acknowledge that research in difficult topics is a good thing and argue that IES initiatives are different from policy, because sometimes you have to make decisions based on the state of knowledge you have, not the ... oh, shoot, there's Donald Rumsfeld phrasing again. But you probably know what I mean: Petrilli and Carey's stances are policy stances based on topic-specific agnosticism, not opposition to research.

But there's a serious question buried here: on big questions of policy, where you have to make choices, and the research is nondirective, how do you make decisions? I think the answer has to be incrementally, to allow research to catch up and influence policy later. If you make a huge political and institutional commitment to a policy path that has no research support and no ethical/legal obligation, then you're committing millions of children and hundreds of thousands of educators to a path that is very hard to change later. 

For that reason, while I think Arne Duncan's four-choice speech earlier this week is not based on research, and Petrilli is correct that there is no particular reason to believe that charter schools will somehow rescue the education of students otherwise stuck in horrible circumstances, the policy itself is good largely because it doesn't make hard and fast commitments to a particular path. The good thing about a charter is that it can be revoked, and in states such as Florida where there is a single authorizer for a geographic area (here, the county school boards), authorizers can be reasonably aggressive in shutting down shady or incompetent operations. So I share Petrilli's skepticism, but precisely because I am skeptical of any particular approach to schools in crisis, and because Duncan is being wishy-washy, I will applaud the Secretary for being wishy-washy. 

Update: I first used the term "know-nothingism" in the title. Ugh. Bad move for an historian. Petrilli and Carey are not members of the 19th century anti-immigrant party. Mea culpa.

June 19, 2009

Conversation often works ... where it's tried

Today, ACTA's Anne Neal thanked the AAUP and AACU for welcoming her outreach efforts.Towards the end of the blog entry, she writes,

ACTA also shares many faculty members' legitimate concern about administrative bloat and about trustees who lack a sensitive understanding of the special protocols and values that underwrite the unique enterprise of higher education. That said, we also believe that it is the professoriate's job to reach out to trustees. Faculty should understand that presidents and trustees are engaged in enormously complex, vital, and often urgent fiduciary endeavors. They should also understand that, going forward, trustees must be included among academia's primary stakeholders, alongside faculty and administrators.

I hope that's possible; that depends both on faculty and on trustees not accepting upper-level administrators as gatekeepers. My experience in Florida is that trustees often accept the role of administrators as gatekeepers of information, so that a president can essentially filter out quite a bit. I know of one UFF chapter at a community college that was able to meet with the chair of the trustees and establish a good working relationship, but that's rare. Far more common is a fairly uncomfortable and unproductive divide between trustees and most faculty, with a handful of administrators controlling the interaction.

I suspect that there's a pretty easy way to prevent greater access from becoming a vehicle for cranks and sophists (who will get their word in, anyway): err... asking faculty to provide the reality-check filter.

For those readers outside Florida, what is your experience with the extent of interaction between governing-board members and faculty?

June 18, 2009

The world is complicated, part 752

So the Center for Research on Education Outcomes has a report on charter-school performance, the Center on Education Policy has released a report on student achievement trends, NAEP released art-education data, and the spin has begun. Missing from almost all the reporting: Statements about the extent of peer reviewing for any of these reports. I'm not too worried about the professionalism of these reports,  since I know that the Department of Education always has an internal review process, CEP usually asks researchers in the area to review draft reports, and I would be surprised if CREDO did not have a pre-publication review process. However, the failure to report on the extent of peer review is a continuing and glaring omission in the reporting of education research.

In terms of the substance of the reports, I'm up to my eyeballs in prior commitments, but it's clear from the brief reading I have been able to do that the findings for all three reports are more complicated than the spin emanating for many of The Usual Suspects.* That's not news, I know, but I am the King of Things That Are Obvious Once He States Them, and I have a job to do.

* a great name for an a cappella group, if you happen to be starting one up.

June 13, 2009

On graduation rates and auditing state databases

I sympathize with Florida's Deputy Commissioner of Education Jeff Sellers, finding himself defending the state's official graduation rate the week that Education Week published its Swanson-index issue and pointed to Florida as a low-graduation state, using numbers far below the state's official numbers.

Some perspective: Florida's official graduation rate is inflated, but it's still better than Swanson's. Florida's graduation rate does more than Swanson (i.e., does anything) to adjust for student transfers and the fact that ninth-grade enrollment numbers overestimate the number of first-time ninth graders. 

Because of Florida's state-level database and the programming/routine that already exists, Florida is much closer to the new federal regulatory definition of a graduation rate than many other states, and Commissioner Eric Smith has been preparing the state board and other interested parties for the likely effect of the change on the official published rate -- i.e., that the rate will be a visible quantum lower than the currently-published rates (and largely for the reasons I have explained in the 2006 paper linked above). So in a few years we'll get a closer estimate of graduation from a lay understanding (the proportion of 9th graders who graduate 4, 5, or 6 years later).

The point in the St Pete Times interview where I winced was Sellers's answer to the question of how the state (and the general public) knows that the exit codes entered for a student are accurate: Sellers said that his department conducts an "audit from a data perspective."

That statement is misleading. It is technically true that there is an audit in two senses: each school district is required to check its data for accuracy before sending the data to the state's servers, and the state conducts a search of students reported as withdrawn in one county to see if they entered another county system before labeling them dropouts. But while I have seen reference to checking that the withdrawal codes are correct, I have not seen any evidence that such checks have actually occurred, and I have been unable to find that evidence anywhere on the Florida Department of Education website. That doesn't mean that it doesn't happen, but call me a touch skeptical. Without random checks, there is no guarantee that a 16-year-old coded as a transfer to another school actually was a transfer.

Given Florida's long experience with a state-managed education database, the lack of published audits of this process should caution us about the magic of state databases. They are important, but they need to be done properly. It makes sense to talk about the internal and external checks that should happen as other states construct databases and all states start to conform to the mandated longitudinal graduation rate:

  • Districts will need to be the first party to check accuracy, both in terms of preventing mistakes/fraud but also conducting consistency checks--are there any records which claim that a 45-year-old is attending kindergarten, for example? The first is supposed to happen in Florida, and I suspect that counties catch the low-hanging fruit in terms of errors. But the accuracy check on withdrawal code is the type of check that requires extensive follow-up to document whether a student identified as a transfer did in fact enroll in another school.
  • States will also need to conduct accuracy and consistency checks, though a state will necessarily be far less likely than school districts to catch outright fraud in claiming students transferred when they did not. 
  • States will also have to conduct the cross-checking that Florida currently performs every year and that I describe above: which students move between districts in the same state, but are counted as dropouts because a county only looks at its own students.
  • Finally, the auditing of transfer records would be MUCH easier if there is a standard way for school districts and individual schools to request the transfer of a student record and simultaneously use that authenticated request as verification that a transfer code is appropriate.

This is an incomplete list, but it's a start.

June 10, 2009

Teachers and school demographics

A few weeks ago, the Journal of Labor Economics published C. Kirabo Jackson's study of teacher moves away from schools in Charlotte that were moving towards single-race, segregated status (see lay description here; subscription-required article here).

Today, the Education Policy Analysis Archives publishes Kitae Sohn's article, Teacher Turnover: An Issue of Workgroup Racial Diversity (secondary site), which focuses on the potential attrition associations with teacher demographics rather than the student demographics. The punchline from the abstract: beyond a relatively small threshold of racial diversity among the teaching staff, "young White teachers are more likely to stay in their original schools when the proportion of minority teachers is smaller." The article was accepted well before I knew of Jackson's study, and there are a few small (and disturbing) nuggets apart from the main findings.

I suspect that for both of these studies, there will be replications, criticisms, and debates, and that's absolutely appropriate. Both articles focus on what is an important issue for policy (how do teachers make choices about where to work), and the conclusions are fairly disturbing. For that reason alone, I hope that they are the start of more work in this area.

June 9, 2009

Drug education is NOT working... at least not at ED in '08

Strong American Schools' ED in '08 campaign became one of the most successful independent advocacy initiatives of the 2008 election season and has helped turn the need for education reform from a low-priority campaign issue into one of the Obama Administration's top policy priorities.
--"Final Report from Strong American Schools" e-mail from Roy Romer and Marc Lampkin, June 9, 2009

ED in '08 spent millions of dollars with almost nothing to show for it. That's not a shame in and of itself, because loads of policy and political experiments fail (and that's why we call them experiments). But when you flop on a big stage and then claim an Academy Award? Sheesh. Education was one of the lowest-visibility issues in the campaign, it's hard to see how education is trumping the economy or health care as a focus of the White House's attention, and it's even harder to see how ED in '08 is responsible for whatever attention is being paid to education policy. 

I don't know what Roy Romer and Marc Lampkin are smoking, but I'm tempted to ask.

June 8, 2009

No one ever accused Arne Duncan of impersonating an education researcher

Hopefully some day we can track kids from pre-school to high-school and from high school to college and college to career. Hopefully we can track good kids to good teachers and good teachers to good colleges of education.

This was an excerpt from a speech Duncan gave today to IES staff about the need to use data warehouses to link individual teachers and test scores and then use that linkage to evaluate teachers (hat tip). Oh, yes, and do it based on research. Some day, Secretary Duncan, but tying an individual teacher to student performance is not something that you can assert is based on research available today. It is more wishful thinking than anything else. The best apparent on-the-ground research of this type with teacher education is nonetheless full of caveats. And that's on a program-level scale, not on the level of the teacher. 

I'd accuse Duncan of spouting fuzzy logic, but fuzzy logic (the real stuff, research-wise, using fuzzy sets) may be one tool we use to get out of this dilemma.

June 6, 2009

Sifting priorities, micro and macro

I had such good intentions this morning. After dropping off my daughter at the High School o' SATs, I figured I'd sit in the local Starbucks and read student work while she was wearing down No. 2 pencils. So there I was at about 7:45 in the morning, listening to slightly-too-loud Sinatra and reading drafts of one section of the major paper for the class I'm teaching this summer. After about a third of the batch, I bailed on both student reading and the environment of too-loud soft music and too-loud jovial fellow customers. I listened to Scott Simon's interview of Naturally 7 while driving a few blocks to the library branch that just opened up, and I'll sit here for the meantime, trying to figure out what to do for the rest of the weekend. As usual, I have Too Much to do, and I have to do some of it and not the rest. May I make the choices wisely, but more importantly, may I make the choices consciously.


In many ways, education policy and policy debates are about the same types of choices: you can't do everything at once, you can't fix everything at once, and being ambitious requires being selective about where you spend energy. It also requires a big-picture perspective. That's part of why I shook my head at Norm Scott's confectionary history of UFT. There's an important role for internal debates inside unions, and I have respect for UFT activists who are willing to go toe-to-toe with the most powerful teachers union leader in the country, but there are huge leaps of logic in Scott's thumbnail history and a failure to see a crucial big-picture issue.

Scott assumed that there was an overarching "sellout strategy" that Al Shanker consistently used after spring 1968, and that the sellout strategy was based on a circumscribed realpolitik vision of unions:

After the brutal '68 strike Albert Shanker knew the UFT could never again win much more than salary increases for teachers, and at some point only those at the expense of selling out. Thus over the next 15 years was born the "new unionism" where the union no longer is an antagonist but a cooperative partner with management.

The problem with this argument is not that it has no basis in fact but that it gives far too much credit to a single individual for the direction of the UFT (and AFT). Shanker was certainly a forceful unionist, and both the UFT and AFT were shaped by his leadership, but the general dilemmas facing UFT in 1968 were not new or unique, Shanker would never have been able to take the UFT on strike without the agreement of hundreds of UFT leaders, and there is something odd about the obsession of union dissenters with a single leader.

It's the last that's the most surprising to me on both intellectual and political grounds. If I were a member of ICE (a dissenting caucus within UFT), I would not be obsessing about Randi Weingarten. While focusing on individual targets can be useful for energizing one's base, it's useless for public discussion and the nuts and bolts of organizing and campaigning. To put it bluntly, it's following the reasoning template offered by the New York Post, whose editorial board loves to focus on personalities and the imagined virtues and vices of key figures. Imagine for a second that Shanker had died fifteen years earlier than he did, in 1982 rather than 1997. How would the history of the AFT have been different?

Oh, wait. We don't have to speculate. We can look at what's happened to the AFT in the past 12 years, since his death. There have certainly been stylistic differences, and the AFT has a far less closed culture (and is thus healthier) than it was at Shanker's death. But many of the strategic decisions taken in the late 1990s and early part of this decade would probably have been taken if Shanker had been alive, and it wasn't because anyone at AFT held seances to figure out "what Al would think" (despite the jokes made about Richard Kahlenberg's attempt to channel Shanker and probably some debates framed in that way). 

Consider the debates about mayoral control in New York City. I don't pretend to know the inside politics, but anyone looking at the picture three months ago could have predicted a few things:

  • Mayoral control would not be extended precisely as is, but neither would it end, and whatever came out would be a political compromise.
  • There would be test scores released that would be spun by multiple sides, and almost surely inaccurately on multiple sides.
  • Weingarten would have to make choices about where to push for change in mayoral control.
  • Someone would accuse Weingarten of being a sellout no matter what position she took, because she would be presumed to have given her okay for whatever came out.

I can't see either the logic in Scott's understanding of his own local or how Scott thinks teachers unions should behave in public debates such as over mayoral control. He either is using Shanker as a synecdoche for the strategic choices many UFT leaders have made over the decades or truly thinks that the key problem is that the wrong charismatic leader is in charge. Okay: Weingarten will be gone from the active UFT leadership in some months, so who's going to be the next target? I suspect that Scott knows deep down that his fight is with a very large group of fellow unionists who just disagree with his desire for more open conflict. 

One of the dilemmas with collective bargaining is the fact that the act of collective bargaining channels an adversarial conflict into a pattern of routines that then circumscribes relationships between union and management. Sit down and bargain, ratify, enforce agreements, picket and strike, lobby publicly for your members' interests and values: these are the public tools of power for a recognized union. A skilled union leadership knows how to use more than one of the tools at any time and if both wise and lucky will use the right tools more often than the wrong tools. An unskilled union leadership relies on a narrow set of tools in a predictable and increasingly less effective way until its members have essentially lost all the advantages of representation. But as several labor historians have pointed out (and my apologies for forgetting the names right now), there is no way to avoid the fact that if you buy into the legal authority of a union, you then buy into the set of tools that gives you.

Buying into that set of tools is not the only choice, of course; there's the historical example of the Wobblies who disdained contracts and collective discipline. I don't mean to suggest that the alternative is to match the violence by some Wobblies, but suppose for a moment that a union's leadership essentially ignored contracts, contract enforcement, and the like, and instead let the union culture evolve into wildcat direct action much of the time. There are two problems with arguments that unions should look more like the Wobblies (absent violence) than the UFT. First, I don't think it's a very smart political move. Because this country has 70 years of at least putative legal protection/recognition of union organizing and close to 40 years of effective public-employee organizing, most of the general public would conclude that anarchic direct-action participants over the age of 22 are trying to eat their cake and have it, too -- have the benefits of legal recognition without trying to take on any responsibility to follow the consequences of that recognition. In addition, in the internet age, glaring inconsistencies in the explanations of direct-action participants will make a union look like its members are less in touch with reality than George W. Bush, more manipulative than Dick Cheney, or both.

Perhaps more importantly, a lack of collective discipline and strategic choice is a path that is going to lose more often than win. Direct action does work where it's organized and lucky. It does not always work, and as one observer noted about the United Teachers of Los Angeles one-day strike fizzle, if it's intended as a public show without a broader strategy around it, it's nothing but street theater, perhaps entertaining and good enough for the evening news, but not enough to shape policy.

Maybe Weingarten needed to drive a harder bargain (and I think that's a reasonable position to take, that she made her peace too early), but you are making an implicit argument against collective discipline if you pretend that a union doesn't have to make strategic choices, make bargains with adversaries, or decide what is a reasonable settlement.

June 1, 2009

The Procrustean bed of teacher tests

Mike Petrilli's stab at the Sonia Sotomayor nomination via the Massachusetts teacher tests is a little askew, and I'm surprised he didn't look at an obvious dilemma that's deeper than the politics of a judicial nomination. Several former teachers have sued the state (and Pearson) for what they claim is a discriminatory impact of teacher tests given the disproportionate failure rate of minority teachers. This is the employee side of impact-analysis law that most school lawyers probably know better under the graduation-exam cases in Florida and Texas.

The landmark case here is Debra P. v. Turlington, which led to a number of federal decisions that guide the use of tests that have disparate impact in schools. To wit, tests with disparate impact by protected classes are acceptable if...

  • There is a rational state purpose for imposing them (guarantee graduate skills, in the Debra P. case)
  • There is sufficient notice to those affected
  • Those affected have a reasonable opportunity to learn the material on the test (the key reason for delaying graduation test applications in Florida, where federal judges did not want to hold the victims of segregation responsible for the unconstitutional behavior of schools)
  • The application of the test is professionally done (I'm bundling together several separate issues, including the composition of the test, defensible setting of cut scores, multiple opportunities to retake the tests, etc.)
  • There is no better way to meet the state's purpose that also reduces the disparate impact.

In the employment context, Petrilli is probably correct that the translation of the first item is essentially whether the test is a reasonable proxy for necessary teacher qualifications. But there is almost no way for anyone engaged in the current debate over teacher qualifications can defend these tests or defend the teachers' lawsuit without having some fairly severe inconsistencies.

Consider first the folks who have the approach that we should not care who enters teaching as long as we measure student achievement and make personnel decisions as a result. Several (whom I will not name to protect the guilty) have accused the High Quality Teacher standards in NCLB of obsessing about inputs (i.e., what teachers know) in contrast to outputs (what students learn). Anyone in this camp should abhor the Massachusetts teacher tests (and all teacher tests) because they continue the "let's look at the teacher qualifications absent the kids" approach, and we should be moving away from proxies for teacher effectiveness.

But the lawyers for the teachers and their supporters are not in much better shape, logic-wise. It is going to be very difficult to knock the legs out from the state's teacher testing program. They have to argue that the tests are a poor proxy for teacher skill, or that the tests were poorly constructed, or that there is a better option with a reduced disparate impact. If they cannot convince a judge that the tests were constructed and administered unprofessionally, the lawyers are going to be in the uncomfortable spot of arguing that the testing is an inferior proxy for judging teacher quality, in contrast to ... [The conclusion is left as an exercise for the reader.]

Summary: If you are in favor of judging teachers by student learning, then content-testing knowledge is a poor proxy by your own arguments. If you are against the content-based testing, then you have to come up with a better standard that will hold up in court. No, I don't think there's a way out of this for anyone with skin in the game, but if there is no summary dismissal and no evidence of rank incompetence in test construction, the fireworks will be interesting to watch.

Texas, South Carolina, Missouri, and Alaska

I know that the reports of the common-standards agreement shepherded by the Council of Chief State School Officers and the National Governors Association describe a few different reasons for why four states have not joined in a standards framework that is probably going to be about as close to a less-is-more approach as one can get in a bureaucratic standards document. Yes, I know Texas has just drafted standards (as has Florida, which is joining), that Missouri is searching for a new state superintendent (my guess is others are as well), that South Carolina has Mark Sanford (which is enough for any state to deal with), and that we haven't heard from Alaska. But here are my imaginary real reasons for why these states have opted out (thus far):

  • Other states refused to agree that everyone in the country would have to pronounce Harry Truman's state as mizZURah.
  • Texas would have to admit that bidness is not a word.
  • South Carolina did not get its way that there would be history standards with the required benchmark, "All six-year-olds will understand that each state is required to have at least one completely nutty elected official at all times, and this is a heritage of the Founders." 
  • There was a riot, not when Alaska insisted that NAEP math exams all use the Iditarod as an example of measure, rate, and general all-round toughness (other states just wanted to add their own events), but instead fisticuffs broke out when the Alaska rep. insisted that the current accepted size of the Earth was incorrect because if it was as large as most people thought, then you couldn't see Russia from your house.

Unfortunately, I suspect that the truth is far less entertaining. That's okay. We still have Joe Biden and George Will to mangle the facts in an interesting way.

Addendum: Lest anyone think I am making fun of other states, I should be very clear: I grew up in California in the 1970s, and I now live in Florida. That's enough ridiculous states to live in for a lifetime!

May 29, 2009

Unhappy with my brain right now

  • Fuzzy logic
  • Responders/nonresponders
  • Donald Rubin and multiple imputation
  • Dichotomous variables
  • Record linkage: whether a linkage allows one to determine outcome
  • Limits
  • Category theory

You have now been infected. That is all (for now).

Disappointing debate over teacher unions

I wish I could say I had learned something from the education globule's recent debate over the role of teacher unions, but I haven't. When the apparent tail end of the discussion ends with a claim that "unions... are tenacious and need to be defeated, over and over and over again if reform is to advance," I shake my head. Insert "Fordham Institute and other think tanks" where Mike Petrilli had written "unions," and you probably have Jerry Bracey's views on one of those days when the air conditioning breaks, the power goes out, and the roof begins leaking. It's more than a touch of demonization, or what's worse, facile reductionism (a more damning intellectual sin, in my book). 


Surprisingly, Andy Rotherham's rejoinder isn't much more substantive. Maybe there is a role in recapitulating the arguments for people who haven't heard them before, but this blog conversation has read to me much like Joan Scott's 1986 article Gender: A Useful Category of Historical Analysis ($JSTOR), to which one of my fellow graduate students in the late 1980s accurately responded (and the following is a rough paraphrase), "Well, yes, this makes sense, but by now it's obvious rather than productive." 

One of the missing pieces in all this is some sense of the historical roles teachers unions have played over the past century, at times when they have been both powerful and not. Petrilli and others are focusing on three roles of teachers unions: collective-bargaining agents, public representatives for teachers (including lobbyists in legislatures), and scapegoats. The collective-bargaining role of teachers unions is relatively recent, dating from the 1960s and 1970s, and given the variation in legal authority (the "why is Mississippi so bad if it doesn't have collective bargaining?" question), the facile answer now is "because they lobby."

That's an interesting hypothesis, but I have yet to see a single study documenting evidence for the claim that the reason why many school structures in Mississippi are similar to those in Massachusetts is because of the tremendous lobbying power of the Mississippi Association of Educators (the NEA affiliate), or that those school structures are the primary reason why Mississippi's education is inferior. Which structures are the same? Ah, things like changing classes in high school. Bureaucratic rules. You want to throw away things like an academic curriculum? And teacher lobbying is responsible for all that? Maybe it has something to do with institutional isomorphism, or the authority of administrators at midcentury, when many of these structures were consolidated, or the inertia that Mary Metz calls the script of "real school" and Tyack and Cuban call the "grammar of schooling." Homework, folks: do your homework first.

I stick "scapegoat" in that list because teachers unions have been scapegoated in the past in matters entirely unrelated to the concerns of today's... I'm with Elizabeth Green here in needing a better descriptive than "reformer," "reformy person" (I think Alexander Russo gets credit for that), or "wannabe reformer" (and I don't know from whom I've heard that phrase). In Florida in the 1960s, teachers and their unions were accused of various things from communism to sheltering gay teachers. In the early 20th century, the Chicago Federation of Teachers was accused of ... being a union and consorting with unions. Now Petrilli blames "unions" writ large for not being reformy-ish enough for him. Been there, done that, bought the t-shirt, gave it to Goodwill.

Two other roles Petrilli (and many others) are ignoring. One is the role of unions in social movements that extend beyond them. An example of that is the type of innovative organizing drive that UFT had with day-care workers, which simultaneously addressed issues of social class, gender, race, and early childhood education, not to mention the historic focus of teachers unions with K-12 employees in bureaucratic systems. To put it bluntly, childcare workers are on the low end of the education totem pole, women who work for pittances given the huge responsibilities in caring for young children. Childcare is also one of the hidden underbellies of the changing gender dynamics of the American workplace, making possible hundreds of thousands of two-earner and two-professional-earner households, not to mention professional single-mother households. Organizing childcare workers is the type of thing you'd expect SEIU to do (such as in its janitorial organizing campaigns), not UFT, and there will be consequences down the road inside UFT in terms of policy and leadership, and interesting possibilities in other cities.

Reaching back further in time, teacher unions have been involved in a range of social movements from the Progressive Era (with the Chicago Federation of Teachers, Jane Addams, and other progressives suing to recover uncollected taxes from corporations to pay for city services) to the post-WW2 civil rights movements. Teachers unions often have struggled with these issues, but it has also bolstered them. Case in point: the 1968 teachers strike in Florida, where according to my colleague Barbara Shircliffe the public images of teachers was often explicitly multiracial, a message of cross-racial solidarity that's hard to miss as dramatic in the 1960s.

That relationship has not always been negotiated smoothly, as Daniel Perlstein describes in his history of the Ocean Hill-Brownsville controversy, and that touches on the fifth role of teachers unions historically, as organizers of teachers' social identities. Probably the best theorizer here is Ira Katznelson, who has argued in multiple palces that people construct their social identities and roles around different contexts. In the U.S., he argues that there is often a split between the identity at work and the identity in one's home, and the outcomes of political conflict often revolves around how and where those active in a controversy define themselves. Perlstein's book Justice Justice! is an uncomfortable reminder that workplace solidarity is not always synonymous with justice.

A more interesting and productive conversation could revolve around the last, largely ignored issue. How are teachers' social identities formed, and how do workplace politics (including unions) feed into that? To the extent that teachers see themselves as either technical test-preppers or astructural "facilitators," they're ignoring real needs of students, and the context of those tendencies are important. Even the reformy-ish-ist folks believe that, or they wouldn't argue so hard for "reconstitution," "reconstruction," and other proposals to disrupt local school culture. So we all agree with school culture. We all agree that teacher unions matter. Does anyone else see a huge research opportunity rather than a place for pat answers?

May 24, 2009

The three-year degree already exists

Yesterday's Washington Post article on the three-year degree argument skimmed over what most such proposals ignore: there already is a three-year degree, and I don't mean the small number of three-year degree options that have largely failed to attract students. I mean the way that students currently speed up their college education: AP classes and dual enrollment in community-college courses while in high school. The Post story briefly mentioned George Washington University student Justin Guiffre, who might graduate a year early with AP credit. A college friend of mine did the same in the 1980s. I have known some students at USF who have also used AP class credit to finish general-education requirements early, which makes graduating a semester early almost automatic, and a year early quite possible.

Maybe I am naive or out of touch, but I don't recall this being a focus of any discussion vis-a-vis the three-year degree. Instead of blathering on about "better marketing" (which always rescues flops regardless of the merits of an idea), maybe American Council of Education President Molly Corbett Broad should be asking where students use AP credits and where they don't, and why. And maybe we should be asking whether a three-year-degree option would address the reasons for swirling or academic probation or lack of academic support from the institution, or any of the many reasons why degree completion is lower than many of us would like. Until then, the three-year-degree proposal is facile, not substantive. 

No-shoe-leather-used alert: did anyone else notice that the only students Valerie Strauss quoted were from George Washington University, less than two miles from the Washington Post headquarters, and Howard University, which is within three miles. They're both private, nationally-known colleges and not the typical college or university. Maybe she should have talked with University of District Columbia or University of Maryland students to see what the public-university student perspective is.

May 23, 2009

U.S. Secretary of Mixed Metaphors

"Investing in the status quo is not going to move the ball down the field," said Arne Duncan in discussing California education, and I wince. This is a Secretary of Education who thinks "incent" is a word, and when asked yesterday what happens when he plays the president in basketball, he answered,

Everyone asks me that. We usually don't play one-on-one. We usually play on the same team. We do pretty good.

No one has spoken using perfectly-correct grammar since Peter Jennings died, and Duncan is nowhere close to Joe Biden on the embarrassment scale (let alone GWB). But for someone who runs a Department of Education? Even if it is largely irrelevant to policy debates, this is at least a little embarrassing. 

Addendum (2 pm): And to prove that no one is perfect, I realized that I should have used using in the paragraph above, where I have now added it. That correction does not necessarily make me any better a user of language than I was at 9 in the morning, but maybe I am a little less of an abuser of language.

May 20, 2009

A blogger to USDOE?

Peter Orszag is being joined by Thelma Melendez de Santa Ana, the newly-announced assistant secretary of education nominee, who has maintained a blog while running the Pomona school district.

Worst-governed state?

A few thoughts on the budget catastrophe that California is facing:

  • Before voters imposed a supermajority requirement for crafting a budget several years ago, California's political system was broken. Voters approved Prop. 98 to guarantee a slice of general revenues going to K-12 and community colleges for a reason, even if the rigidity was not well thought through.
  • The "fix" of a supermajority and the elimination of nominal deficits several years ago didn't work.
  • The "fix" proposed on yesterday's ballot was an awful mess, because it refused to face the central problems. Nonetheless, the ballot measures would have been better than what California now faces.
  • Don't expect the problems to be solved anytime in the near future.

The fallout--an additional gap that has to be filled with taxes (not going to happen with the supermajority requirement) or cuts--is going to devastate a number of schools and colleges, and while I thought I was in the least governable state in the union, I have now seen the Florida legislature act at least a little more rationally than other major political actors in various states. South Carolina's governor, both political branches in California, Nevada's governor, and Arizona's legislature are among the actors who have outdone Florida for destruction in the name of political expediency, and that's hard to do (and comes after I exclude the notable folks who were shooed out of office because of scandal rather than mismanagement). But I have a single person's perspective, so maybe there's a contest we can have about the worst-governed state in the country. I wish I could suggest criteria, but there's a wealth to choose from, and maybe we should have different parts of the contest, sort of like there's the talent competition, etc., for beauty pageants:

  • Duct-tape governance competition--the state with the worst constitution, that is impossible to change, too easy to change, warped, inviting conflicts of interest, etc. Alabama and California are probably going to vie for this one.
  • Ostrich-impersonation competition--states where politicians are the best at sticking their heads in the sand to avoid uncomfortable choices. Florida's going to place highly in this one.
  • Lotus-growing competition--similar to ostriches, except that everyone points to the obvious problems and somehow argue that lotuses grow out of them, interpreting a dungheap as a site for beauty instead. Florida's hydra-like higher-education system, where every community-college president dreams of running a four-year college with a "leadership institute," is my nominee, but I'm sure you can figure out others. 
  • Mushroom-feeding competition--the state with the worst "keep everyone in the dark and feed them ****" decision-making.
  • Mushroom-eating competition--the state where politicians are the best at delusions about the future.
  • Dollar-grab competition--politicians that (would) do their best at shamelessly grabbing someone else's money in a transparent box even if they knew they'd be watched the whole time. I'll put my bets on Illinois or New Jersey, but Florida's got a shot at this one, too, with its former Speaker of the House.

Put nominations for each competition in comments!

May 19, 2009

A day in the life of a summer course

Third class of the summer session this morning, first one where students were supposed to have finished readings. This is an undergraduate social-foundations class, and the readings for this week include Gary Becker on human capital, Sam Bowles on social reproduction, and either the start of Paul Tough's Whatever It Takes or Joe Williams's Cheating Our Kids (which is apparently out of stock now).The summer session is only ten weeks, so they need to hit the ground running. I tossed the schedule around a bit to put the suck-the-reader-in books at the top of the term.


We start with an ungraded quiz. The incentive to do well here is because the questions might show up on the (absolutely graded) final exam. The last item is to propose scoring criteria (jargon: rubric) for one question on the back of the sheet. The substantive questions are of the compare/contrast sort with an implied 3-4 sentence answer, and I provide the broad hint that the authors "cannot all agree." Student groups talk about their answers, propose them on the dry-erase board, then we talk about the sketchy phrases, and they turn in the sheets, which I've now read.

Next is a quick exercise suggested by the latest edition of Wilbert McKeachie's college-teaching classic: they read drafts of their weekly papers to a peer, give reality-check feedback (i.e., after the reading, the listener summarizes what she or he thinks the main point is), and then hear me remind them to use formal citation mechanics, even if the format for the paper may be informal. 

We then started to talk about the books -- they first had to find someone at a different table who read a different book and represent their book to their classmate. Then as a whole, we compared the settings, the (inferred) motivations for each author, the (implied) major questions in each book, and the assumptions behind those questions. Students decided they wanted to discuss Bowles and Becker rather than have me lecture, so we spent the rest of the two-hour class talking about those ideas, discussing how Bowles and Becker would interpret Geoffrey Canada's personal history and education, and figuring out where Barack Obama's stated views on college would fit.

Somewhere in there we had questions on the logistics of the class, I met the three students who registered after the second class last week, I discovered that a PDF I thought I had locked for editing had been locked so students couldn't open the file (ouch), and we left loads of potential issues on the table. That's life. Thursday they upload a draft section of the major paper for the course (the section where they don't need a critical mass of readings under their belt yet), and Friday they upload the final version of the weekly paper. And somehow I will return feedback and grades Tuesday morning.

In some ways I am "working without a net" this semester, with a little more turnover on readings than usual. In particular, I dropped Kozol's The Shame of the Nation and paired Williams with Tough this semester. Maybe I should have dropped Williams because of the limited supply of books, but while Kozol and Williams were great contrasts the last time I taught this course (they both express outrage over unequal education in many of the same cities, but their explanations are worlds apart), I wanted to get Tough in there, and I may stick with Tough as a universal reading because of Chapter 2. But switching books always creates a little more demand in thinking-on-my-feet skills because I don't have experience in how students will respond.

Also, because of the compressed schedule, I made a commitment to learn student names in the first week. I'm awful with names and used every mental trick I could. I think I'm about 80-90% of the way there, and for a class of approximately 40, that's good for me. Right now, students are trying to keep up with the readings. My challenge is to keep the class rolling, to identify students who are behind from the get-go, and to manage the reading/feedback in a compressed semester.

I know that guy! (Delta higher-ed cost project)

I am pleased as punch that Nate Johnson's new Delta Cost Project report is being publicized nationally. I first met Nate about a decade ago when he was working deep in the bowels of the Ralph Turlington Education Building in Tallahassee, before he was plucked by the former state university system chancellor to be his data guru, and it was a good choice. Nate now works at the Florida branch of the Evil Empire (aka University of Florida), but I'm sure we can rescue him from the Dark Side some day, and until then, maybe he can have occasional opportunities to do work like the project released today. If it's like his other work, it'll careful and cognizant of a range of ways to look at important issues.

May 14, 2009

Changing higher ed, from Mr. Obvious Man

Craig Smith tagged me in an AFT FACE entry asking about the future of/a better vision for higher education, and given the way that Mark Taylor's schizophrenic vision of higher ed prompted not only a flurry of comments but thoughtful comments by Dr. Crazy, Dean Dad, Marc Bousquet, Timothy Burke, H. Saussy, and Michael Berube, among many others, not to mention Andrew Delbanco's review essay, it's time for me to underwhelm the universe with ten obvious comments about the future of higher education.

  1. Marc Bousquet is wrong in some very significant ways, but he's absolutely right in many others, and if his creative ravings prompt a healthy discussion of higher ed in the long term, my hat is off to him.
  2. In addition to other criticisms of Mark Taylor's curricular utopia, an important purpose of a stable curriculum is to eliminate one huge potential (expletive) waste of time reinventing wheels. It's far more productive to improve the wheels we've got and maybe invent a few carbon-fiber ones than to figure out how to make wheels made of hemp, green beans, recycled computer parts, and spent nuclear fuel rods.
  3. The entire discussion of college "costs" and tuition is off the deep end even while there are interesting sub-arguments. The discussion of tuition almost always ignores opportunity costs and generally ignores non-tuition costs (such as books or the cost of living). The Delta Project's analysis is interesting but entirely ignores the definitional problems in IPEDS reporting and the division of labor in colleges and universities. (I'd love to wave my hands and say, "Yes, fire all the student-life administrators, plow the money into faculty, and don't ask me to advise students!" Somehow, I don't think that's a practical suggestion) The human-capital arguments in favor of debt ignore the fundamental way that college student loans privatize the risks of going to college. At the same time, we have the chance to make a substantial incremental improvement in helping students with a shift to entirely direct lending and the automatic indexing of Pell Grants. I'll take the incremental improvement (it's HUGELY necessary) and still wish for some better model-building. I have no grand theoretical synthesis, but anyone who wants to buy me a good whiskey some evening and talk this over is more than welcome to!
  4. The vocational rhetoric surrounding higher education benefits the liberal arts because it implies that college students are responsible for their own affairs and should not be babied. This is in tension with arguments that liberal-arts programs and either a core or general-education curriculum should be at the heart of undergraduate studies, but on balance the vocational rhetoric of higher education has drawn far more students to college than would otherwise have been the case. We liberal-arts folks should be happy to have the chance to evangelize rather than preach to the converted. Give me 100 enrollees in my classes for a requirement, and I will convert 90 of them into students.
  5. The only national organization right now with a productive agenda on higher-education accountability is the American Association of Colleges and Universities. I'll take that good with the other, mediocre attempts funded by Lumina, but this is not a healthy state of affairs in the long run. The Shopping Mall High School's thesis is as applicable to large universities as to high schools, and until we can clone Cliff Adelman, we need a group of people with intellectual depth discussing the curricular problems at universities. 
  6. Right now, discussions of student learning are largely isolated from the widespread reliance on contingent faculty. Half of the discussions I see blame tenured faculty for avoiding teaching (as if all tenured faculty work at the University of Chicago). Does anyone else see the problems with this?
  7. Academic freedom can survive with a core of tenured faculty at an institution with non-tenure-track faculty, but we don't know the minimum size of that critical mass. For a variety of reasons, while the aftermath of 9/11 threatened academic freedom, it has been far more robust in the past decade than the worst fears in late 2001, including at my campus. At the same time, there are continuing threats, both inside and outside colleges and universities. In many places, tenured faculty are the most active defenders of academic freedom because they are safe; that was a crucial rationale for tenure in the first half of the 20th century, and it remains a valid argument. I have yet to see anyone who simultaneously advocates the abolition of tenure and can also point to a place that survived a real threat to academic freedom without any tenured faculty.
  8. Faculty are fragmented into too many communities of interest to defend academic values in a robust way. All too often, two-year and four-year faculty fail to understand the worlds that the others work in, let alone teaching institutions vs. research institutions, or even primarily teaching faculty and primarily graduate or research faculty in the same institution. Unions and the AAUP provide national organizations to defend values, along with disciplinary organizations, but the barriers are significant.
  9. When administrators ignore faculty organizations or do their best to do end-runs around them, they are missing substantial opportunities to advance institutional interests and feeding the behavior they presumably hate. I winced when I read one book by Derek Bok advising university presidents to do their best to go around the faculty senate or equivalent, because they're largely dysfunctional. Let me see if I understand the reasoning: if faculty senates are full of deadwood, and you go around them, what faculty support can you claim for your initiatives, and what incentive do you give the faculty you think should be in the faculty senate to serve? Oh, yes, and any monolithic model of your university demonstrates an essentially anti-intellectual temperament.
  10. Conversely, faculty who think that all administrators are evil are doing a remarkably good job of undermining collegial governance. There are serious problems with the development of academic administration "tracks" in the past 50 years (see item above), but the fact is that colleges and universities have administrators, you want the administrators to understand faculty and work with them, and what incentive do you give your colleagues to be willing to serve as administrators if they know you'll be the first one putting a target on their backs? Oh, yes, and any monolithic model of your university demonstrates an essentially anti-intellectual temperament.

May 12, 2009

Should artists know something about money?

It's cringing time for this union activist: Teaching is an art, not a business wrote Hans, commenting this evening on a story about a judicial mandate prohibiting a UTLA one-day strike this Friday. That statement is irrelevant in the specific context (teacher layoffs), is a false dichotomy, and is wrong-headed in other ways. Let's start with the literal claim that art is incompatible with business. The daughter of a friend and colleague went to SMU on a dance scholarship. She was smart and after a minor injury decided to get some business training and is now an administrator in an art-related New York nonprofit. Artists and non-profits need people who are passionate about art and can also manage money (ask members of the Florida Orchestra, which I hear is surviving today in this economy because its new executive director is very competent).

Or to take another example, there's a wonderful segment of Stuart Math's documentary on desegregation in Shaker Heights, Ohio, where one of the old-time activists describes a post-WW2 meeting of residents who were trying to figure out how to create a stable housing market, and a business owner said, "You know, we can be liberal and effective, too."  And they were, running a neighbor-managed real-estate outfit that was crucial in maintaining a stable, desegregated, prosperous community.

So much for the claim that art can't be business and warm-hearted liberals can't think in terms of getting stuff done. But the whole premise is wrong; I don't think teaching is an art. You can make a good argument that teaching is a craft, but there has to be solid practice at the bottom of it. In addition, anyone who is skeptical of the value of high-stakest testing, as I am, has to have something that's just a tad, a teeny, a tiny bit more astute than a statement that screams, "Just let me do what I want when I'm paid with the public purse." That's nuts, both philosophically and politically.

May 11, 2009

"Governance reform" is not reform

While New York rages over mayoral control, which is all the rage, schools in Pinellas County are headed towards The New Site Based Management, which was the rage in the late 1980s and early 1990s and which Bill Ouchi hopes will be the rage again.

While there are plenty of ways that governance can affect the classroom, I am consistently underwhelmed by the argument that governance reform improves what happens in the classroom. I've seen it all before.

May 8, 2009

No holy grail, just inexpensive texts, please

I love the inspiration of the Student PIRG Open Textbook campaign as well as the Hewlett Foundation Open Educational Resources initiative, not to mention the excitement over H.R. 1464 over at iterating towards openness (relevant entries one, two, three, and four). I think I'll take the latter's subtitle as my theme on this topic: "pragmatism over zeal." The blog's motto is about open content. I'm going to apply it to the practical issue when common-course texts are more expensive than community-college course tuition: we need good, inexpensive texts.

Open content may be one way to the goal (good, inexpensive texts), but it is not the holy grail. It's a possible path. There are a number of reasons to avoid putting all one's eggs into the open-content basket: the need for development and updating material, respect for the effort that good text authors expend, and the legitimate need to provide an incentive for good texts as opposed to any texts that don't count as highway robbery. In this, I take my philosophy from John Willinsky's The Access Principle: we'll take improvement as it comes.

What are the different paths towards this goal? Let me imagine a few:

  • Open content writing supported by private or public grants.
  • "Loss-leader" investment in texts by institutions.
  • Open content writing supported by communities of users.
  • Self-published textbooks using print-on-demand technology.
  • Some combination of the above.

Some explanation is in order on each of these. Currently, Hewlett is banking on the first: if the foundation can support the writing of text material for some of the most common college courses, it will save thousands of college students. That's pretty good leverage where appropriate. But that's not the only path, and it's important not to rely on that for a few reasons.

One reason to be cautious is because an institution can and should be free to innovate, and sometimes that innovation requires a different approach to material. Or faculty in a department may decide that a grant-supported open text in accounting or college algebra is just junk. So what else to do? In many public universities and colleges, the cost of a textbook for a single large-enrollment class is often greater than even a noticeable tuition hike. (Think calculus texts at $200+.) If a community college or university subsidizes textbook writing for a handful of large-enrollment classes, it can simultaneously save students hundreds of dollars, make a substantial point in public about how it serves the public, and protect its political legitimacy.

A variant of the grant-supported development of open content is the community support of open content text materials. This is a lot harder to organize (even along an open-source software model), but especially in technical fields, this may well be developing even as I write. But it does require some organization.

But what about the many college classes that have a niche but not enough enrollment to attract the attention of a Hewlett Foundation, the federal government (if the bill on open-content support goes anywhere), or an institutional investment? And where there isn't a community of faculty nationwide or worldwide to write and rewrite texts? In essence, grant-funded and community-supported open-content textbooks are going to be most feasible for the largest-enrollment classes. For many other classes, I suspect that faculty could develop texts inside the classes they teach, make electronic versions of the texts available for free inside the institution (to avoid conflict-of-interest problems), and then self-publish the material through a print-on-demand outfit either for their own students who want hard copy (and because that is optional, the conflict of interest is mooted) or for other institutions. Or publish through the Kindle mechanism at Amazon. For a variety of reasons, this allows faculty authors to bet and win on the long tail in niche courses. And for students, the cost of a text can be minimal while still providing net income to authors comparable to royalties through standard text publishing.

There are variations on the theme, but I hope that the obsession with open content for its own sake is replaced with the end goal: cheaper texts for students. I suspect students don't care whether the $25 text they might have access to is published through Lulu.com, is available on their Kindle, or is published through LightningSource and bought online. I suspect that if the text works for them, they'd be happy to pay $25 rather than $200.

"My university administration has asked me not to speak to the press"

Fellow education policy blogger Sara Goldrick-Rab wrote a painful entry earlier this week about how her administration treats her speaking up on a policy issue in her area of expertise (in this case, her opposition to UW-Madison's tuition hike), and I'm sorry I haven't followed up before now, because if she is reporting correctly (see the comments attributed to her in the Madinson Capitol Times), the University of Wisconsin-Madison administration is infringing on her academic freedom.

I was contacted the night before the initiative was rolled out by vice provost for enrollment management Joanne Berg, who informed me of the news and told me to refer all press inquiries to the University Communications office.

I should note that while I am sympathetic to Goldrick-Rab's policy perspective, I think she's wrong about the policy (for reasons I'd rather explore in a different entry). But my disagreement with her on specific policy grounds is very different from my absolute support for any colleague who is speaking on a matter of public concern, including employers' actions, from her or his expertise. This is one of those cases where I'd prefer knowing more about what's going on at the ground level, but at a first glance, it looks like Berg was acting the bully. Even if there were a miscommunication involved, Berg owes Goldrick-Rab a blunt apology for not remembering that tenure-track assistant professors have a pretty rational paranoia and a finely-tuned power meter. Berg could even use the wording President Obama has to acknowledge error: "I screwed up." 

Anyone want to guess what the odds are that she'll do that?

Does Duncan need a program-closing commission?

The recommended chopping of a dozen programs and half a billion dollars from the Obama recommended budget, and the expected political defense of those programs, reminds me of the various efforts to eliminate programs in the Pentagon that have developed political roots. A few weeks ago, I was wondering if the Pentagon needed a weapons-program-closing commission so that weapons programs such as the F-22 could be killed. But I suspect that the F-22, most of the dozen education programs Obama is trying to kill (several of which were also targeted by Bush), and some other programs that make me wince will instead survive because they will have fierce defenders on Capitol Hill who have a greater reason to fight for their continuation than other Congresscritters will have to kill them. And the outcomes will have little to do with intrinsic merit.

What is needed is the domestic-policy equivalent of the old base-closing commission: something that develops a list of programs in the discretionary domestic budget that should be closed, a list that is submitted to Congress for an up-or-down vote with no or few amendments allowed. There are multiple ways of doing this in a way that will allow ineffective programs to close, but the dangers of ineffective programs go beyond the money wasted to the general feeling that the federal government wastes money. There is a political cost in the long run to rampant corruption or political protection of ineffective programs.

May 5, 2009

Florida could still jump forward on end-of-course exams

The St. Pete Times is reporting that the death of the Florida House bill mandating end-of-course (EOC) exams in high school starting in science is the death of end-of-course exams, at least for this year. I'm not so sure. If I remember correctly, the legislature authorized EOC exams in principle last year, and there is an alternative funding mechanism: stimulus dollars. Embedded in the stimulus bill is section 14006, which is part of the $5 billion discretionary amount given the U.S. Department of Education. The state's application for state stabilization funds probably satisfies the nominal requirement for Florida to be aligible for a state incentive fund, if the state asks for incentive funds to develop EOC exams. This is precisely the type of project that the state incentive fund is designed for; it would replace the single comprehensive test with a number of tests tied to specific courses and instead of having to upset science teachers (such as in physics and earth sciences) with subjects not included in the first round (the filed bill in the House excluded them), there could be development of a full range of EOC exams in science. Seems like an obvious "yes we'll do that" to me.

I could be wrong; there may be legitimate reasons not to apply for state incentive funds to develop EOC exams. What surprises me is that during the legislative session, there was no public discussion I am aware of about the possibility of using federal stimulus dollars to develop EOC exams. I have heard nothing publicly at all about this, yet it's been an obvious possibility, at least to me. Has any reporter asked Commissioner Eric Smith about this? Is there any legislator or legislative aide who has asked about it?

May 3, 2009

AP feature in St Pete Times

I have microseconds before my day is captured by other things, but this morning's St. Petersburg Times gave Ron Matus inches and inches of space for his feature on AP classes. Kudos both to Matus for capturing both the promises and risks of expanding advanced-placement classes with a portrait of a student before the exams start ... and to the Times for devoting as much space to this story as it did, the same day that it also devoted pages to a scandal in the state's public investment bureaucracy.

May 1, 2009

Charters beat the pants off Florida Virtual School on the disruption scale

Maybe it's my training as an historian, but book titles such as Disrupting Class bring out my inner Larry Cuban. Disrupting Class author Clay Christensen points out that he tried to respond to Cuban substantively throughout the book, and the Florida Virtual School is a case in point, both for Christensen and also Bill Tucker. Yet I think the reason why the Florida Virtual School (FVS) both was in danger from and survived a legislative threat was not because it was tremendously disruptive but rather the opposite: it has matched parents' and students' expectations of "real school" to a remarkable degree.


A few times this month, education reporters contacted me, asking about the attacks on FVS, and all I could reply was that it smelled like a typical Florida legislative back-room deal to help someone's friend (or friends): require counties to start virtual-school programs, then the next year cut funding to FVS dramatically and also restrict its mission (thus feeding the county programs--presumably outsourced to for-profit entities--a bunch of guaranteed students). I don't know why that came unbidden to mind; maybe 13 years of watching the state legislature honed my preexisting cynicism? As in many other areas of schooling, it looked like someone saw money in public education and tried to sidle up to the legislative trough.

But that did not come to pass, in large part because a broad coalition of interests pressured legislators to keep the existing mission for FVS and to minimize the cuts (relatively speaking), essentially removing the class-size funding for FVS but (I think) not much else. There are a few notable elements in this battle that readers of Cuban (and David Tyack and Mary Metz) would recognize:

  • The different purposes people identify in using FVS--or the flexibility in the construct "distance learning" and a specific institution (FVS)
  • The way that defenders of FVS used language that reflect a perceived "normality" in online schooling: students, teachers, classes, credits, graduation, honors, etc.

In this context, last month's policy brief by Gene Glass and the response by Cathy Cavanaugh and Erik Black are far closer to Cuban than to Christensen. Glass's recommendations focus on accreditation, teacher certification, curriculum, and assessment. And Cavanaugh and Black agree. Wow, those sounds like standard school policy issues to me!

With one important exception, my own experience as a teacher and parent tracks with all of this. Students in my online course behave in ways similar to face-to-face students: many work very hard, some try to see what classroom (or Blackboard) deals they can cut, all need a certain amount of scaffolding, and their performance varies. My daughter has used the Florida Virtual School, and while the work is nominally independent, she has homework with due dates, times when she must speak with the teacher directly, and she receives grades. If her high school had independent-study options, her experience would probably be no different except that the conversations would be face-to-face and the homework submitted in person rather than online.

The one caveat is the one I have written about before: what/where is dramatic engagement online? There may be nothing wrong with online education as a vehicle for massive multiplayer online parallel play (i.e., independent study), but that's not the face-to-face dynamic. There may be nothing wrong with classes organized around online bulletin boards, and I have been told by several friends how that can generate the type of drama and thoughtfulness that concerns me, and maybe the relevant way to frame the issue is to think about the conditions necessary for such engagement. But enough about me: the fact that I have one generic quibble after approximately a decade of teaching online courses, my daughter's experience, and watching the policy environment in Florida suggests how much I think about online education in terms of the standard structure of schooling.

Maybe I'm not trustworthy on this because of my own biases. So maybe I'm an awful Luddite-prone troglodyte with no imagination, but I've had a blog for most of the decade and edit the English-language side of an online journal. Maybe Gene Glass is also an awful troglodyte, but he started the journal I now edit.

So take from this what you will, but I do not think that the FVS is an example of "disruption." As Bill Tucker's essay in Education Next suggests, FVS did not compete with public schools, private schools, or home schooling but complemented all of them. Maybe that's disruptive, but it continues a long history of supplementation of the school program. Anyone who attended religious schools outside public-school hours should understand that, as well as anyone who participated in private chess clubs, Girl Scouts, Boy Scouts, Campfire, various volunteer organizations, martial classes, private music groups, and so forth. Or, to take another example, those who have watched television programs with explicit educational purposes in mind. If Clay Christensen is right that online education is fundamentally disruptive, he should be able to point to the disruptive effects of another technology with some educational content: television. Please don't get sidetracked into the "how television ruined young minds" debate. This is about the relationship between explicitly educational television and formal schooling. What happened in that case was not competition or disruption but complementarity and hybridization. The spread of VCRs happened while I was in high school, and I saw Cuban's classic hybridization in process: in selected cases, teachers recommended that my classmates and I watch a program, or they taped a program to show in class for a specific purpose. 

There are four historical cases of potential disruption of schooling routines in the past century, and here I mean honest-to-goodness challenges to the legitimacy of public schools: private commercial schools in the early 20th century, federal youth programs in the Great Depression, Mississippi Freedom Schools and segregation academies in the 1960s, and charter schools in the past two decades. In the first two cases, the challenges were to high schools, and administrators responded in different ways. In the early 20th century, urban public high schools were in the midst of developing tracking, and while there were few challenges to the urban high school after the demise of academies approximately half a century before, one did: the private school teaching commercial skills such as typing and shorthand. Since young women were beginning to see pink-collar jobs as a reward for one or two years of secondary schooling, these commercial schools were practical and valuable. Harvey Kantor explained what happened next: public schools began offering courses to recapture the students. I suspect that the courses were more expensive than most other classes (and that would be consistent with the costs of most vocational education), but the point was to recapture the legitimacy of being the place where adolescents should be in school. How disruptive was that? I think that's arguable: it probably did more to confirm school officials' belief in the rightness of vocational programs than to push them off where they would have driven high schools otherwise. But that is a case where competition (if for legitimacy, not dollars) truly drove public-school behavior.

Edward Krug has the short canonical version of public-school officials' reactions to federal New Deal youth programs: AIIIEEEEEE! The criticisms of the National Youth Administration, the Civilian Conservation Corps, and some other programs focused on federal-state relations, but there is no doubt that school officials saw those programs as direct threats to public high schools at a time when teenagers were flooding schools as the place to spend adolescence. In the end, it was not the criticism of the educators that ended the programs: conservative Congressmen of the late 1930s were uncomfortable with federal work programs in the first place and ended the programs, at probably the first point that they could (in part with the excuse that WW2 and economic recovery made the programs obsolete).

I would probably not put the Mississippi Freedom schools and segregation academies in the same boat for any other question, but in one sense you could say that they both challenged public constructions of race and schooling. In the case of the freedom schools, operators challenged segregated schooling; in the case of segregation academies, operators challenged desegregated schooling (even the mildest desegregation). In several places (such as Jackson, Mississippi), segregation academies successfully siphoned off children of segregationists, and that success in some places drove public-school behavior for desegregation. In Tampa, as my colleague Barbara Shircliffe has documented, county school officials closed the historically all-Black high schools because their primary concern was keeping white children in the schools. There were both financial reasons for doing so and also political reasons, the same protection of legitimacy that drove educators to expand vocational schooling in the early 20th century and criticize New Deal youth programs in the 1930s.

Charter schools have represented a different type of challenge to public schools. At this moment, from a long-term historical perspective, I think charter schools are primarily challenges to urban school systems. Cities are where charters have repeatedly captured a noticeable minority of enrollment, and while there are some isolated attempts to "capture" charter opportunities for other purposes, you could legitimately say that charter schools are disruptive in several cities. Whether that changes the construction of formal schooling inside the classroom is an interesting and entirely unresolved question, but there is no doubt that in some cities such as New Orleans and DC, there are now several sectors of schooling that are public in the senses of both public access and public funding. I would not be able to say whether the relationship between those sectors right now is either complementary or truly competitive, and part of my uncertainty on that score is probably with the organizational leaders of the "public" public sectors (Paul Vallas in New Orleans, Michelle Rhee in DC).

This professional judgment is not about the comparative worth of either the Florida Virtual School or charter schools, though my impression of FVS is consistent with Tucker's. But historians commonly argue about continuity vs. discontinuity, and when someone uses the word disruption, it immediately starts up my mental circuits in that area. So this is an historical judgment about systematic effects. And here's the bottom line: if I were to pick between the Florida Virtual School and urban charter schools as disruptive, I'd have to pick charter schools.

April 29, 2009

The NY Post is shocked, shocked to find that donations are going on in here!

The New York Post is playing to a double standard this week in counting up donations from UFT to state politicians. Not only is there no examination of donations from others (Hi, Mayor Bloomberg and allies!), but one of the UFT's adversaries in mayoral-control politics is Democrats for Education Reform, which prides itself on getting down in the dirt and political.

So for the 37 Big Apple adult citizens who are not yet aware that there is shmuts in the big city, I have news for you: people donate to political campaigns. Organizations donate to politicians. If you don't have public financing of campaigns, you either have to have donations or all your major politicians will be billionaires. Well, maybe not all of them. Just your mayor.

April 27, 2009

PCAST, mostly very good except for Rensselauer's president

Today, the White House announced President Obama's picks for the President's Council of Advisors on Science and Technology (PCAST), and with one exception, they look great to me.

The exception is Shirley Ann Jackson, whose scientific reputation is fine but whose administration of Rensselaur Polytechnic Institute is rife with signs of problems, from the close vote of no confidence in 2006 to the dismissal of the faculty senate in 2007, stripping a professor emeritus of e-mail privileges, and (just discovered a few months ago) the provision of a second home in the Adirondacks for her at a time when RPI was laying off dozens of staff members.

I don't think that PCAST's reputation is well-served by one of the poster children for administrative arrogance in higher ed.

April 26, 2009

What are the costs of education at universities? A quibble

Sara Goldrick-Rab reports on Kevin Carey's visit to the University of Wisconsin at Madison. One thing about his comments and the Delta Project on higher-ed costs makes me wonder about the failure to talk about messy data with college costs, or rather what Goldrick-Rab reports on his comments:

Of course, Madison is a research university, a very good one, and research is expensive. So let's set all that research aside and look only at spending on what the feds classify as "instruction, academic support, and student services."

The problem is that it's not possible to rely on IPEDS reports to separate out the costs of research from the costs of instruction etc. If you want to read the relevant glossary items from IPEDS, you can scroll down this page to "instruction," but the gist is that IPEDS cost reporting for instruction can include a broad range of stuff you could describe as research-oriented including the salary of faculty (WITH time spent on research), salaries of academic deans, and even in some cases the depreciation of buildings when distributed to different functional categories. I don't know where graduate research assistant stipends and tuition waivers would be counted, but the point is that even without delving into support and student-services categories, lots of spending at research universities that is research oriented is counted as instruction for IPEDS purposes. Essentially, the IPEDS cost categories are functional to a moderate extent but not comparatively useful in the way that many assume.

That messiness makes it hard to have productive political conversations around instructional costs. On principle, Carey is right: students deserve the same general education wherever they go, and flagship public universities are often favored over community colleges and regional or directional state universities. But the key adverb is "often," and in some states it's a favored community college that receives interesting treatment (e.g., Northwest Florida State College and the Destin airport hangar... oops, educational building at the airport 15 miles from campus). And historical trends are relevant: many states ramped up raw-dollar investment in community colleges in the 1970s and 1980s as they were starting to disinvest in universities when examined per-pupil. That doesn't make the institutions equal by any means, but I suspect institutional leaders can point to inequities in how their sector has been treated by the legislature. They're different inequities, of course.

I don't mind Carey's asking the question about the relative costs of instruction -- even based on mediocre data, it's the right question. But I don't think it's easy by any means to have a single formula that apportions instructional costs per student FTE, advising and support costs per head-count, and research infrastructure with some other function. I'd love to be proved wrong with something that would be politically robust and not end up with all state support being zeroed out (in which case all institutions are certainly treated equally), so kibitzing is most welcome!

Brevard County schools want to go out of business?

Apparently the Brevard school district has a problem when the nondenominational church renting space on Sundays from a Melbourne elementary school wants to attract new parishioners by talking about sex within marriage (hat tip). Never mind that this is now standard fare in churches, synagogues, mosques, etc.

Despite teaching many children of NASA workers (Brevard County includes the Kennedy Space Center), Brevard school officials must be afraid that even if they are nowhere near school on Sunday, students of the elementary school will somehow be polluted/corrupted if a mailing to adults mentions "sex" and the school name on the same page. As my wife shot back when I explained why I was laughing, "Don't they know where their students come from?"

I suspect that Brevard has no legal basis for discriminating against the church on the content of sermons, and I hope they are ridiculed to the point of backing down before the contract is up for renewal. I'm not Christian, but this is absurd.

April 22, 2009

Margins for error in policy

We're operating without water in my building for today and tomorrow, at least until the lab reports come back after the drop in pressure overnight. Faculty and staff know, so they'll get to know their colleagues in nearby buildings. I feel sorry for students who run to the right (usual) place and then hope to dip into the bathroom right before class without knowing in advance that the bathroom is unavailable; I hope they all give themselves a little extra margin for error in timing.

I wish the same for policymakers, that they give themselves and their desired/favored policies some margin for error. A policy that falls apart without perfection is a doomed policy, and while everyone understands this, it's sometimes hard to put in place. Here's the practical difference between the stimulus package and Geithner's management of financial policy: the stimulus package can do a lot of good even if implementation is imperfect. I may not get my desired high-speed rail line going from Tampa to New York or Chicago, but someone will get jobs, take the money and spend it, and thus help replace the demand we're losing in this downturn. I'm much more concerned about Geithner's management of the financial mess and the resuscitation of credit markets, because I think there is far less margin for error without nasty consequences (either a waste of money or ineffectiveness).

To some extent, the discussion of the difference between assurances and firm enforcement in federal education policy is an issue of margin for error. I'm an historian, so I can go back to the prehistory of the 1960s (for all you young wonkish types) and Gary Orfield's first book, The Reconstruction of Southern Education (1969). As Orfield explains, Southern states resisted and tried to work their way around the requirements of the Civil Rights Act's Title VI (nondiscrimination), and it took some years for the federal government to make a bona fide threat of enforcement before school districts would desegregate in response to demands by the Office of Civil Rights. Someone who expected instant enforcement would have been sorely disappointed. Someone who expected that lawyers and the federal government would have to push and push hard for several years to begin the ball rolling -- and really rolling -- would have been more realistic.

There's isn't very good language for talking about this with education policy other than the vague terms implementation and transition. Some fields do have practical terms, though. For example, in meteorology the term I have heard tossed around with regard to hurricane forecasting is the "path of least regret." That means that if the choice for hurricane forecasters is between alerting people to evacuate when there is a definite chance it's unnecessary or failing to alert people to evacuate when they might really need to, the forecasters see evacuation or other preparations as the path of least regret. That term does not mean that forecasters always pick the path of least regret, but the language allows them to discuss choices in a clear fashion.

I'll be clear: while I agree with some of the mandates in ARRA money, I've already gone on the record as skeptical of unique student-teacher record linkage. But because regulation is a fact of life in both state and federal education policy, I think it's important to step back occasionally and think about the broader issues involved. That's part of why I've come to think that accountability systems need have a positive defense that teachers and schools can use, because it can allow systems and individuals to manage risk in ways that benefit students. But I'm getting ahead of myself. Basic takeaway: all regulatory policies need some margin for error.

April 18, 2009

Research blog started

For those who want to walk into the weeds with me on a new research project, feel free to follow my new research blog hosted at USF. Dorn's dangerously public research blog has the subtitle "conducting research without a net," and I am likely to fail in public view. [Update 4/20/09: the blog server's database had a problem over the weekend, but it's fixed this morning. I swear, my entry did not break the internets.] See today's entry for an an example of a "duh, this is why you don't look at your project at 9:30 pm" story. That's not quite true: looking at the project at 9:30 on Saturday showed me something I didn't pick up the last time I worked on the data at a perfectly sane time. But that's what being a tenured faculty member is supposed to allow and even encourage: taking greater risks either in terms of potential failure or the time required for a project.

For those who are curious about the background for this project, we currently don't have a good way to translate administrative reports of enrollment by grade into a trustworthy measures of graduation. Chris Swanson's work doesn't count without considerable assumptions, but that's not a shame at all, since no one else's does with the exception of measures adjusted for interstate migration (such as Rob Warren's), and that's not feasible except with states and other large population units. Longitudinal measures such as the NGA and federal regulatory graduation statistics will go a long way to fixing this, but there will continue to be an important need to be able to work with administrative data. And it's an interesting intellectual puzzle.

In my spare time in the past few years I've been trying an analytical approach using whatever meager skills I have in formal demography. There are limits to that, and I've decided to try a different approach, simulating a range of conditions of potential high schools and looking at relationships that way. This'll start with the simplest approach, a hypothetical world where the student population at schools never change, each ninth-grade cohort has identical experiences, and no one transfers in or out. If I can look at that artificial world, I might be able to relax those assumptions one at a time.

But I need to be able to generate data for that world that is plausible, as opposed to something I could generate by my imagination. So I'm playing around with data from the National Longitudinal Sample of Youth cohort beginning in 1979 to have a set of nationally-sampled data from real, historical adolescents with a year-by-year longitudinal record of school attendance and high school graduation. From that, I'll generate a set of synthetic (or Monte Carlo/simulated) cohorts with a range of grade retention and graduation. Consider it a pilot, or proof-of-concept, or just playing around.

If your spectator sport of choice is not baseball or opera, follow the new blog. As I've said, I'm as likely to fall flat on my face as not.

April 17, 2009

Are GPAs dirty while the SAT-I score is clean?

Before I dive into a minor patch of weeds, some basic issues: Above all else, the vast majority of college and university admission slots are not at selective institutions, so the debate over SAT use for deciding admissions should largely be tangential to policy concerns about postsecondary attainment. This is akin to spending all one's time thinking about the undergraduate curriculum at Harvard or the civic engagement of students at Oberlin. Even if you look at the institutions that require SATs, I suspect the vast majority of slots are at selective institutinos only in the barest sense of rejecting some applicants. But the use of SAT scores is a political hot topic because it stabs into our ideas about meritocracy (as Nicholas Lehmann has written) and also because it has been used for status purposes by institutions or pushed down on institutions (either by state politicians or U.S. News & World Report).

So when Michael Kirst gives us a heads-up that a forthcoming book will argue that SAT I scores have no added predictive value for finishing a degree (not first-year grades, but finishing a degree), I am not surprised one whit. I will wait for the book to see if the evidence is convincing, but I don't think that it will change either the use or the dominant themes in the debate. When SAT scores are used for things it was never intended to and for which there is no documented validity (as a placement tool in college, or for use in judging high schools), you're talking about culture rather than rationality and evidence. A case in point is Chad Aldeman's recent discussion of the SAT debate:

It may or may not be biased against minorities and low-income youth, and kids can be coached on how to improve their score. But, what else do we have that's better, that elite colleges and universities would trust as a replacement? High school GPAs are tarnished by grade inflation and high schools themselves are yoked to reputations. Personal statements are no less coachable than SATs, and extracurricular activities favor the children of parents with time and money. Even worse, none of these things are objective; a student in Abilene, TX cannot be compared to a student from Anchorage, AL on these things. The SAT, on the other hand, is a national test.

Since Aldeman had previously argued that selective institutions should set a basic "we think you can do the work" threshhold and then run a lottery, this is a fascinating defense of a largely defenseless practice. Here's the gist: plenty of research documents that despite all of its problems, a high school GPA is (roughly) at least as good as the SAT in predicting first-year grades. But while many people understand that imperfect data can still be useful (and I suspect that would be Aldeman's defense of the SAT), there is a theme in the excerpt above that appears commonly in debates about admissions standards: GPAs are dirty, SATs are clean.

The argument is almost always laid out the way that Aldeman does: high school GPAs are inconsistent from place to place. Even course titles don't mean the same thing; first-year algebra in one place can be remarkably different from algebra in another. Grades are often a reward of students' putting up with seat-time rather than a demonstration of accomplishments. In contrast, the SAT is a nationally-normed test, and whatever weaknesses it has, it more than makes up for that in its being objective.

One practical problem with this argument is that college is not a set of SAT-like tests. College is messy in all sorts of ways, and for all its flaws, there is something in a high school transcript that has more information about a student than an SAT score. We'll have to wait for the book to come out to see more, but there's a reason why a regular diploma is a more valuable credential than a GED, and the GED is also a nationally-developed test.

A second problem with the "GPA dirty, SAT clean" argument is that the use of the SAT can most harm the chances of students who come from high schools with the lowest graduation rates, schools where one could argue a relatively high GPA says a great deal about relative persistence. As Ted Sizer argued almost a quarter-century ago in Horace's Compromise, suburban schools are filled with the types of classroom treaties that result in grade inflation. But in a school where roughly half of the students never graduate, grades tell you a great deal. They may not tell you if someone who finished algebra I with an A can derive the standard binomial-equation solution (the SAT-I doesn't tell you that, either), but they tell you how much a student has persistence, guts, bureaucratic navigation skills, etc. And if someone from such a school writes an essay (we're talking about selective institutions, again), I suspect it would be far less likely to be coached or professionally edited than the essay of a student in a comfortable suburb. 

As an historian, my professional judgment is that the debate over the SATs has almost nothing to do with whether there is a rational justification for its use in admissions. Instead, the public debate is almost entirely over our ideas of merit, and the framing by one side of the debate as a claim that the high-school GPA is dirty while the SAT is clean is confirmation of that judgment.

April 16, 2009

Migration and graduation

I'm experimenting with publishing working papers on the Social Science Research Network, with the first one, Migration and Graduation Measures, freely downloadable on some technical issues with graduation rates. The gist: without knowing accurate information about migration (and transfers), non-longitudinal graduation rates are going to be inherently problematic.

April 11, 2009

Bewildered at arguments about rent-seeking

Not to accuse the conservative left hand of not knowing what the conservative right hand is doing, but I am bewildered by the latest publication of the Center for College Affordability and Productivity. (Hat tip.) The report repeats an argument I have heard before why student aid is horrible: it feeds rent-seeking behavior from colleges and is therefore counterproductive in terms of larger spending patterns. Apart from the thin evidentiary base and failure to consider alternative hypotheses (primarily, that public colleges universities raised tuition as state appropriations per student have fallen), there's a gaping inconsistency between the "it just encourages them" argument against student aid and arguments in favor of publicly-funded vouchers that pay part of private-school K-12 tuition.

Some K-12 voucher programs are conditioned on schools' accepting vouchers as complete payment, but that is not true with either Florida's tax-credit voucher program or its voucher program for students with disabilities. Yet--not to my complete surprise--I don't think that anyone who has argued that college student promotes rent-seeking basis has lifted a finger to see if there is rent-seeking behavior with K-12 voucher programs. This is not a call for anyone to research this, particularly, since I don't think the salient issues with K-12 vouchers are the possibility of rent-seeking.  But it's an inconsistency in conservative education policy arguments that is rather curious.

April 10, 2009

Remedial math in community colleges

The anonymous community-college Dean Dad wrote this morning about remedial math classes in community colleges, and I'll use this as an excuse opportunity to bring together several thoughts I've spread around in different places or have not articulated:

  • Remedial education in community colleges should be the logical place where we try Carol Twigg's approach to improving essential common instruction.
  • We should stop blaming a mythical lack of alignment between high school and college for remedial-education needs in community colleges. I'd bet a bundle that every high school counselor tells students that algebra is required for college, and I'd bet a bundle that students who pass algebra and then are slotted for remedial education in community college knew far more algebra at the end of the algebra course than when they took the placement test for CC. My alternative hypothesis: students forget, especially if their hold on algebra in high school was by the fingernail. I'd be happy to be disproved wrong here, but someone has to do the research to keep stating the myth without my tossing tomatoes.
  • There should be no conflict between the National Mathematics Advisory Panel's final report recommendation emphasizing fractions as a central pre-algebra skill, on the one hand, and the desire to teach probability and statistics, on the other hand. How can you teach probability without students' understanding fractions? 
  • My guess based on observing weaknesses in communicating math-ed expectations is that one key stumbling block in learning fractions and teaching them is grasping/explaining how they can represent multiple properties and how the same properties of a fraction can be represented in different ways. If someone understands that 2 is a fraction, and that 67% not only is close to 2/3 as an abstract value but also can represent an approximation of the same darned thing (for a whole load of values for "thing"), then the concept of rational numbers is a small step, or at least a much smaller step when someone responds to the first statement with "huh?"

April 7, 2009

Mediocre education reporting #357

Sam Dillon's story this morning on the evident pending Giant Teacher Shortage IV: Postapocalyptic Horror... uh, pending retirement of many teachers struck me as extraordinarily poor reporting on a National Commission on Teaching and America's Future report because Dillon wrote his lede from the first two pages of the report.

Here's what the report says, or the gist of it: let's develop alternative models of socializing new teachers, models that take advantage of the demographics of teaching.

Here's the lede in the article: "Over the next four years, more than a third of the nation's 3.2 million teachers could retire, depriving classrooms of experienced instructors and straining taxpayer-financed retirement systems, according to a new report."

Do you see the same problem I do? The main argument of the report is not mentioned until paragraph #4 of Dillon's story. (In addition, like many other reporters Dillon fails to mention whether the report was peer-reviewed.) Then Dillon apparently called up Michael Podgursky for a publishable quotation, who dutifully responded with his usual skepticism about demographic Chicken-Littleism.

Except in this case, the primary Chicken Little was the reporter. Look at the chart in Appendix B, on p. 18, and you'll see quite a bit of room for optimism. For those who need a hint: the Baby Boom echo cohort is now beginning careers.

Update: Apparently USA Today has the same take on the report as Dillon, and like the Times, it fails to note that the report did not appear in a refereed journal. C'mon, reporters: raise your game when reporting on research.

April 6, 2009

One teacher's response to Ron Matus's article

There's been lots of coverage of the Ron Matus story March 29 on firing teachers in Florida, but there's been no follow-up online about the letters to the editor that were printed April 4 (last Saturday), and at this point, I can't even find the letters on the Times website. But I think one needs to be highlighted, because it's from a teacher and makes a few important points:

The premise in the article [by Ron Matus] is that tenure makes it too hard to fire bad teachers, yet the few examples given don't demonstrate that, but rather, simply show inaction on the part of school districts.

If the writer had found districts attempting, but failing, to fire bad teachers, he might have a point. I see this drive to get rid of tenure as an effort to instill fear in teachers and keep them silent. Teachers living in fear for their jobs can't afford to speak out.

Getting rid of tenure (read: due process) might make it easier to dismiss the rare teacher who shouldn't be in the profession. It would also make it easier to dismiss the good teachers--even the great ones, because the great ones are the ones who stand up and advocate for their students, themselves and their profession, and in doing so sometimes step on toes...

John Perry, Tampa

I've known John Perry for a number of years; he's an activist in the Hillsborough Classroom Teachers Association, but I don't think he was when we met. I think Perry's wrong about the order of magnitude of "the rare teacher who shouldn't be in the profession" (emphasis added), but since a good portion of teachers leave the field within a few years, I don't think that there's a shortage of ways to discourage teachers from continuing.

More broadly speaking, I think more sophisticated critics of teachers and their unions understand that administrators are the ones who fail to fire teachers, but Perry's other point is important: while K-12 teachers do not have academic freedom in the same sense that higher-ed faculty do, they're the ones I often hear a certain style of reformers praise for precisely the type of dissent that would be in danger without due process.

So let me phrase the question in the following way: does anyone want administrators to be able to fire teachers summarily after teachers do the following?

  • Refuse to change a grade to let an athlete play.
  • Complain that the new math textbook series is confusing to new teachers and likely to lead to poor teaching.
  • Sign and date a request that a child be evaluated for eligibility for special education services.
  • Complain when girls have fewer opportunities than boys.*

As far as I am aware, the only case above for which K-12 teachers are clealry protected when they speak out is the last one, and that's because of a Supreme Court decision stemming from Title IX; I suspect that the are likely to be protected if they push for assessment to gain services for a child, but I don't know of anything as clear-cut as a Supreme Court decision. And I don't see people who are in favor of "tenure reform" rushing to replace workplace due process with greater whistleblower protections.

April 1, 2009

Sharpton paid off? Please tell me this is an April Fool's joke

The New York Daily News is reporting this morning that former NYC Schools Chancellor Harold Levy is involved in a $500,000 payoff set of donations to the Rev. Al Sharpton's organization, with payments beginning shortly after Sharpton and Joel Klein launched the Education Equality Project in June 2008. With friends like Levy,...

In other news, I am hereby announcing my support for the public flogging of teachers whose students' test scores decrease from year to year, my hope that NYC invests an addition $1 billion in the ARIS system, my trust in the market to determine the true worth of schools within a voucherized environment, and my death last Thursday from reading Michele Foucault. In lieu of flowers, my family is asking that donations be made in my name to the John Birch Society, except for my son, who would appreciate iTunes cash cards instead.

Okay, it looks like the DN story is serious. Yikes. That'll take the wind out of the Education Equality Project (EEP) conference starting today. Then again, maybe "eep!" is the reaction of participants and fans of the Klein-Sharpton effort.

March 31, 2009

Lies, damned lies, and the Pacific Research Institute

Often, it takes a bit of time to uncover statistical flummery; one needs to dig deep inside arcane methods or the details of data collection. But sometimes it takes just a few clicks. Matthew Ladner's blog entry today, $243,000 per student school districts?, quoted work by the Pacific Research Institute that claimed two small districts in California each spent more than $200,000 per student. Scandalous! Criminal!

Well, it might be, but so is the credibility of anyone who quotes those statistics without taking a few minutes to look a wee bit more closely. According to the quoted press release, California spends something over $10,000 per student ($11,600 is the figure quoted for 2006-07), but the Mattole Unified and Mineral Elementary districts supposedly spent $225,256 and $242,610 per student.

My first thought was of the tiny New Jersey districts that literally had no students for odd administrative-law reasons. Or maybe these were essentially fictive districts created by companies that incorporated towns and funneled money into specialized programs for a tiny population of executive children. But the locations (Humboldt and Tehana counties) didn't fit with either hypothesis. I was curious: I clicked. And then I saw the magic words in the PRI website pages: "Revenue Received per ADA." ADA = average daily attendance and is not always the same as enrollment.  Okay, so what was up?

Mattole apparently has 905 enrollees but the ADA listed on the PRI site is 35.2. Mineral has 123 enrollees and an ADA reported by PRI as 4.7. If you divide the revenues by the enrollment, the average revenues are a much more sensible $8,761 and $9,270 per student, respectively. That sounded very odd: WHY would ADA be so low? So I checked out the districts. In Mattole, the vast majority of enrollment is in a single charter school, Mattole Valley Charter, which had 864 enrollees in 2007-08. In the Mineral Elementary district, the eScholar Academy virtual charter school had the bulk of enrollment.

If gambling on proposition bets were legal in Florida, I would bet at least a little money on charter-school enrollment NOT counting for the district's official ADA. So in any small district where the majority of enrolled students were in charter schools (or in this case, one charter school each), the official ADA would be far lower than the actual enrollment of students in schools receiving public support. 

But instead of betting any money, I'll just bet my reputation. Does anyone want to prove me wrong? 

Update: In comments, Ladner and PRI staff member Vicki Murray acknowledge that I was correct, and the original claim was incorrect.

March 30, 2009

Seattle will be drier

I spent some time this weekend finishing the first complete draft of a talk I'm giving in Seattle on Thursday. I'm going to be heading there while a few thousand historians are leaving Seattle after the end of the Organization of American Historians meeting. I'm either expecting to find a time machine or I am heading there for a different meeting (Council for Exceptional Children). Last time I was in Seattle, it was wetter and colder than what's forecasted for the middle of this week. We had a drenching rain in Tampa this morning, so things will even out in my personal experience this week, even if not for the world.

I hope my neighbors weren't paying close attention while I was timing the draft. I don't read papers word-for-word, but I wanted to get a sense of how far I'm off on time, so I read it aloud while alternating between the laundry room and the kitchen.

Oh, the topic? Accountability and students with disabilities. I think I know how I'm ending the hour, but the cliffhanger before the third set of commercials is the tough part right now, and I haven't yet decided if Jason's going to live. If he does, I'm going to have to tear up the last act and start fresh. I've given a spoiler, haven't I?

More seriously, this talk is giving me the opportunity and prod to think through some connections between areas of education politics that I mentally put on "percolate": the democratic rationale for public education, tensions between public and private purposes of schooling, and what technocratic mechanisms may be useful for (and in what circumstances). When I get back, I have to think about potential outlets and how to get a potential coauthor to give up enough time to participate (and the value involved in that). 

The only serious performance question I have is the extent of corny jokes and how far I can/should push them.

  • An RTI Tier 2 intervention plan and a Writ of Mandamus walk into a bar...
  • Peter Singer dies and finds himself at the Pearly Gates facing St. Peter: "So your most important goal right now is to avoid pain?" St. Peter begins...
  • How many IEP team members does it take to screw in a lightbulb?...
  • A rabbi, a minister, and a psychometrist are in a rowboat in the middle of the lake...

Maybe not those jokes.

March 23, 2009

Fordham Fellows

Once again this year, as in 2007-08, the Fordham Institute has brought in a group of Fellows with a diverse set of views. Today's blog entry by Catherine Cullen is a case in point. I side with Cullen on the substance about Charles Murray, but Mike Petrilli and Fordham in general deserve kudos for creating an environment where their fellows are free to speak their minds.

March 22, 2009

Grokking social-science statistics

Several comments in the past few weeks have expressed some wonder that I use statistics when I am publicly skeptical of several policy-related uses of education statistics. I am a little confused by the comments (and implicit accusation of inconsistency), since many of the most articulate critics of high-stakes testing are assessment experts, but for the record, here are a few of my personal stances towards social-science statistics:

  • If for no other purpose than to engage in political debates in a conscientious and credible fashion, adults need to have some rudimentary knowledge of statistics and probability and also be able to listen to and discuss essential concepts without doing enormous violence to them. This is on the same order as needing to have some rudimentary knowledge of Newtonian motion, thermodynamics, electricity, algebra, natural selection, etc., to engage in public policy debates in a constructive fashion. Know why perpetual-motion machine patents require extraordinary (and highly improbable) evidence; know why regression to the mean invalidates many change-over-time claims when the baseline comes from a sample of outliers. 
  • If you're tempted to be proud that you don't know statistics, see what happens to the following sentence if you replace "in French" with "using statistics" and "French history" with your current interest: "Yes, I'm writing about French history; what do you mean, I need to read stuff that's written in French?"
  • One of the reasons why one needs that basic knowledge is to know the limits of statistics and be able to ask probing questions of the claims that are made in public debates. Probing questions are not of the formalist type that could be applied to any claim, "You can say what you want by picking a statistic" or "It's unethical to use statistics without talking about the metause of statistics." Probing questions engage the specific claims made in debate: "Politician Yodel says we saw a 102% increase in the incidence of Echoing Disease last year, but I want to know what the incidence was the year before so I know if this is a serious problem."
  • Though social-science statistics are inherently constructed objects, they can nonetheless be enormously useful. For a thoughtful and useful discussion of social-constructionist arguments, see Ian Hacking's The Social Construction of What? (1999). (Michael Berube and I both very much like Hacking's discussion of dolomite, though I suspect I am closer to Hacking's end view than is the Paterno Family Professor of American Airspace and Dangeral Studies.)
  • To work with social-science statistics, at least I find it tough to simultaneously criticize every character that I type in a statistics program and also work the darned program and think about what I'm doing. So I engage in a form of suspension of disbelief, work the statistics, pause and think about the larger meaning and doubts, work again, doubt, work, doubt, etc. I know I'm embedded in the statistical machinery when I hear, "Sherman, are you going to get any sleep tonight?" And I know when I've doubted enough when I realize I forget the syntax for calling up multiple regression.

And tomorrow morning, because of the idiosyncrasies of the USF IRB-02 records, I need to write and print an IRB protocol so I can finish a long-delayed project ... assuming I can climb the learning curve for the R-Project language.

March 21, 2009

Michael Crow and Bernie Machen up the yin-yang

Monday's New York Times story on Arizona State University stole my point earlier this month about the expenses of public research universities and the tension between undergraduate teaching and the building of a research infrastructure. Either that, or I was stating the obvious (I think I was stating the obvious). The Times quoted the ASU student State Press in pointing out that the budget cuts had turned ASU into the Neutered American University instead of Crow's "new American university."

Today, Timothy Burke has another thoughtful entry about the future of higher education, this time on the difficulty of building the core of a great teaching university, on top of September's argument that the party's over as far as a several-decades boom is concerned. And Thursday's news about prospective cuts at the University of Florida's College of Liberal Arts and Sciences should be sobering given that UF is putatively part of the Association of American Universities, the country club of higher education in the U.S. Michael Crow's ambitions did not protect ASU from a fiscal fiasco, and after Bernie Machen continued a decade-long trend to turn UF into a medical center with a university appendage, budget cuts have resulted in a layoff grievance that my union won decisively this month, pending disaster for science at UF, and the widespread destruction of morale around campus. So much for the value of being a member of the country-club set.

ASU and UF are the extremes of this pattern of overweening administrative/political ambition, stories of mission creep having become mission sprint and now the mission trots. Other institutions may survive this downturn without as much of a visible fiasco, and well-placed institutions might even benefit, at least in comparison. The irony of the entry title is that while "up the yin-yang" is slang for extremism (well, in one of its uses), the reason why Crow and Machen are in the Academic Hall of Shame right now is because they have not understand balance at a public university. I suspect that there is a reasonable balance, and part of my job as a faculty union leader is to do my best to push that balance. But I recognize that historical trends and current budget crises make that balance much more difficult in most places.

March 18, 2009

By request: on teaching quality

I am not going to write today about the new report, An Evaluation of Teachers Trained through Different Routes to Certification, because I haven't read it. (Other things take higher priority for me right now; the report is heading to my to-read pile. But after reading the praise and also Aaron Pallas's criticism, let alone Sean Corcoran and Jennifer Jennings's review, my curiosity is piqued.) But I have an outstanding request from a reader to discuss teaching quality, and I'm going to pull the exam-writer's trick and reformulate the question: what should policymakers know about the history of "teacher quality" in the U.S.?

Short answers: the long shadow of character, the education bootstrap, the short history of the single salary schedule, and the porous nature of certification/licensure. 


The long shadow of character

First, teaching as a career is less than 150 years old. In North America teaching was largely a short-term and part-year occupation until sometime in the 19th century (depending on where you're looking). In part because of the mix of private and taxpayer funding, and the short sessions in many places in the country, few people in the early 19th century could make a living teaching full-time. So many of the mostly-male teachers were in schools only part of the year, filling in when they didn't have opportunities to preach, attend college, or engage in something else.

Because of the multiple missions of schooling, academic qualifications were low on the priority list for those hiring teachers. The key qualification was high character, and the most common practical qualification was the ability to control a classroom. The source in many history of ed texts illustrating the second is Edward Eggleston's The Hoosier Schoolmaster (1871), a novel whose subtitle tells the tale: "A Story of Backwoods Life in Indiana." At the beginning of the story, the new schoolmaster is asked by trustee Jack Means,

"Want to be a school-master, do you? You? Well, what would you do in Flat Crick deestrick, I'd like to know? Why, the boys have driv off the last two, and licked the one afore them like blazes. You might teach a summer school, when nothin' but children come. But I 'low it takes a right smart man to be school-master in Flat Crick in the winter. They'd pitch you out of doors, sonny, neck and heels, afore Christmas."

In the nineteenth century, a smart teacher had the ability to control older boys, presumably by making them smart when necessary. 

That wasn't universally true; one of the common arguments for hiring women as teachers rather than men was their presumably nurturing nature. The gender stereotype of who was the right teacher inevitably involved questions about who could properly motivate students, especially boys. Never mind that women could use a switch on a student as easily as a man could. Or the rather clever way that hiring women allowed urban school districts to have a workforce that was cheaper and less likely to hop to other jobs -- because women had fewer higher-paying opportunities. (The same dynamic was true with African American teachers in the 20th century, at least until the 1964 Civil Rights Act; teaching was one of the best opportunities for upward mobility.) The rationale was all about sweetness and light, nurturing and character.

The legacy of all that history is that academic qualifications became an issue decades after the spread of mass primary schooling in the North. Part of the resistance was a fear of centralization; as New York state politician Orestes Brownson said, once the first normal schools were established, then states would try to work it so that no one but a normal-school graduate could teach. (He was partly right; see the "porous nature" section below.) Concerns about morality led schools to bar women from teaching after marrying, then forcing maternity leave when pregnancies began showing. Even now, morality will trump academics in the news. When was the last time your local television news show ran a story about teacher qualifications (either academic background or effectiveness)? When was the last time it ran a story about a teacher having sex with a student?

As I have argued elsewhere, this focus on virtue has caused serious long-term harm in how we look at teaching. And in the long run, those who argue about whether it's most important to intervene in teachers' disciplinary background, pedagogical training, or effectiveness in raising test scores are having a debate that could not have existed 100 years ago. So to the partisans in that argument, you are all light-years ahead of Jack Means and his real-world counterparts.

The education bootstrap (as in lifting up onself by one's, not engaging in violence with a)

Teaching was not a career in the early 19th century, but women could be teachers by the middle and end of the century, because the start of mass schooling generated an adult population with at least a minimum of formal education. A few weeks ago when I heard Joseph Kisanji of the Tanzania Education Network talk about the state of special education in Tanzania, what struck me was the low proportion of primary students who continued to secondary grades. That plus the high fertility rate in Tanzania puts the country behind the eight-ball, having a very high ratio of children in need of a teacher to adults with enough education to teach. Add sex discrimination in the form of requiring girls to work and thus discouraging them from secondary school, plus the legacy of "villagization" in the 1970s (the Tanzanian equivalent of Soviet collectivization) and you've got a serious dilemma for the country. While the average student-teacher ratio is something like 50:1, according to Kisanji in some areas of Tanzania, that ratio is 70:1, 80:1, or even 100:1.

At some point, that dilemma exists with every population, at least in the abstract if not with 100:1 ratios) because you start out with less knowledge in the adult population than you'd like, and to get there, you first need a critical mass of adults who are both educated and also willing to teach. Let's call that the educational starting hole.

The United States essentially lifted itself out of the starting hole through coeducation and mass primary education (even if it was inconsistent). The pool of available teachers grew in the 19th century with the willingness (and eventual preference) to hire women and also by declining fertility and mortality, so that the proportion of the population in elementary and secondary school ages shrank. That demographic transition gave the next few generations a chance to keep expanding the critical mass of educated adults.

One stumbling block since WW2 has not been the number of adults with bachelor's degrees but the consequences of reduced discrimination for fields such as teaching that have historically relied on discrimination elsewhere as a recruiting device. In terms of generating an educated adult population, we're doing fairly well as a country. (That's an historian's hindsight, not a statement of satisfaction.) What is remarkable to my historian's eye is that so few college graduates today need to enter teaching to satisfy the bulk of school needs. The struggle to attract great college graduates to teaching is less the total number of graduates than the question of who goes into teaching and the alternatives that pull potential teachers into other fields.

That doesn't mean that teachers know everything they should. The accessible availability of "content knowledge" (an awful phrase, to be honest) is far more widespread than access to great repertoires of teaching techniques and the opportunities to practice them. There's a long story and debate there, and I'll just suggest that while you can learn a great deal about physics online from Walter Lewin, there's little parallel for how to teach high-school physics. (Fans of sciencegeekgirl, please understand I'm talking about videos... I know there's plenty of text-based material online.)

This also suggests that what Tanzania desperately needs is to boost its secondary schooling. The country is one of the world's poorest, and while it is not in the same awful shape as Zimbabwe or Darfur, that's saying very, very little. Get a critical mass of young adult Tanzanians reasonably educated, and the following generation will be much better off in part because there will be a greater mass of potential teachers.

In terms of the U.S., we should understand both where our strengths lie (a much more broadly educated adult population than many countries) as well as weaknesses. Maybe one example will illustrate: the teaching of math in elementary and middle schools. In the past two decades, there has been a generational change in the amount of math that high school graduates have taken, especially among girls. (This change comes from the mid-80s increase in graduation course requirements in many states.) At the same time, there has been a deliberate effort to improve the teaching of math. I'm not going to get into the debates over the 1989 National Council for the Teaching of Mathematics standards statement or its more recent revisions, but the discussion is out there in the ether and should not be ignored.

The first trend is something that is a strength as far as the academic skills of potential teachers is concerned: more high school graduates (and thus college graduates) have exposure to math through any level than before 1982 or 1983. It is certainly not universal or complete; there are still too many elementary teachers who fear math and pass that fear on. (As far as I'm concerned, a single teacher who does so is too many.) The second trend? I'm not sure, and I'll hedge my bets by referring to Larry Cuban's hybridization thesis: I'd bet more elementary and middle-school teachers are using manipulatives and activities that try to "make sense" of math, but probably few are engaging in what its critics might refer to as unstructured teaching in the name of constructivism. Some part of that, but probably not much, is related to a deeper understanding of how children do or could learn math. Both issues (knowledge of math and knowledge of teaching math) have changed over the past generation or so. One of them, possibly both, is likely to be responsible for Florida's steadily increasing math scores on NAEP for eighth-graders since 1990.

The history of the single salary schedule

Advocates of differential and performance pay for teachers sometimes portray the single salary schedule as a long-term legacy of an inefficient bureaucracy, and that's partly true. You can find some sort of salary schedule in the growing school bureaucracies of 19th century cities. But there are some substantial features of salary schedules before World War 2 that suggest how short the single salary schedule's life has been.

First is the difference between elementary and secondary teacher pay. In Philadelphia, the teachers at Central High School were treated like royalty in comparison with all other teachers in the system, at least at the beginning of Central's life when it was the only high school in the city. Teachers were called "professors," were paid much better than elementary teachers, and were largely autonomous. And they were men. As Philadelphia added more high schools, Central High and its teachers lost prestige and authority, but the gap between elementary and high school teachers was persistent and reflected in the structure of teacher organizations (including nascent unions) and pay.

Second is the treatment of teacher pensions and gender. In many cities in the mid-20th century, pensions had conditions that disadvantaged women who had children. In Nashville, for example, I've come across age guidelines that eliminated all teachers who began a job over age 40 from being eligible for the pension plan. What that did was eliminate from pension plans the women who taught for a few years before having children, left teaching as their children were growing up, and then wanted to return to teaching later.

Third is the persistent racial inequalities in teacher pay, even after the 1940 Melvin Alston equalization case. Scott Baker has argued that in the fight for teacher equal pay, many Southern school districts began to use the National Teacher Examination as a basis for pay differentiation after they were told that African American teachers scored lower on the NTE than white teachers.

In the history of teachers in the U.S., the development of bureaucratic pay schemes fit comfortably with discriminatory practices, and one of the victories of unions, civil rights activists, and women's civic groups has been the elimination of explicit discrimination in pay schemes. Need one require a single salary schedule to maintain that accomplishment? I don't think so, but to ignore the history is foolish, and there needs to be a watchdog so that there isn't a resurgence of pay discrimination among teachers.

The porous nature of certification/licensure

Nineteenth-century New York politician Orestes Brownson was partly right when he thought that the creation of normal schools would centralize the qualification of teachers. The normal schools of the 1800s became recognized and eventually grew to teachers colleges and regional state universities, and "teacher training" has become a common feature of what people do before they become teachers.

At the same time (and in a related way), if slowly, inconsistently, and unevenly, school administrators began to give preferences or require teachers to have some formal training, whether provided at county training schools or in state university schools or colleges of education. As the curriculum expanded in the early 20th century, administrators pushed the  generally minimal state bureaucracies to expand specialized credentials (or endorsements); one mark of the expansion of special education in postwar Tennessee, for example, is the creation of a licensure specifically for special educators. 

By the late 1960s and early 1970s, the vast majority of states had a licensure structure for teaching that at least nominally required licensure for teaching, recognized divisions between elementary and secondary education, and recognized specializations at all levels (whether subject specialization in secondary education or specializations in services provided at multiple levels).

The alert reader may note that I did not claim that teacher education has a lock on teacher training or other professional-role entry in schools. Far from it! Even when states have established laws mandating that permanently-appointed teachers have licensure, the loopholes have been plentiful and large. Substitutes and temporary or emergency licenses have been common ways around certification/licensure requirements, and the proliferation of alternative certification programs has eroded the minimal barriers that certification/licensure poses. I suspect it would be a feasible dissertation project to document that as we have gone through two waves of babies in the past 70 years, there has been a consistent pattern in licensure practices: certification/licensure is loose when there is a shortage of teachers and tightens when there is a surplus of regularly-licensed teachers. (Who says that history can't meet the Popper definition of science: there's a disconfirmable prediction! Okay, so the claim is probably trivial to document...)

The nature of the loosening depends on geography and period, but my guess is that poor rural and urban school districts have been the most likely to have a "fog the mirror" standard for teachers, even when there has been a shortage. (Another prediction that can be checked...) That is less a cause of unequal provision of teachers to disadvantaged children and communities than a consequence. But the historical fact is that licensure has developed but not enclosed teaching.

An article by Donald Boyd et al. in the December Educational Evaluation and Policy Analysis, "Surveying the Landscape of Teacher Education in New York City," is particularly interesting in that historical perspective. What Boyd and his coauthors convincingly demonstrate is that the NYC Teaching Fellows program has not really competed with college-credentialed new teachers. Instead, Fellows replaced the emergency/temporary licensure population of prior years. Consistently over this decade, college-credentialed programs have been unable to supply enough teachers for New York City schools. This pattern is not an anomaly. Instead, it demonstrates the historically porous nature of teacher licensure. 

Implications

I hate sections that are titled Implications. Yeah, right, as if I know all the implications of this: I don't. I can spot when a policymaker has a theory of action that ignores the history, but it's not clear how to draw lines from this history to current policy dilemmas. Not that I don't have some ideas about "teacher quality" policy issues, but this entry took about 6 weeks to take shape in the evenings and on weekends, while lots of other things took precedence, and this is the type of question that could justify a book. (Someone else take this on, please; I have enough to write about for the rest of my life!) The reader request was interesting enough to make me think about this in at least some depth, and for that, I am grateful. If you find this of some value, that's great, and please let me know in the comments if you can draw a straight line from this stuff to policy.

March 17, 2009

Longitudinal data systems, good; unique teacher linkage, bad

Diane Ravitch's blog entry this morning seriously disparages the value of longitudinal data systems, including the linking of teachers to students, and John Thompson's entry discusses the abuse of data by administrators. Essentially, both Ravitch and Thompson fear the brain-dead or conscious abuse of data to judge teachers out of context. That's also the reason why NYSUT (the New York state joint NEA-AFT affiliate) worked hard to convince the legislature to put a moratorium on using test scores to make tenure decisions; Joel Klein was moving very quickly, and I think UFT and NYSUT had good reason to believe that without the moratorium, there would be substantial abuses of test data in NYC (and elsewhere) in tenure decisions. 

My take: longitudinal data systems are a good thing, but linking teachers to students is a much more fragile undertaking.

Florida has a longitudinal data system that began in the early 1990s and has been used for 10 years to judge schools based on test data. Approximately ten years ago, I sat in a windowless room in Tallahassee as a Florida DOE member discussed the new A-plus system and a variety of technical decisions tied to it, and for which he had brought stakeholders and a few yahoos from around the state to give advice. I was one of the unpaid yahoos who had the great joy of flying in tiny airplanes several hundred miles a few times a year to give advice on the matters. 

We had so many matters to discuss that one minor conversation was almost overlooked: a state mandate that required that the FDOE link each student to a teacher primarily responsible for reading and math. One state official showed us a draft form and then explained the concerns he had about it: in his view, the state that had tried that a few years earlier (Tennessee) had multiple conceptual difficulties connecting individual teachers to individual students. But they had run roughshod over those concerns, and he anticipated that Florida would do the same.

It wasn't a matter of letting teachers off the hook (this now-retired professional staffer is what I think of as an accountability hawk) but logic and sense. How many physics and chemistry teachers help students understand algebra better? How many history teachers help students with writing or reading? For students receiving special education services in a pull-out system, do you want only the special educator to be responsible for a subject, or do you want both the general-ed classroom teacher and the special educator to have responsibility? This spring, my wife (a math major and special educator) is tutoring a local child in math on weekends or evenings; so who should get credit for how he performed on testing in the last week, his teachers in school or my wife? Today, you can add NCLB supplemental educational services (or after-school tutoring) to the mix. 

The larger point: even if you decide to wave away the concerns of Richard Rothstein and others, even if you focus entirely on what happens in academic environments, it is fallacious to link every student performance with a single teacher. If we are providing the appropriate supports for children, then the students with the lowest performance are the ones for whom such unique linkage assumptions are the least justifiable, because they may be receiving academic support from general education classroom teachers, from special educators, from after-school tutors, and maybe mentors or other providers in neighborhood support organizations (such as Geoffrey Canada's). Today, I do not think one can parcel out responsibility without making assumptions that have no basis in empirical research. Those who support individual teacher linkage have the burden to demonstrate otherwise.

March 13, 2009

Recession-and-education humor

You know things are weird when late-night comics start channeling socialist intellectuals:

"The president said we can't stick with the school calendar that was created during a time when most Americans were farmers, and he is right. We need a new school calendar for a time when most Americans are unemployed." Jimmy Fallon, March 11, 2009
"The postponement of school leaving to an average age of eighteen has become indispensable for keeping unemployment within reasonable bounds. In the interest of working parents (the two-parent-job-holding family having become ever more common during this period), and in the interest of social stability and the orderly management of an increasingly rootless urban population, the schools have developed into immense teen-sitting organizations, ..." Harry Braverman, Labor and Monopoly Capital (1974), p. 439

And then there's the pseudonymous community college dean's great quip:

"Surest sign of recession: our Admissions staff reports that it's raining men." Dean Dad, March 5, 2009

In the early 1990s, I collected a number of cartoons tied to my dissertation research (which eventually became Creating the Dropout), and the "what good is education?" theme was behind many of them. I forget who drew the 1993 or 1994 cartoon that had graduates walking across the stage, receiving parchment scrolls, and unrolling the scrolls to discover that each read, "Will work for food." Expect more along these lines in the next year or so.

Back to the serious issue here: The problem with arguing about the value of education is that the human-capital arguments are all at the level of a population. From a population standpoint, more education is a great thing. From a family or individual perspective, it's likely to be a good thing on balance, but you always take risks in individual choices.

As I pointed out in December (revised for One-Blog Schoolhouse), there currently is no mechanism for reducing risk of educational choices, but there are both costs and risks--students who attend or return to college face the opportunity costs of foregone income as well as the possibility/probability of having a lot more debt upon graduation. And we all know quite vividly now how graduation does not guarantee one a job. Recessions encourage people to return to school because the opportunity costs are much lower. Laid off? Hey, then there's no downside to attending community college for a few classes. That pattern also demonstrates why it's dangerous to ignore opportunity costs when discussing student debt, but the larger point is that recessions make it all too clear how educational choices involve calculations of risk and probabilities, not average rates of return.

March 12, 2009

Will Charlie Crist threaten a budget veto if the legislature won't go along with using ARRA funds?

Mark Sanford's political ambitions might cost 7,500 South Carolina teachers their jobs because he doesn't want his state to accept federal ARRA funds. Charlie Crist's political ambitions might save thousands of Florida teachers their jobs because he does want Florida to accept federal ARRA funds. There are some legislators in Florida who agree with Mark Sanford. Florida's constitution requires a supermajority vote in both legislative chambers to use nonrecurring funds for more than a small portion of recurring expenses, and for a day or two I was very worried not enough legislators would agree to use (nonrecurring) federal dollars to plug the hole in the state's (recurring) revenues for education expenses.

Then it occurred to me what Crist has the power to do if a budget reaches his desk without using federal dollars: veto the entire thing and call the legislature back into session. I have no pipeline to the governor's office, and I suspect that Crist's answer to any question along these lines would be, "I am sure that I will not have to do that." So this is likely to be unverifiable and therefore unscientific speculation, though I think my speculation is based on the right levers in the state. In South Carolina, Sanford has the parallel power, which in his hands might wreak enormous damage by striking out parts of the budget that rely on federal ARRA funds.

Two ambitious Republican governors. Two very different potential directions for their states' futures.

Joel Klein as DM

John Thompson's blog entry today, God Does Not Play Dice, is in response to Charles Barone's Ed Sector report on value-added or growth models used for high-stakes accountability. (It's on my to-read list along with the IES/Mathematica study on teacher ed programs and various other things.) Thompson describes a number of caveats and then says,

...none of my objections would be major if the model was used for purposes of diagnosis, science, or a "consumers' report." We should pursue social science fearlessly, but we must not play dice with the lives of teachers by evaluating them with some theoretical work in progress.

That plays off Einstein's quip, "God does not play dice," in reference to quantum mechanics. That comment always made me think that if God does not play dice, maybe God forces you to pick up the dice and roll.

And that gave me the image of Joel Klein as Dungeonmaster.

A troll has just entered your classroom. He has a mace, a strength of 11, and 16 hit points.

After the Cafeteria Blob you threw at us, I only have 4 hit points, and I lost my Spitball Blocking spell.

Fight or run away?

Better fight; if I run away, I lose the Memo Spindle.

Better hope you're lucky. You need to roll a 17 to block the mace, 20 to break it.

But you're only giving me a D12!!

This is New York. You're tough enough. Roll.

Inconsistency illustrated

From an Andy Smarick blog entry yesterday afternoon:

I can't imagine how sad and frightened thousands of DC kids and parents are today.... The scuttling of this program will have a swift and severe influence on about 1,700 low-income boys and girls.

From Smarick's blog entry about 10 hours earlier:

The schools chief in Baltimore unveiled a laudable plan last night to close low-performing schools...

From the Baltimore Sun article:

While in the past, closing city schools have phased out over time, Alonso wants to make all the moves this summer.

As I wrote last week on the issue, there's a substantive question of how you handle school and program closures. But if you assert emotional trauma when one program is going to end suddenly, you shouldn't expect readers to take very seriously your praise of another policy decision that has the same lack of transition. The "sad and frightened" line may well have been ridiculed as cheap emotional blackmail if it had been used by parents upset with school closures in Baltimore, the District of Columbia, or New York City. Yes, there's a serious question on change vs. stability for individual students, but if the question is there in one setting, it's there in all settings.

March 10, 2009

"Five pillars" and paranoia

I'm waiting--just waiting--for some yahoo to point to the "five pillars" language in the president's education speech this morning and say, "Hey, there are five pillars of Islam, too. This means...!" Then again, such a statement would require that the said nutcase would have to know something about Islam. 

And while we're speaking of conspiracies, is there anyone else who is completely fed up with the "birthers," who do not understand that people from different backgrounds can fall in love and have a kid who has to be born somewhere? At some point, the tin-foil lunatics have to face up to the fact that Barack Obama was born in the United States, even if he is half-Vulcan.

Walking the talk...

The education news of the day is the president's speech before the Hispanic Chamber of Commerce on education policy. No one should be surprised at the themes (investing in early-childhood education, raising standards, improving teacher quality, fostering innovation, and increasing access to higher education). Nor is the next level of policy surprising in terms of Obama's advocacy of raising caps for charter schools, promoting performance-pay for teachers, having tests that aren't just fill-in-the-bubble, shifting subsidized loans to the direct-from-fed lending, and so forth. (There are two exceptions, below...) The details are largely absent here (or at least the level of detail that would identify those who might agree or disagree with the details), but that's because the president's name is Barack Obama, not Bill Clinton. 

The two exceptions to the "yeah, yeah, I guessed he supported that" litany: state data systems and extending time spent in school. That's supremely wonkish and... well, more on those another time, but I did want to note them.

And, finally, it looks like Obama is starting to hit a post-inauguration stride on speeches. This isn't the high oratory of the May 2008 speech on race, but regardless of whether I agree with specific ideas, I think this had more of a natural rhythm than many of his speeches from the nomination acceptance through his inaugural address. I mean, many of his speeches from late August through late January are fine, but they're not Obama at his best. I know that this is getting much closer to the better end of the Obama speech spectrum. Well, except for one factual flub...

P.S. President Obama? Don't look at anything called "dropout rates." They're garbage. Look at graduation measures instead, even if they're technically weak.

March 9, 2009

Set-aside questions answered?

Per David Hoff's question Friday on whether the "spend it quickly" provision of ARRA will conflict with the mandate to set aside 20% of Title I funds in schools or districts not meeting AYP, the initial guidance on the relevant USDOE webpage doesn't appear to be that clear. But a few things come to mind:

  • In terms of a macroeconomic effect, 20% of the Title I budget of affected schools or districts may or may not be significant, but it will surely be swamped by the larger decisions taken in the passage of ARRA and also in states that decide either to take the money or not. From what I understand, one reason why the Japanese central government's claims to stimulate the economy in the 1990s was always exaggerated was because local governments didn't always follow through. At some point, the question in Washington will be, "Okay, what do we do if states refuse to take the money we're offering?" That's one of my fears in Florida.
  • If 20% is set aside but used at some point within 15 months, there's going to be a delay effect but I'm not sure how awful that would be. That sounds like a technical question that few of us in education are competent to answer.

I'm not an economist, but I suspect the gist of this is that the educational consequences of the set-aside will be a bit more important than the macroeconomic ones. I have concerns about the set-aside provisions, but unless an economist points out differently (calling Dr. Rouse!), the issues should be decided based on education policy.

Update: Sheesh, I forgot some critical words above... Title I budget of. I hope adding them makes my meaning clear... or at least a little less like mud.

March 7, 2009

Closures and transitions

There is a double standard at work right now in the discussion over the federal DC voucher program. I'm not just speaking of the evidentiary slipperiness Aaron Pallas pointed out Monday and Thursday. In addition, I am thinking of the argument that students in DC-area private schools on the vouchers should not have their schooling disrupted for a policy change, a curious argument to make by those who have been advocates of disrupting the lives of students in public schools when the schools are closed or staff are fired en masse. Voucher programs should be protected from disruption while decisions about public-school programs hedge on the side of disruption? Secretary Duncan is one of those who have tried to make both claims, and given his Renaissance program of school closures in Chicago, this inconsistency fails the smell test.

Beyond the issue of consistency, there is a legitimate question of how to handle transitions in ways that respect the legitimate interests of students and of adults in the system. Sy Fliegel spoke openly about program closures and transitions in Miracle in East Harlem (1993), his memoir of open school choice in Spanish Harlem (Community District #4). You think of that experiment in association with Deborah Meier? Good for you! But Fliegel makes clear that a number of programs were clearly failures, either of ideas or mismatch with the community or management problems. As he explains in the book, his job was to handle program closures and to do it in a way that left the least pain possible. For some reason, his approach struck me as something akin to the attitude toward business owners when an idea fails: "Okay, that didn't work out. Dust yourself off and try again, and thanks to the separation of personal from business assets, you're not wiped out." Well, not quite a painless exit: there's the opportunity cost of time spent running a business. But a failed business is not inherently a scarlet letter, and Fliegel wanted to make clear that failure in developing a Community District #4 school did not mean that an educator (or set of teachers) was worthless.

For some reason, though, neither school systems nor wannabe reformers have paid much attention to Fliegel's approach, either in dealing with schools in crisis or in closing down schools for financial reasons (as in Rhee's closure of schools in DC). In the first case, teachers are often "passed around" or told to sink or swim in a transfer/waiver system. Neither approach is appropriate. 

Students are also affected deeply by any massive transition. Some welcome the change (when schooling dramatically improves) or are resilient. Others have their lives traumatized. My daughter had her preschool experiences disrupted twice when one school's management fired her teacher, and a few years later in a different city, the management fired the director. My daughter did well through the transitions, but the corporate approach by the centers in each case simply stank from the perspective of most parents: the employee was shepherded out of the center by security personnel with no chance for the children to say goodbye to a teacher or center director that they loved. In neither case was the termination for disciplinary reasons or anything where the escort was for safety reasons. It was just a knee-jerk "this is how terminations are done; damn how this will teach children that some people are disposable" decision. Sometimes reformers and managers forget that children are watching what happens and how the adults around them are treated.

March 6, 2009

"Cut the c***, governor, and tell us what you'll support"

I'm reading between the lines on this report of a Florida legislative committee hearing from yesterday, but I think the reported interaction between (majority) committee members and the governor's representative is an indication that my guess a few days ago was correct: Florida's state senate leaders are almost to the point of saying out loud how fed up they are with political games.

Next Friday, naf eht stih tihs eht with the March revenue estimating conference, when the state's economists will reveal the state's equivalent of the Fed beige book: we're in deep trouble still, deeper than the assumptions built into the governor's budget proposal. As I've said, Governor Crist is the second most disciplined politician I've seen in my life, but it's one thing to strategize over one of your campaign promises and it's another thing to put the entire state budget at risk for something... I'm not sure what, but something. And while I disagree with state lawmakers who want to refuse federal recovery funds, I understand their not wanting to stick their necks out for unpopular tax increases or long-term commitments if the governor's going to claim that the state can increase education funding with no tax increases. This is a rosy budget proposal that Salvador Dali could write. 

In the least painful of all possible worlds, the state would accept federal assistance, streamline sales-tax collection to make it enforceable with online sales, eliminate egregious sales-tax exemptions, raise tuition 5% and allow universities to raise tuition 10% more as long as large chunks go to financial aid, raise the state sales tax one penny for three years, and raise the cigarette tax to $1.50 or $2 a pack. What is easiest politically are the elimination of a few hundred million dollars of sales-tax exemptions, the tuition hikes, and a cigarette tax of $1 a pack, possibly the streamlined sales-tax collections. But that's not going to be enough to do what the state needs over the next year and make schools, colleges, and universities a priority. I hope the legislature does more than what it's signaling now is likely. As with the recovery package, half a loaf is better than none, but it's my job as a citizen to point out that we really need the whole loaf.

March 5, 2009

Sherman Dorn is called a dirty name: behaviorist

Not really, but in this week's Education Sector forum on technology and assessment, Scott Marion of the National Center for the Improvement in Educational Assessment comes close. In reading Bill Tucker's report on the subject, I had previously been concerned that it was too close to boosterism for bells-and-whistles approaches, broad claims that we can address concerns about the quality of testing if we can just get stuff online. Yes and no, I thought (as I explained February 20). So when the online discussion this week mentioned one alternative, the quick-and-dirty approach of curriculum-based measurement (CBM, also called progress monitoring), I asked a question:

This type of formative assessment is not only too-often ignored by general-ed researchers, it's also being ignored by the think-tank community. If you read the NCIEA's The Role of Interim Assessments in a Comprehensive Assessment System: A Policy Brief (recently published), you'll see that the report entirely skips between a definition of formative assessment that is entirely informal and casual to the type of periodic/benchmark assessments that are much more complicated than CBM/progress monitoring. What's necessary to turn technology-based assessment away from the bells-and-whistles assumption and devote enough attention to the "here's what we can do now that has documented research support?"

Marion responded in part as follows

Sherman sounds like a supporter of CBM, so he should probably be happy that we stayed away from doing a critical analysis of this enterprise. I have only begun to look into CBM and what I've seen makes me very nervous. These multiple "measures" are treated as if the inferences drawn from each of the measures are based upon some sort of valid equating. Everything I've seen thus far-and admittedly I need to look into this much more--suggests that inferences made about student growth and not supported by a psychometric foundation.

Hmmn... there's a broad base of research on curriculum-based measurement, and Marion blithely skips by that. Essentially, CBM suggests a regime of regular testing that samples the entire year's curriculum. Is it possible to construct alternative forms that aren't precisely equated? Absolutely. But the same standard should be applied to so-called benchmark/periodic assessments, and as far as I'm aware, the NCIEA brief referred to above doesn't raise that issue at all. The deeper technical question is whether score movement reflects real achievement change and whether lack of score movement indicates real achievement stagnation, and that's a thorny issue for almost any system of measurement you can shake a stick at. I'll let Stan Deno, Lynn Fuchs, and other researchers in the area defend themselves, but there was something in Marion's answer that just didn't sit right. (Among other things, someone who claims to be an expert on assessment who has only "just" started looking at the CBM literature? It's been around for more than 20 years.)

Then there's the label slapped on at the end:

Finally, and related to Bill's report, CBM fits smack within a fairly outdated behaviorist conception of student learning.

I've apparently become a behaviorist in my free time. Quick: someone find me a deprogrammer! If you're curious, step back and think about the decisions a teacher has to make about classroom time and his or her energies. Might it be useful to give a teacher or a principal a tool to gather information efficiently, with little time stolen from classroom instruction? At least theoretically, a consistent sampling frame for CBM could include free-response items with interesting cognitive demands. Not a problem! Well, until you get to the nuts-and-bolts of practice, when you need to write and decide on items. Then it gets interesting. There is nothing wrong with quantitatively-scored items used for screening/informational purposes as long as those tests are not taken as the only source of legitimate information on children (and I don't believe I've seen that in the CBM literature). If Debbie Meier thinks it's valuable for third-grade students to learn their times tables, I think we can safely assume that some mix of information is fine, and the behaviorist label is pretty silly.

Besides, I think we experienced silly arguments about two decades ago around the construct of "authentic" assessment, and at least a few states spent millions and millions of dollars on performance exams that were beautiful in theory but problematic in different ways from multiple-choice exams. If I remember correctly, at least one or two studies from the U. Minnesota National Center on Educational Outcomes concluded that performance exams gave children with disabilities at least as many problems as multiple-choice exams.

So let's not reinvent the wheel and cast aspersions on our least favorite formats. It's the use and not the format that matters. I don't want to see a few billion dollars spent entirely on blue-sky test development projects when a few million dollars could develop something that is practical across millions of classrooms. Let's ensure that at least some of the current money is spent on stuff that's useful today and also invest in long-term development projects.

Incidentally, I should have given credit to Charles Barone for his suggestion of open-source testing. I knew I had seen it somewhere, but the only stuff I could find for the February 20 entry was something else.

March 4, 2009

Higher-ed policy conundrum: I'm a cheap date, but my brother isn't

Yesterday's hearing on federal science funding highlights one of the dilemmas of increased funding: how do you do it in a way that is sustainable and does not lead universities to invest in a research infrastructure built on untenable assumptions (i.e., building lab space that will be empty when funding falls and hiring postdocs who will have to be let go once a single grant period ends).

But that's not the major problem I see. One of the inevitable tensions in the Obama administration's higher-education policies revolves around the relative investments in teaching institutions vs. research infrastructures. The vast majority of college students attend nonselective 2- and 4-year public institutions, and if one goal of President Obama is to increase the American public's time in higher education, that is where the time will expand. To do that without making the initiative counterproductive, both the federal government and state governments will have to put money into teaching institutions. For community-colleges, that investment is usually an easy sell, up to some limit; community colleges always argue for their budgets as cheaper than universities, student-for-student. And for state 4- year colleges and universities that offer no more than master's degrees, that's also fine.

But for public universities that either claim to or aspire to conduct significant research, the policy focus on undergraduate education is in tension with another Obama administration goal: increasing research in health and science, especially that connected with energy conservation or renewable energy production. That tension isn't direct: ask any president of a large university if it can both educate undergraduates and conduct stunning research in bench sciences, and the answers, "Absolutely." Rather, the tension is subtle, indirect in terms of the implied investments at the state level.  Last year, Kevin Carey asked why Illinois cheated Chicago State by favoring the University of Illinois, and his pointed question is another form of the one any state should ask: how do you divide the available dollars between education and research?

Part of the question is about teaching loads of individual faculty, but that's not really true in major research universities. Even if you tell a department chair to produce N student credit hours, that doesn't tell the chair how many classes an individual faculty member teaches. There are a variety of ways to provide time for faculty to research. Large universities can shift teaching from tenured and tenure-track faculty to contingent academic labor. Wealthy liberal-arts colleges have generous sabbatical opportunities to compensate for consistently heavy teaching at other times. Time is the cheaper resource to provide faculty, relatively speaking. To put it bluntly, as an historian whose research interests lie in the U.S. and where my projects sometimes rely on secondary analysis of quantitative data available for free, I'm a cheap date. Give me time, a reasonably up-to-date computer, occasional funding for travel, and I'm on my way.

But my brother's another story. He's a geomorphologist at Arizona State University, and his research activities have required spectometry and X-ray microscope time as well as equipment, graduate-student funding, and travel funds to collect specimens in the field. And a lab at the campus. The funding required to make this work is a different order of magnitude from what I do. Then there's comparative medicine; a former VP of finance at USF once told me that it is cheaper to house me in my office than a lab mouse in a tiny cage.

Because there is no feasible way to make bench sciences and medical research operate entirely on grants--you can try to do that for a time, but funding rates go down as well as up, and you can't expect institutions to rebuild an infrastructure from scratch every time that federal research funding spikes--there has to be some decision somewhere on research infrastructure investment. Here, sabbatical opportunities are nice but don't begin to satisfy the bottom-line needs of research. This is investment in equipment, in reasonable expenses, and in people as well--the professional lab employees, the graduate students--and bridge funding to preserve teams is one sensible (I'd say required) element in building a research infrastructure. 

Easy, says the observer who hasn't lived in Florida: just limit which institutions engage in capital-intensive research. Then you can concentrate the necessary funds, gain economies of scale, and we can satisfy both goals reasonably. I've seen two attempts to do that in the state in the past decade, and both efforts were swallowed whole by the maw of the Higher Education Status Machine. The Status Machine is fed by the nature of modern academic administration and also by the local booster role of higher education. In Florida, we have runaway institutional ambitions, where even a small community-college president in the panhandle dreams of turning his institution into a four-year college, and where half of the university presidents are insulted if you point out that their research programs are just a wee bit smaller than Princeton's. Then the legislature gets into the game: the soon-to-be (and now erstwhile) House Speaker maneuvered an "everyone can become a four-year-college" bill through the legislature last year, funneled millions to his friend at the now Northwest Florida State College, and was still less wasteful of state resources than the legislators who pushed through two new law schools and three new medical schools in the past decade. Huey Long would be so proud: in Florida, every man can be a king, just as long as she or he runs a public college or university.

Let me step back from my local and immediate cynicism. One of the persistent patterns in U.S. higher education is the upward institutional status trajectory over time. Many normal schools later became teachers colleges and then undergraduate state colleges before they transformed late in the 20th century into universities with significant research programs. Boosterism likewise not a new phenomenon. And the expansion of doctoral programs focused on post-degree employment at research institutions has played a role in this, even if it is much smaller than historic boosterism in higher ed and administrative status envy. But the latter two? Let's put it this way: no candidate would say the following at a campus interview for a public university presidency: "You're perfect as you currently are, and I will do nothing to advance the institution beyond where it is now." 

I am not certain if there is a solution to this dilemma. Theoretically, a state could simply divest itself of research ambitions. That appears to be what Arizona is doing, and it's going to be a disaster. A state could underfund its community and 4-year colleges, which is what Carey accuses Illinois of doing. A state could attempt to ration the upward ambitions of institutions, but you can see how short that idea lived in Florida. You could also pretend that a state's public institutions can be all things to all people. That's where Florida is now.

At the federal level, I suspect that this topic won't even be discussed, because there is no easy solution and the same booster dynamics exist at the federal level (witness the earmarks throughout the federal budget to help specific university-based projects). There's also the dynamics within an administration to consider: to put it honestly, someone like new White House aide Roberto Rodriguez might be articulate and sharp, but that's not going to hold a candle against a cabinet secretary with a Nobel Prize. It would be great to be a fly on the wall for the conversation, though.

March 2, 2009

Raising a stink

My home office stinks right now. It's a temporary "new plastic object" stink from the large inflatable ball I'm sitting on in an effort to strengthen my back muscles and keep me from being completely sedentary while typing.

My on-campus office stank this morning. The electricity went out in the building sometime over the weekend, and somehow the HVAC smelled not only as if an animal had died in the ductwork but that someone had newly painted the dead animal as well. So I spent most of 6 hours figuring out how to conduct some work well away from the building. (If I were of a conspiratorial mental bent, I'd assume this was connected with the fact that my Outlook installation hasn't been able to connect with my campus account in about 3-4 months... but I found a workaround for that, and I figure an hour or two of lost work from moving around isn't a tragedy. And the staff were the ones stuck in the building for their work.)

There are two glorious thing about these stinks, however. One is that they are likely to be temporary. Offgassing plastic is highly unpleasant, but it ends. I know, I know; I'm ruining various organs in my body. But I like my back muscles, and I'd like them to like me back. Tradeoffs... And there are enough faculty, staff, and students who traipse through my ugly yellow-brick campus building that getting the HVAC working right will be a priority. 

The second glorious thing about these stinks is that I can smell them. Yes, this is an improvement in my quality of life, at least compared to last Tuesday, when several people told me I looked like death warmed over. By Friday, I was death microwaved. Today, I think I'm at least room-temperature.

But there's the other side of this; too many people work in fairly-permanently stinky buildings. (The official term is "sick building syndrome," but mold stinks.) I hope that the stimulus funding for renovations addresses the buildings on my campus, on my wife's long-term school campus, and other places where it can be permanently unhealthy to work or study.


Take a breath (if you don't have asthma) and go on

I don't have asthma, but as my head cold morphs into the ordinary misery of seasonal allergies, I realize it's a darned nuisance not to be able to breathe comfortably. With luck I'll shortly be back to normal (or at least for what passes as normal for me), and in times like these, it pays to take a deep breath on receipt of almost any news and criticism. Evidently, my perspective lies somewhere between former Hill staffer and new DFER policy guru Charles Barone and NYC union activist Norm Scott, because I'm getting dished on by both. I'm not going to use the lazy journalist's excuse, "Because both sides are criticizing me, I must be right," in part because I'm not a journalist, in part because it's easily possible to be wrong about multiple things at once, and in part because while I disagree with Barone's and Scott's posts, they (generally) have the guts to say where they disagree with me. Oh, yeah, and they spell my name right. That counts for a lot with me.

Barone criticizes me (and others) for writing too much from an adult's perspective. I've written about that topic before (at length in Accountability Frankenstein and in more digestible chunks in One-Blog Schoolhouse), so let me provide a somewhat different gloss here: I could easily turn my blog over to several guest writers, my children and their friends. I suspect Barone's response to their criticisms of high-stakes testing would be, "Well, I know a little more about the world and your own best interest than you do." That statement would be absolutely right (at least in the first half) and an absolutely adult perspective.

(Incidentally, I agree with his substantive point in his entry that teacher happiness is not the point of either education policy or teacher education. I don't think that you can usually have effective teaching with completely miserable teachers, but I suspect or at least hope Barone would agree with me, and there's plenty of ground between avoiding total misery for teachers and seeing their euphoria as the primary goal of policy.)

Scott criticizes me (and others) for ignoring the fact that Arne Duncan was flawed as head of the Chicago Public Schools. Er, no. I'm fairly sure I'd have disagreed with him on a number of his decisions in the same way that I am fairly confident on where I'll disagree with him on federal education policy. But that open expectation of some disagreement does not mean the Obama administration is evil. Scott asks, "Exactly how much 'context' do these people need?" I'd say 20 years of Republican presidencies divided by 8 years of Bill Clinton. In comparison with Bill Clinton on the whole, Obama is good. And in contrast to the others, he's very, very good. That doesn't mean that I'm going to stay quiet when I think the administration is doing something wrong. It means I do have some perspective. Breathe, folks, breathe. For those who are worried about Arne Duncan, I think you'd do much better to putting your energies into worrying about Timothy Geithner instead.

February 27, 2009

Education funding politics in Florida

Later this morning, a few thousand teachers and their friends will be gathering in Orlando to rally on behalf of public education in Florida. The Make Our Schools a Priority campaign has been making waves, most recently with a Brevard County 10,000-person "town hall" meeting with the county's legislative delegation. If I weren't still coughing and sneezing, I'd be on a bus this morning to Orlando, but I don't think my friends from other unions really want what I have, and I trust that there will be enough people there.

Until Governor Crist announced his proposed budget at the end of last week, most people I'd talked to had assumed that both the governor and leaders from at least one chamber were open to increasing revenues. But then Crist dropped a bomb and suggested a budget that relied heavily on the federal recovery package, a gambling pact he wants the legislature to approve, and a handful of user fee increases. In other words, Crist refused to be out front in support of tax increases. Instead, he's set up the dynamics of state politics to be able to lay the blame elsewhere for either tax increases or service cuts.

If I were a legislative leader, I might well feel as if the governor had stabbed me in the back, especially if I had been discussing tax increases as one option. As a voter, I think the proposed budget is Crist's most cowardly moment in office, because it is unnecessary to defend his popularity (his ratings are high) and because it bollixes up what could have been a reasonably civil legislative session in horrible times. Instead, the session will start out with legislative leaders who will not be able to trust the governor to work with them and provide them cover for tax increases.

This is why the campaign on behalf of education funding is crucial, providing external pressure on legislators and convince them that not only is funding education the right thing, it's also the politically smart thing.

Just call it like it is, after you've paid your dues

It looks like a bunch of interested parties are starting to judge the Obama administration based on its appointments and early policy direction. And that's just fine. But when there's Fordham's Reform-a-meter, and Diane Ravitch proclaims Duncan's USDOE to be Bush's third term, I'll chime in with Fred Klonsky: judge people for what they do, but remember the context.

Thus far into the Obama administration, I'm fairly sure on the majority of key issues where I'm going to agree with the administration on education policy, where I'm going to disagree, and where I'm not going to be sure or not going to care. That leaves some issues where it's not clear where the Obama administration is going. I'm willing to call out administration officials when they make mistakes, as well as give them credit when they shove things in the right direction.

There's a totality to be considered: even if the Obama administration goes way too far in the direction of paying teachers for student test scores, they still get credit in my book for pushing a recovery package that will save thousands of teachers' jobs (even if the package was too small). And for proposing to index Pell grants, shift all subsidized loans to the direct program, etc. If you really expect to agree with everything a president does, you need to run for the office yourself. Other than that, expect to disagree with a few hundred decisions of the person you voted for, because presidents make thousands of decisions every year.

Case in point: FDR, who did a bunch of great things, but here's an incomplete list of the completely sucky actions of his administration during 12+ years in office ("completely sucky" is a technical term in policy evaluation):

  • Forcing Japanese Americans into internment camps during WW2
  • Allowing the Federal Home Loan Bank Board to create redlining
  • Not pushing Congress on anti-lynching legislation
  • Not putting more teeth into the Fair Employment Practices Commission
  • The practices of the Agricultural Adjustment Administration, which gave landowners in the South an incentive to push tenant farmers and sharecroppers off the land
  • The court-packing scheme
  • Deciding to cut back on stimulus spending early in his second term, which created the 1937 recession within the depression.

The stupid or immoral decisions did not eliminate the great ones, nor the converse. In the same way that book reviewers have an obligation to recognize what authors are trying to accomplish, there's a similar obligation when evaluating a segment of an administration's policies: pay your dues to the context and then call it like it is. The Bush administration was a disaster in many, many ways, so the fact that it pushed assistance with AIDS responses in Africa was a tiny good thing in a morass of incompetence. I suspect that my long-term evaluation of the Obama administration will be the converse in many ways.

And right now, like Paul Krugman, I'm more worried about the economy than performance-pay policies.

Real news on education

For some reason, U.S. News and the Christian Science Monitor decided to take my comments on the president's speech Tuesday out of context and spin one clause as an "I'm not impressed with the president" remark. Sheesh. I'm generally happy with his actions thus far as president, but I can also recognize that there was relatively little emphasis on concrete education policy Tuesday night. Stating a desire for higher educational attainment is not exactly new in presidential speeches.

The real news on higher-ed policy this week came yesterday with the proposed budget and how it addresses college affordability. It's one thing to say that we want people to attend and complete college, and it's another to propose how to get there. For any reporter who happens to read this, I'm delighted with the proposals to index Pell Grants, shift loans to the direct-loan program, and create a partially refundable tax credit for tuition and fees. Here's where the most important battles will be fought.

February 25, 2009

On exaggerations in the service of bitterness

Today, Charles Barone indulged in some recriminations about the use of test data to evaluate teachers: "In fact, in many states there is tremendous pressure to pass legislation which assures a firewall-like separation between teachers and student performance. Such laws have already passed in California, New York, and Wisconsin; ..."

But let's examine that claim with regard to New York, about which others such as Kevin Carey and Jennifer Jennings wrote last April. The language:

3012b. Minimum Standards for Tenure Determinations for Teachers.

(a) A superintendent of schools or district superintendent of schools, prior to recommending tenure for a teacher, shall evaluate all relevant factors, including the teacher's effectiveness over the applicable probationary period, or over three years in the case of a regular substitute with a one-year probationary period, in contributing to the successful academic performance of his or her students. When evaluating a teacher for tenure, each school district and board of cooperative educational services shall utilize a process that complies with subdivision (b) of this section.

(b) The process for evaluation of a teacher for tenure shall be consistent with article 14 of the Civil Service Law and shall include a combination of the following minimum standards:

(1) evaluation of the extent to which the teacher successfully utilized analysis of available student performance data (for example: State test results, student work, school-developed assessments, teacher-developed assessments, etc.) and other relevant information (for example: documented health or nutrition concerns, or other student characteristics affecting learning) when providing instruction but the teacher shall not be granted or denied tenure based on student performance data;

(2) peer review by other teachers, as far as practicable; and

(3) an assessment of the teacher's performance by the teacher's building principal or other building administrator in charge of the school or program, which shall consider all the annual professional performance review criteria set forth in section 100.2(o)(2)(iii)(b)(1) of the Regulations of the Commissioner.

The part that was added last spring is in italics, but the rest remains, including clear performance references in bold. How are we supposed to read the combination of "the extent to which the teacher successfully utilized analysis of available student performance data... when providing instruction" together with the ban on granting or denying tenure "based on student performance data"? I'm not a lawyer, but obviously there has to be data for one to judge teachers on how well they use the data. My reading (which I think is plausible) is that one couldn't make a blanket decision based only on test scores, but you could grant or deny tenure based on how well a teacher used the data in adjusting instruction. This latter is pretty close to the best-world scenario of Response to Intervention (RTI) policy, which has a lot of research at least in core areas in elementary schools. In comments on Barone's entry, I wrote,

I think we may be reading the same legal language with very different lenses. To me, the tenure-qualifications language in NY state essentially conforms with RTI -- teachers have to show that they can use data. Those upset with the added language for this year -- which bars a brain-dead statistical formula -- must think it would be as appropriate and also easier to define effectiveness with test scores as what is currently allowed/required by law. Me? I don't think there's anything that's easy here to implement in a fair way, and there ain't yet no Holy Grail. I also suspect that there is no provision in NY law that prohibits the type of analysis of teacher education that Louisiana has been building for the last 5-7 years. Either I'm reading your definition of a firewall too broadly, or I'm misreading NY law.

Here is Barone's response, word-for-word (the bold-faced sentence is my emphasis):

It seems to depend on how you define "brain dead." The data can't be used, thoughtfully or otherwise, to inform tenure decisions. Whether there is a holy grail, or it hasn't been found, remains to be seen. But surely everyone agrees that poor and minority kids are getting the short end of the stick, and data available now can and should be used to help level the playing field for kids while we adults have our fun little debates. I notice you rarely use the word student or child, unless you are quoting me. I think we need to err on the side of the kids for a while even if it makes adults uncomfortable. If we wait for there to be a consensus among academics, today's kindergartners will be collecting Social Security before anything is done. If then.

The "bitterness" referred to in the title of this entry refers to this response. I'm disappointed by Barone's avoidance of the substantive topic by applying a rhetorical litmus test (how often I mention children in my blog), as well as the politician's logic here (something must be done; this is something; so we must do it). But let me get to the point: Barone is misreading the law. Data can be used to inform tenure decisions, and in fact, they must be, because the law requires that part of the tenure decision depends on teacher use of data. No data, no use of data -- no tenure. It may not be Barone's picture of how data informs a personnel decision, but Barone's claim is just plain wrong

Addendum: In comments, Barone argues that the New York state law is clear and bars use of test data for making tenure decisions. Here's the way to decide it:

1) Does New York law prohibit a district from denying tenure because a teacher refuses to implement Response to Intervention practices?

2) Is Response to Intervention something based on student performance data?

If the answers are "no" and "yes," respectively, I'm right. Any other combination, and Barone is right. Let's try another scenario:

Main office conference room, where the assistant principal is meeting with a new teacher. "Let's look at your student's last quizzes and talk about where they learned the material well, and where you might want to reteach."

The teacher holds up his hand. "Wait a minute. Am I going to be judged based on what I say in this meeting?"

The assistant principal nods her head. "In part, what I'm judging with your effectiveness is how you respond to student needs. C'mon. Let's just look at the quizzes."

"No way. State law forbids the use of student performance data in tenure decisions. I'm talking with my union rep!"

If Barone is right in the global sense, this conversation could really happen. But I don't think it could (or has). When Barone claimed that New York had put a "firewall" between teachers and performance data, I know he was thinking in the narrow sense of "if students perform poorly on standardized tests, then we should be able to deny tenure." But regardless of whether that is a good or bad policy, that's not the only way one can connect teachers and student performance. Expecting teachers to look at student performance and change instruction based on data is a second way, and New York does not bar it. Looking at teacher education and student performance is a third way, and New York does not bar it. Which of those three is good policy is an interesting and debatable question, but what is not debatable is that all three connect teachers to data.

One-Blog Schoolhouse: contents and index

Want to know precisely what is in the new One-Blog Schoolhouse? You can now look at the prefatory and index pages as well as the references (with one spelling error fixed, thanks to A.G. Rud).

One-Blog Schoolhouse prefatory and index pages

Publish at Scribd or explore others: blogging books

Okay, now you've seen the contents; go buy it.

Not quitters

I'm sure that there will be much tea-leaf reading in the next few days, but there isn't that much from the president's address that's dramatically news-breaking on education policy. The call for everyone to attend some higher education is a little new, but the theme isn't. The most obvious way to read the statements about reform, charter schools, and higher education is that President Obama is now the 6th president since 1960 to devote significant lip service to education as human capital and education policy as investment in human capital.

Well, that and Ty'Sheoma Bethea's letter. Let's hope she's part of a full generation of non-quitters.

Addendum: Apparently reporters took this entry out of contest; see Friday's note for a little more.

February 24, 2009

Sick leave and pensions

I'm taking one of my rare sick days today, and while I don't have that much stamina to concentrate for a long time, I'll note just a few thoughts about debates over teacher pensions a la Chad Aldeman's comments. First, the entire debate strikes me as largely unconscious of or compartmentalized from the larger context or debates over retirement in general. First, if pension plans are underfunded, the problem is not that they're overgenerous but at least partially that state legislators aren't willing to set aside the money to put them on a sound actuarial basis. Florida's retirement system used to be underfunded, and one of Lawton Chiles's primary accomplishments as governor in the 1990s was changing that. And I figure that if my dysfunctional state can fix a pension system, so can any state.

Second, Aldeman's point about looking at defined-benefit and defined-contribution systems together rather than in an either/or sense makes sense... and anyone who talks to a financial planner will (or should) hear the basic point they all make about the triad of funding retirement (Social Security, pensions, and personal assets). Michael Katz makes a similar point in The Price of Citizenship about the public-private nature of the modern welfare state that combines different categories of institutional structures. I don't know about you, but when Michael Katz and financial planners agree on a description of retirement, I'm going to believe that.

Third, the debate has this odd "leveling down" tone to it--not in the literature that Alderman is referring to but in political debates I've occasionally seen among state legislators. Because some people don't have decent retirements, then teachers shouldn't, either. (Somehow the higher pension payouts for police and fire aren't brought up in those discussions...)  I don't understand why that is either a practical or a moral claim for public policy, but that may be my unwell state. Maybe if I had a few gazillion fewer viruses in my system, I'd understand the reasoning better.

Finally, and perhaps most importantly: any concerns you or I might have about underfunding of defined-benefit pension plans should be dwarfed by concerns about health-care costs. Social Security's structural problems are a pittance compared to health care. State pension plan underfunding is minor compared to the looming costs of health care. If you're really concerned about the next generation, give some time in the next week to learning about health-care costs. I'd start with Peter Orszag in 2007, last spring (courtesy of Brad DeLong), or yesterday.

So we're back to my being sick again. Or health care, at least. Hope you're feeling better than I am.

February 20, 2009

Technology and assessment

Education Sector's new report Beyond the Bubble is shorter than I had expected, so I finished it while watching the end of my son's tae kwondo class last night. It looks to be a decent summary of the optimistic side of technology-and-assessment literature. Its tone is, "Yes, we can dramatically change and improve assessment with technology that is either just about to come online or that deserves some investment." And I think that for some things, that's absolutely right: an online/computerized science exam could have color images of tissue slides, videos of animal behavior, and so forth. But, while author Bill Tucker bowed his head in the direction of friendly technoskeptic Larry Cuban, there are some flies in the ointment:

  • Students with disabilities. This is true for pencil-and-paper tests as well, but when you only have black ink, there are a few other issues you don't have to worry about that on-screen designers have to: red-green color blindness, epilepsy and screen movement, etc. The half-page on universal design is good, and any CFP will need to specify (and budget for) disability/accessibility awareness.
  • Code creep. I don't mean internet safety but the fact that programming languages grow up and die. We've gone from perl to python, from HTML to XML, and languages and interfaces will continue to evolve. I wonder how many of the cases pointed to in the report are essentially one-off projects that will die at some point because the platform no longer exists. (Any readers remember Infocom's text games?)
  • Holy Grail syndrome, also known as a belief in "the leap in cognitive science that will allow perfect, automatic scoring of essays is just around the corner." Same with the great and brilliant analysis of hundreds of microstate data that a single student can generate in a simulation environment. I trust colleagues who work in cognitive psychology to do some great things in the next decade, but this seems a bit utopian. Okay, more than a bit.

All of this doesn't say we shouldn't be engaged in using technology, but maybe we should work along two tracks: encourage the fast, frequent, and flexible for now and also invest in the medium- and long-term projects.

There is something that the paper never addresses: intellectual-property rights. Part of the imprisonment of assessment in an oligopoly is the ownership of assessment materials, backed up by the fear of security problems. (Here's reality for you: the day after a state test is given, assume NO security for that test. None. Despite all the laws. Just give that idea up, folks, unless you believe in the tooth fairy, have never heard of BitTorrent, and don't think college students ever cheat.) I am curious what the position of various folks are on open-source assessment. I am not entirely sure what it would consist of, or how it would meet adequate technical standards, but it's tough to argue that despite the testing industry's oligopoly status, we should suddenly think that a brand-new investment will erase both the proprietary rights of the major firms or the start-up threshhold for the creation of commercially-viable products.

February 19, 2009

The new/old myths about tenure

If you read an entry earlier this month on Fordham's Flypaper blog, you might have the impression that Ohio Governor Ted Strickland rolled the unions by wanting tenure to be awarded after nine years and changing the dismissal standards to "just cause." You might also have the same impression by reading some news stories about the proposal.

You would have the wrong impression, though. As the Plain Dealer's coverage of the proposal explains, both the Ohio Education Association and Ohio Federation of Teachers were on the inside of roundtable discussions on teacher quality, and while they have some quibbles about the reform, it looks like they're generally behind them. That's probably because there's now statewide policy support for early-career mentoring. (The implementation in this budget environment's a different story.) And the quibbles? Surprise! At least as far as I can tell, it's not about the just-cause standard for discipline, since that's standard in union contracts around the country. Let's hear from one union representative:

Michelle Prater, spokeswoman for the Ohio Education Association, said her group likes the added support for beginning teachers, especially since new teachers are spending little time with mentors under the current entry-year system.

But it would be better to tie tenure to meeting standards than to grant it after an arbitrary number of years, she said.

So the quibble of OEA is that they're not getting enough of ... standards.

At Ed Week, Stephen Sawchuk exaggerates moves in Ohio, a bill in Florida, and Michelle Rhee's proposals to assert a mini-trend of attacks on tenure. This is both amnesiac reporting (as if no one criticized tenure before 2007) and overgeneralizing. In Florida, the legislature eliminated the term "tenure" from K-12 statutes years ago, and what happens now is that teachers receive professional contracts... which operate just like tenure. The bill proposed this year is just that right now, a bill. Michelle Rhee's move has attracted lots of attention, but we'll see where it goes. And Ohio's move is different from the others because, well, teachers are inside the process. Not necessarily agreeing with everything, but inside the process.

Now, for the policy questions, especially for the proposal by some of Florida's legislators: what makes you think that eliminating tenure is going to raise teacher quality? In most districts for the first few years (when a lot of teachers leave anyway!), teachers can be fired at any time and their contracts can be nonrenewed. In collective bargaining agreements, there are provisions for gathering evidence that a teacher has problems in the classroom, putting the teacher in a corrective or probationary status, providing support, and then firing the teacher. When administrators don't spend the time supervising teachers, they're not in a position to fire them. So, if tenure for experienced teachers is eliminated, and administrators are still not spending time supervising teachers, what's going to be improved? The most that advocates of tenure elimination can claim is that there will be slightly improved capacity to fire a few experienced teachers at the margins. And all that advocates usually trot out are anecdotes of the administrative headaches experienced when firing teachers. My guess is that behind each of those headaches is a history of passive or incompetent administrators.

Basic fact: it's only administrators who set a fog-the-mirror standard for teachers. It's not other teachers, because they know they'll have to deal with the mess left by colleagues who are incompetent or leave in the middle of a year. It's not unions, because no union officer wants to defend incompetents. (It's part of the legal duty of representation, and it comes with the territory--one of our vice provosts calls us the equivalent of public defenders--but it's not the most thrilling part of the job.) 

In higher education, the debate over "post-tenure review" was about 10-15 years ago, and in the case of Florida, one of our public universities was established on condition that it not grant tenure. Within a few years, from what I heard, administrators at Florida Gulf Coast University (FGCU) were pleading with their political overlords to create a system of rolling three-year contracts that remain in their first year as long as the faculty member is satisfactory on the last annual review. Why? No one wanted to come to Florida Gulf Coast without some stability, and they left for greener fields given the instability of fixed-term contracts and the hazard of renewal depending entirely on the whim of administrators in the last year. So FGCU ended up with a system that's hybrid... and where it is easier to keep a job than at other institutions, such as mine (where tenure-track assistant professors need to be rated as excellent in research or teaching and outstanding in one to earn tenure).

That inability to keep people at FGCU without a tenure system has its parallels in K-12. Not everywhere, but there are persistent shortages of qualified teachers in math, science, and special education (for almost any definition of qualified you want to pick). To keep good people in those spots, you'd need to pay them decently and give them some reason to believe that they're not going to be fired capriciously.

February 14, 2009

Stimulating thoughts

Now that the stimulus package has passed, a few thoughts:

  1. The speed of the conference-committee work was breathtaking. Years from now, apart from what happens with the economy and the rest of his presidency,  political historians will remember the fact that President Obama achieved an unprecedented legislative victory less than 25 days after becoming president. FDR's First Hundred Days, ha! Obama's just set a new standard. Well, not quite: FDR's start was more astounding in terms of the change in federal power. But this week was still remarkable, and my jaw was on the floor when I read of the conference-committee agreement at the end of Wednesday.
  2. This bill will save thousands of teachers' jobs. Thousands of teachers will still lose their jobs, but it would have been much, much worse without this bill. That fact will change the conversation in Washington. 
  3. We still do not know the consequences of the millions in the Secretary of Education's discretionary spending authority, what Mike Petrilli is calling a slush fund, or the larger incentive fund, what Charles Barone hopes is the authority of Arne Duncan to mandate that states move on existing mandates. Let's keep things in perspective: $600 million is a lot of money, and $5 billion is more, but the first is about 0.1% of the discretionary authority handed to the Treasury Secretary in the bailout funds, and the second is also a small amount of money compared with all education spending each year. Big?  Yes. Consequences? Not quite known yet.

Is it the fulcrum Andy Rotherham wants? No. As the Bush education officials found out (and what Petrilli explained on the last Gadfly podcast), regulations still circumscribe what would otherwise appear to be discretionary. And as I've implied above, it's the saving of teachers' jobs that is more likely to change policy conversations. It's better to ask, "what can you do with $600 million/$5 billion?"

But I'm going to ask something different: what are the standards that we should expect for any "innovative" project? Here are some down-to-earth ideas that could easily be the standard:

  • Development of software for formative assessment should prioritize the fast, frequent, flexible, and simple: see my February 6 entry on periodic assessment for why.
  • Local infrastructure standards that minimize the time wasted by teachers and others waiting for software and servers to respond. Right now in one Florida school district, the software/hardware for scheduling students is so horrible that counselors are waiting 30 minutes for the server to process all the tasks for a single student for one semester. The IEP-drafting software for a Florida school district is likewise a good time-waster for special-education teachers, being so modular that almost every operation requires a click and then waiting for the next page. If it wastes teacher time, it should be cut out.
  • Evaluation does not mean a single organization collecting and analyzing data. Evaluation with federal dollars should mean collecting data with some quality and then letting a variety of people have access to it.
  • Development of longitudinal databases need to be accompanied by auditing mechanisms, not just consistency and sense editing. Hire a data-entry clerk for each school, as Florida does, and you still have a massive editing task by school districts. And even after that, researchers occasionally find data quirks such as 26-year-old first-graders (i.e., birthdate entered wrong). And that doesn't address issues such as marking dropouts as transfers.

February 9, 2009

Cutting reading coaches

Yesterday morning's Lakeland Ledger story on the cutting of reading coaches has a good (if brief) discussion of the issues. While the current economic catastrophe is much worse than any other recession since WW2, this dynamic is a result of the way that schools respond to reform pressures by adding pieces to the school: you hire a special educator, you hire a speech/language therapist, you add vocational education, etc. It's what The Shopping Mall High School described as a curricular centrifuge (my term, not the authors'), but there's another consequence: if pieces can be added, they can be subtracted. This has happened with every wave of dropout-prevention programs since the 1960s: something is added with temporary funding, it remains at the margins of a system, and then after the funding dries up, the program disappears. Reading coaches in Florida appeared and expanded with federal Reading First funding. Reading First is on its way out as a program, and with the funding crisis, so are the reading coaches.

Everyone's job prospects are sinking

In discussing new employment data from the Bureau of Labor Statistics when disaggregated by educational attainment, Andy Rotherham argues that "problems are hardly equally distributed and again the returns to education are apparent."

Looking at the table (and at the seasonally-adjusted figures), it's clear that there are differences in January's unemployment rates (12.0% for high school dropouts, 3.8% for college graduates), but there were also differences before the economy fell off the table. How do you tell whether the current differences are changed from prior differences?

My usual tool to use in this situation is the relative odds ratio. Consider one unemployment rate, January's 12.0% for high-school dropouts. The odds that a high school dropout in the labor market was unemployed is the probability of being unemployed divided by the probability of being employed, 12.0%/(1-12%), or about 0.14. The odds that a college graduate was unemployed in January was 0.04. The ratio of the two is 0.29 -- or a college graduate has odds of being unemployed that's a little under one-third of the odds for a high-school dropout. The closer to a relative ratio of 1, the closer the odds are and the less of a difference. A relative odds ratio of 0.29 indicates a HUGE difference.

But the $64,000 question is whether that odds ratio has changed from January 2008 to January 2009. In January 2008, the relative odds ratio was 0.26 for the BA-to-dropout comparison, 0.46 for the some-college-to-dropout comparison, and 0.58 for the high-school-grad-to-dropout comparison. Last month, the relative odds ratios were 0.29, 0.48, and 0.64, respectively. While there is greater movement for the high-school-grad-to-dropout comparison, the story here is that the employment crisis is bad for everyone and slightly compressing the attainment consequences, not expanding them, at least with regard to the official unemployment rate.

There are significant problems with the official unemployment rate, though--it excludes the "discouraged" unemployed and those not in the labor force. Unfortunately, the BLS table on alternative measures (including U-6) is not disaggregated by attainment, so I'll pick on the employment-to-population measure that's in the first table linked above. There, the odds ratios are reversed in meaning (the odds of being employed vs. the odds of being unemployed), and they hardly budge: for the high school grads to dropouts comparison, the relative odds ratio is 2.0 in January 2008 and January 2009; for the some-college to dropouts comparison, the ratio is 3.1 in both months; for the college-grad to dropouts comparison, the ratio is 4.4 in both months. Again, the cross-sectional gaps are huge, but there's no change.

I could go further into the weeds with relative probability ratios, but I'll give you a chance to stay awake and just say that they're going to have the same basic message as relative odds ratios: the job market is getting worse for everyone, and at least at a first glance, being well-educated protects you no more last month than it did a year ago.

February 6, 2009

Klein compares Bloomberg to Putin

No, he didn't, but at the mayoral-control hearing in Albany, according to the indefatigable Elizabeth Green,

Klein defended himself passionately, arguing that mayoral control is a democratic governance structure, not an authoritarian one, as some members painted it.

The logic here is weak: under that view, a plebiscite dictatorship is democratic because every few years the head honcho could be kicked out of office. 

I think there are multiple reasonable approaches to the policy question, such as UFT's "you need two (more) righteous people to save Gotham" proposal of giving the mayor a plurality on the main policymaking body (so the mayor and chancellor would have to convince 2 out of the other 8 members) or something that would give an independent body subpoena authority and the responsibility and right to issue reports on the schools.

But the gist is to inject public accountability beyond the one-person constituency of Joel Klein. I'm a little curious why advocates of mayoral control don't grasp the fundamental irony that you don't create accountability by removing it. There are multiple ways of addressing the messiness of urban politics, but if the appointed chancellor has spent several years ignoring parents, he's getting his natural comeuppance today.

Getting into the weeds: when did $25 billion become the weeds??

Charles Barone has been writing about the stimulus debate with the type of detail that no one else has, at least publicly. That's the importance of having people with legislative experience who really can get deeply into the weeds. What's jaw-dropping is that the stimulus package is so large that the Senate cut of $24.8 billion from the package is in the weeds. Not much in the weeds, but your boots are going to get muck on them to look at the details. 

I think anyone who understands Keynesian macroeconomics should be shocked at what the Senate is doing with the stimulus, both the patently nutty stuff like Barbara Mikulski's suggestion for car-purchasing incentives and the proposed deletion of aid to states so that local agencies don't fire thousands of teachers, public-health workers, and the like. I'll let Brad DeLong and Paul Krugman explain the basic theory here, but spending money on people who pay taxes and spend most of what they receive is a higher stimulus bang for the buck than almost any other conceivable way of stimulating the moribund economy. As long as we are stuck in a world where the Fed's interest rate is effectively zero, monetarist approaches will be insufficient. As long as businesses and people are reluctant to spend money, tax incentives will be far less powerful than direct spending. To borrow from Jeffrey Frankel, there are no libertarians in liquidity traps and deflationary spirals. There also should be none when either is an imminent threat.

Because the House and Senate will have a conference to hash out various differences, it would be tempting to say, "Just get the Senate to approve something, and then work it out in committee." I'll let others in the blogosphere point out the political flaws in this argument, but the gist is that Senate Republicans are far more powerful than House Republicans. With both Franken and Kennedy out of the picture for now, the Democrats have a harder time getting 60 votes to stop debate. So there will be some compromise with the Republicans, and there will have to be some triage in what can be accomplished with the stimulus. The better the bill that comes out of the Senate, the easier will be the negotiating in conference.

Having said that, it's the crisis-oriented big-money issues that are or should be the priority here. If the battle in the conference committee has to be about the $25 billion cut in aid to states, the stakes are too high to focus on conditions attached to the money. That's the unfortunate truth, and I'd bet that Joel Packer and Charles Barone will find agreement here: they'd rather debate how to attach conditions to $25 billion that will save teachers' jobs and help the economy than to have to fight for the $25 billion in the first place.

UTLA and "benchmark" or "periodic" testing

Last week, the United Teachers of Los Angeles called for the cessation of every-few-months testing in the district. The response of the district: such testing is an important tool in improving student achievement, which they know because schools with such testing have had annual-test scores higher than schools without such testing.

The flaw in the district's reasoning is left as an exercise for the reader, because I'm more concerned at the moment about what this debate shows about our attitudes towards assessment. UTLA is wrong to attack frequent testing on principle, though I think they may have a good point about this type of assessment. Such periodic assessment may help schools target assistance to students, or they may serve primarily to mimic the state test and encourage teaching to the test (the predictive success of which principals would know by results on the quarterly assessments). Without knowing more about the details, you can't say which is which, and both phenomena are possible (including in the same school).

What concerns me is the direction in which the machinery of testing is taking formative evaluation. There's a lot of research to suggest that when used to guide instruction, frequent assessment can dramatically change results. There are a number of technical questions about so-called formative assessment (or progress monitoring) that are the domains of researchers in the area: how to create material sufficiently related to key skills or the curriculum, how to create assessments where score movement is both meaningful and sensitive to change, how to gauge appropriate change, how to structure the feedback given to teachers, and so forth. My reading of the literature (which is not complete) is that the most powerful uses of formative assessment require very frequent, very short assessments--on the order of once or twice a week, and about the same length as your typical elementary-school spelling test (i.e., a few minutes at most). 

So what do we see as the evolving, bureaucratic version of formative assessment: long tests taken every few months. That's better than once a year in terms of frequency, but it's still a blunt instrument and absorbs a large chunk of time. The reason for this preference is obvious: a large, unwieldy school system can organize systematic evaluation/feedback around quarterly tests. That's doable. But organizing around something that's taken weekly and would often require data entry (e.g., a one-minute fluency score for first- and second-graders)? That's a different kettle of fish.

That doesn't mean it's impossible. It's easy, if you're a principal who's willing to devote the right resources. Consider reading fluency, for example. (I'm not saying that fluency is more important than comprehension. I just have the experience with this to imagine what I'd do as a principal.) Teach a paraprofessional to have every first- and second-grade student in the school read to them one minute a week on a sample reading passage (there are sets of roughly equivalent passages one can purchase for this purpose). Have them enter the data through a Google Docs form, a SurveyMonkey survey, or some other tool that will send the data to a spreadsheet. Get someone to program the results so that you can show data per child with trend lines and sort by grade, classroom, etc. For a few extra lines of code, you could add locally-weighted regression trends to be really fancy, but that's beside the point.

Here's the point: this is not rocket science, this does not require a gazillion-dollar software package from TestPublisher Inc., and it's very different from the type of quarterly testing that superintendents are buying into in a big way (including that gazillion-dollar software package from TestPublisher Inc.). It's very different from the quarterly testing that UTLA is protesting.

So, Ramon Cortines, here's my challenge: can you document that the quarterly-testing regime is better than the weekly-quiz-plus-trends proposal I've outlined above? The second can fit easily into the routines of any school. The second can start conversations EVERY WEEK at a school. The second is MUCH cheaper. It's also less sexy: no giant software packages manipulable from the front office, no instantly-printable pastel-colored graphs that demonstrate what kids were able to do on a test six weeks ago. You'd definitely give up the flashy for the mundane. But prove to me that the flashy is better than the mundane.

More charter-school organizing

Congratulations to United Teachers Los Angeles for organizing Accelerated School. If the reporting in the story is correct, some of the same issues motivating KIPP teachers in New York to join UFT are also motivating teachers in Accelerated School: teacher turnover, teacher voice in decisionmaking. It's never all about the money. (Hat tip.)

February 5, 2009

What personality is your Performance-Pay Attitude? (and other mixed metaphors)

Since other bloggers I read have used various quizzes to spice up their entries, or maybe do something online while they're waiting for a bus, here is the all-purpose Performance-Pay Personality Quiz. Oh, wait: "personality" isn't quite appropriate here. But to mix metaphors, what personality is YOUR attitude towards performance pay?

  1. Do you think that there is ever a justification for some teachers' being paid more than others?
    • 1 point -- A paycheck is performance pay: either pay people a good wage for doing their job, or fire them for not doing it.
    • 4 points -- Some differential pay is required to encourage teachers to take hard-to-staff jobs (either by subject or school), and that's more important than merit pay.
    • 7 points -- On balance, performance pay would be a good thing, but it's not the most important thing to change in schools.
    • 10 points -- Performance pay or bust: I'll throw everything else out the window to get it!
  2. What's the most important motivation for teachers and administrators?
    • 1 point -- They love children; that's their only motivation.
    • 2 points -- Personal integrity is a more powerful motivator than salary. Teachers need salaries, but if you can show teachers how to feel better about the job they're doing (including showing them how to do a better job), you can move mountains.
    • 3 points -- Money's an important part of the picture. It's not the only thing, and seeing money as the only motivational tool would be foolish public policy, but to ignore it would be wrong.
    • 4 points -- There's nothing like money to get people's attention, and teachers are people.
  3. How important is it for education policy to encourage educators to work together?
    • 1 -- Teachers are not islands: rewarding individuals will kill the type of mentoring and sharing that's essential for professional development. Doubt me? Go ask stock-market traders who entered their career recently whether individual rewards encouraged their elders to mentor them... or spend every second on the floor trying to make a buck.
    • 2 -- Cooperation is crucial. It's not everything, since all teachers have strengths and weaknesses, and we don't want a school full of Stepford Teachers, but I worry that too much emphasis on individual recognition will discourage teachers from talking to each other, and from any chance that teachers will hold each other accountable.
    • 3 -- Teachers' talking in a lounge is like little kids' hugging each other. Often it's wonderful, but you sometimes worry what they're sharing. Individual recognition is pretty important to give credibility to the better and more professional teachers.
    • 4 -- Teacher go it alone anyway: recognizing their achievement as individuals is unlikely to harm the type of substantive collaboration that happens rarely.
  4. What is the right balance between judging teachers based on the professional judgment of peers and using student performance?
    • 1 -- Peer judgment: they're the ones who know what good teaching looks like, and what we care about is whether teachers are teaching well.
    • 2 -- Er... wouldn't peers be interested in what students are learning? Student performance should be part of the mix, as one springboard for evaluation. But peer judgment should be central.
    • 3 -- Student performance should anchor qualitative judgments of teaching. Yes, peers can judge teachers, but student performance should be central.
    • 4 -- Skip the peers. What matters is whether students are learning.
  5. How ready is the technology of testing to use in judging individual teacher and school performance?
    • 1 -- When the solid historical record of more than a century shows that people have abused tests in every decade, we should assume that tests will be misused, and it's the burden of high-stakes testing advocates to show otherwise.
    • 2 -- Tests are useful, but we're far from being sure that tests tell us what most politicians think they tell us.
    • 3 -- They're imperfect, but we need to start using test scores to judge effectiveness now because we can't wait for tests to be perfect to look at performance.
    • 4 -- They're just fine, and they have been for years.
  6. What role should collective bargaining play in education reform?
    • 1 -- Collective bargaining is crucial to protecting due process and teacher rights, and if possible to block stupid reforms.
    • 2 -- Collective bargaining is crucial to protecting due process and teacher rights, and unions can play an important part of reform.
    • 3 -- Collective bargaining is primarily an obstacle to important reform. Where unions will accept reforms, great. Where they won't, federal and state governments have powerful incentives to change the balance of power at the local level.
    • 4 -- Federal and state governments should do their best to break unions, because they do nothing good. Break them, circumvent them, discredit them with their bargaining units.
  7. What should be the ceiling in terms of paying for performance (both the total amount of money and how many teachers should be eligible)?
    • 1 -- Arguments in favor of performance pay are a cover for not wanting to pay teachers more. Those who work with children are generally underpaid, and while performance pay looks like it's in "the children's interest," in reality it's another way of being cheap.
    • 2 -- Part of my skepticism about performance pay is the assumption that only 10-25% of teachers should receive it. To these brilliant people, I ask: "Okay, suppose there's performance pay and every student meets whatever is your definition of proficiency by 2014. Does that mean you'd be willing to double teacher pay for that result, or is this an education-reform shell game?"
    • 3 -- Part of my acceptance of performance pay is looking at the numbers: there are lots of students, and it's almost impossible to staff every classroom with a brilliant and greatly-skilled teacher. So let's pay the great ones the best. "In a perfect world we'd double teacher pay" is another way of saying "never."
    • 4 -- Competition is the best way to motivate individuals, and you're going to get little competition if everyone can earn a bonus. Limit performance pay to the top slice of teachers.

Psychometrics-free labels to share with frenemies and colleagues:

7-11: You are Alfie Kohn. You'd really like the testing industry to suffer an ignominious death, and anyone who thinks that using tests will improve schooling is smoking something fairly powerful.

11-16: You are Reg Weaver. You are publicly skeptical of merit pay, you think most designed systems are going to be disasters, but you're also going to hold your nose and support teachers who decide it's in their best interests.

17-23: You are Randi Weingarten. You know that the American public is used to people making more money if they do a better job, but you're skeptical of most performance-pay plans in operation today. You think collective bargaining is the best way to moderate the more idiotic ideas surrounding teacher pay and to protect the legitimate interests of teachers and communities.

24-28: You are Thomas Toch. You're well aware of the flaws of testing and accountability systems, but you think moving in the direction of performance pay is essential, and you will trust that the system can be improved incrementally once it's started in the right direction.

29-34: You are Michelle Rhee. The day that teachers have a starkly uneven pay scale, the day that school districts fire a fifth of their teachers, and the day that unions are decertified around the country will be the day you will not only take up that Newsweek broom again but dance with it a la Fred Astaire. 

(Don't like the questions? Fine: make up your own completely unscientific spoof of internet quizzes!)

January 29, 2009

The colonic theory of school reform

Looking at news this morning of university-wide furloughs at Arizona State, where my brother is a professor of geography, has put me in a sour mood. Not only does this affect my family and a bunch of friends elsewhere on the faculty, but it puts the lie to the "this is a great time to winnow out bad programs" argument. When there are draconian budget cuts, and they come quickly, there is no way to avoid damaging good programs as well as bad ones. Even if a university avoids across-the-board cuts for a time, when the cuts become severe enough, everyone feels it. The pain is not felt equally, but it is widespread, and there is going to be damage to good programs.

My friend and predecessor as faculty union chapter president, Roy Weatherford, explained his disdain for the political argument that events needed to get truly bad for the voters to kick out your opponents: "That's the enema theory of politics," he's said on various occasions. There are some advocates of this approach in any political organization, I'm afraid. If you subscribe to it, you hope for the worst instead of arguing for the best.

The parallel argument is now being made with respect to budget cuts: they're good! (I've also heard that argument from one state legislative aide in Florida. That was last spring, and I haven't heard that argument since, at least in Florida.) That assumes that at some point in an organization, the pain is so bad that decisions get better. I'd love to see any research on this, but I suspect this is a seat-of-the-pants argument (see Andy Rotherham and Kevin Carey for more on this point).

For those who still believe that draconian budget cuts somehow make things more efficient, pretend for a moment that there were such as thing as perfectly rational management. Even if that were the case, there is no management system that simultaneously has a perfect understanding of the value of all organization components, is effective in organizational politics, and has not yet optimized stuff. (For the market fundamentalists out there, this is the parallel of saying that stock prices automatically reflect the knowledge that "the market" already has. In this rationalistic world, if there already were information to justify cutting a program, effective managers would have found a way to do that, or they're not effective.) That means that when stuff hits the fan, even if you thought there could be an objective way to make budget-cutting decisions, decisions will be made that cannot be justified based on what's known at the time. That's because there is no such thing as objective ways to make budget cuts, there isn't good information, or the decisions will be made for the wrong reasons.

Decisions should be made for the right reasons at all times, and any claim that "this is the perfect time to do X" strikes me as opportunistic, along the lines of arguments for tax cuts:

  • We have a budget surplus, and it's the people's money, so we should give the money back.
  • We have a budget deficit, and the best way to solve that is through growth, so let's cut taxes as a stimulus.
  • Times are good, so let's cut taxes.
  • Times are bad, so let's not put a greater burden on families.

This reminds me of the classic British argument for tea-time:

  • Working hard in the afternoon? Have some tea.
  • Spending time with friends? Have some tea.
  • Not feeling well? Have some tea.
  • Want to celebrate great news? Have some tea.
  • Nuclear war? Have some tea.

While it's a delight to see intellectual flexibility among my fellow Americans, at some point it is hard to argue that program and policy decisions should be rational when your argument for making those decisions is fundamentally irrational; don't simultaneously indulge in Machiavellian fantasies and then claim that it's all in the service of decency. Maybe I'm the education blogosphere's hobgoblin this morning, but there it is. It's a small consistency I'm asking for, that's all.

January 28, 2009

Jay Greene lands on the side of John Dewey

Jay Greene points out that expertise should not trump public input on key policy issues. In this regard, he's closer to John Dewey's trust of the general community of citizens than to Walter Lippmann, who was sure that we needed a technocratic administrata ruling key matters of public policy. 

I'm on Dewey's and Greene's side, here. In 2006, I winced in the middle of Reg Weaver's main address at the National Education Association meeting in Orlando, for he was trying to denigrate expertise on one hand (with regard to educational psychologists) and then use it on the other (to raise the status of teachers). You can't do both and get away with it easily.

So maybe my best answer to DeanDad and Stanley Fish is that a liberal-arts education gives you both the general skills and also enough knowledge in multiple disciplines so that you're not easily fooled. It's not an inoculation (as all those who fell for Bernie Madoff can attest), but both republics and democracies depend on a population with sufficient education to enter into a debate about facts rather than just accept the facts that self-annointed experts have offered. Sometimes this openness to debate is disastrous, but I much prefer it to a technocracy.

Conspiracy Debunking 101

So rumors are flying that Linda Darling-Hammond may become Deputy Secretary of Education and World Destruction, and that the Senate is Stripping Glory from the Stimulus. Before anyone gets their hair too much out of place on speculation of appointments, legislation, and little green beings that are just about to swoop down on 400 Maryland Ave. and "Oprah"--well, apart from Rod Blagojevich, whose mop isn't remotely green--let me repeat a few words of caution and maybe even wisdom from a long-term perspective:

  • Facial plausibility for your speculation does not make it true, even if it could explain a number of observations. Check that speculation by looking for alternative hypotheses and discomfirming evidence.
  • The enemy of your enemy is not necessarily your friend, and two of your adversaries are not BFF just because of their real or imagined adversarial relationship to you. (Google "exaggerated self-reference.") They are human beings, and their relationship is probably as complex as you think it simple. Some day I need to find time to explain why liberals in teachers unions aren't always going to agree with liberals in colleges of education, but just trust me on this one for now.
  • Rome was not destroyed in a day. Apparently the conference committee took a few hundred years.

From all this back-and-forth about oh no! Linda Darling-Hammond--oh no! Andrew Rotherham--oh, no! Linda Darling-Hammond--oh, no! Darth Vader and the E Street Band--I feel like I'm back in middle school and am just waiting for someone to walk by and sneer, "take a chill pill" (a clause I last heard spoken aloud when I was in high school). The common-sense analysis I laid out the day after the election is generally panning out (but with a few exceptions that I'm learning from): the hard issues are still there, and I don't think anyone is going to be rolled on them in a major way in the stimulus.

Furthermore, the cabinet and sub-cabinet appointments in the Lesser Agencies have a pretty good incentive not to be high-handed: those who are appointed can be dumped, and I suspect that is most likely in this administration for violating the No Drama Obama Unless You're Rahm Rule. You know--Work Hard. Be Nice. I know I've heard that somewhere...

January 21, 2009

One step in the right direction... but let's end single "national evaluation" studies

As Stephen Sawchuk notes, the stimulus bill package requires randomly-controlled studies of federally-funded performance pay (from the Teacher Incentive Fund, which is receiving $$BIG in the stimulus). From pp. 166-167 of the draft:

Provided further, That a portion of these funds shall also be used for a rigorous national evaluation by the Institute of Education Sciences, utilizing randomized controlled methodology to the extent feasible, that assesses the impact of performance-based teacher and principal compensation systems supported by the funds provided in this Act on teacher and principal recruitment and retention in high-need schools and subjects...

First, kudos to the bill authors (legislative staff) who inserted this language. I'm almost rolling my eyes at the randomized controlled trial language because I thought we'd been through the methodology debate sufficiently to understand that RCT is not a panacea. It is a good option for comparing discrete options (e.g., two different "treatments" that are distinct and clearly defined), but it is extraordinarily hard to arrange in education, there are other legitimate options even for that relatively narrow effectiveness question (e.g., regression discontinuity and propensity score designs), and there are other important analyses to consider (depending on the question and the discipline, economists would ask about cost effectiveness, and the educational equivalent of epidemiologists would ask about the "number to treat" or broad population-treatment questions).

But I'm not rolling my eyes because it's the first full day of the new administration, I'm 43, and my eyes might stick that way if I keep doing it. Hmmn. The serious reason why I'm not going to quibble too much with that language is because if done correctly, discrete studies will still tell us something, there's the "to the extent feasible" clause, and on principle, it is a good step to require planning for evaluation at the front end.

On the other hand, I think it's a mistake to require "a rigorous national evaluation... that assesses the impact" as in a single analysis. That language has a grammatical problem: it's using the singular when the plural is more appropriate. There should be a single rigorously-designed and -collected set of data, but it is wrong either to put the analysis in a single group's hands or to frame the question in a singular fashion, as if the answer to any effectiveness of a national program is "yes, it's effective" or "no, it's not effective."

That's the headline for any single, putatively authoritative national evaluation, and if my favorite policy in the whole wide world were performance pay, I would work like the dickens to make sure the questions were framed differently, because it might well turn out (and we should expect the world to work in this way) that the first and second generation of performance-pay plans will largely do squat. "Do squat" is technical language for an average effect size around d=0. Again--if my favorite policy in the whole wide world were performance pay, I'd want to make sure that the questions revolved around differences among programs and pay schemes (including non-performance-pay structures), not just the difference between systems with and without performance pay. And I'd want to make sure that Hawthorne effects were screened out. And all the other things that could subject any such study to criticism within half a day or so. And I'd want the data made available to researchers with different perspectives, so no single person or group could spike the results.

It just so happens that the same diversity of questions and distribution of data would be good from the research community's standpoint, too. It's a shame that the habit in large federal programs is different. If you doubt the wisdom of my advice, seek counsel from those upset about the national evaluation of Reading First.

A wish for honest arguments

Since Jennifer Jennings and Aaron Pallas are providing their wish list for the new administration this week, I'll chime in with a hope that Paul Krugman's observation for the economic policy team is also true in education. Here is what Krugman wrote January 10 when discussing the Romer-Bernstein graph on the anticipated effects of the stimulus:

Kudos, by the way, to the administration-in-waiting for providing this--it will be a joy to argue policy with an administration that provides comprehensible, honest reports, not case studies in how to lie with statistics.

So, too, it would be wonderful to have a U.S. Department of Education that is more interested in asking hard questions than in fobbing off pat answers and spinning sound bites. This is not a problem that lies solely with the former Bush DOE. It's a problem in school districts, in states, and in past administrations.

When opponents claimed that President Bush lied to the country to start the Iraq War, as an historian I thought to myself, "That assumes that we know his state of mind. Is it a lie if he deluded himself?" At the end of eight years, the former president convinced me that he had deluded himself on many topics. My tentative conclusion about education is that much craziness that passes for education policy comes from the delusions of policymakers that they know "what works" in schools because it fits with their internal thumbnail sociology.

We can no longer afford such delusions and the policy consequences of confirmation bias. Here's the test: will political appointees to the Department of Education state openly where they are not sure, where the research is inconsistent or inconclusive? Will advocates inside and outside the Beltway? That's the difference between focusing debate and shutting off debate, the difference between knowing your values and knowing the limits of research, and the difference between being confident in your judgment and thinking you have a direct line to the truth.

January 19, 2009

Redesigning remedial education

The Bailey, Jeong, and Cho study of remedial education reported on by Inside Higher Ed today is not surprising, but it is still depressing: for the community-college students who most need direct assistance in skills, they are also the least likely to finish a sequence of remedial (aka developmental) courses and also likely not to start on such a sequence at all. I have a strong suspicion that these are not the students in their mid-20s who passed high school algebra courses with a B or higher and forgot the content over 6-10 years (and for whom high-school-age "college readiness" is an irrelevant concept). These are students who are barred from the regular curriculum by testing prerequisites and, at least according to this paper, are the least likely to finish a developmental sequence and start earning college credits.

In 1960, Burton Clark wrote an article that extended the 1952 Erving Goffman "cooling the mark out" argument (in the Goffman Reader) to community colleges; in 1984, Mark Ginsburg and Joanne Giles echoed that, and that's what the Bailey et al. paper appears to suggest: when remedial courses and a sequence of several courses is a gatekeeping mechanism that colleges use before a student can take a for-credit class, it discourages students not only from completing the sequence but often from beginning the sequence in the first place. (Also see John L. Johnson's article a few decades ago in the Journal of Special Education for a parallel argument with a sharp twist.)

Community colleges are in a bind here: faculty and administrators do not want to use the limited resources available to community colleges by giving seats to students who are unlikely to pass a class. But remedial classes are not costless, and I assume most faculty know that testing prerequisites also screen out a significant number of students whom colleges are supposed to be serving.

Here is where Kevin Carey's argument from the November Washington Monthly applies, if it applies generally. I shook my head when I read Carey's article a few months ago, because he was assuming or implying that most spending in public four-year institutions is on instruction (something the Delta Project should be disabusing us from). While there's another entry I need to write about how to think about spending on instruction, research, and football, let me get to the meat of this. Carey argued that there could be much better instruction squeezed from existing resources. This argument is based on the work of the National Center for Academic Transformation (NCAT) and the evangelism of Carol Twigg on course redesigns. Twigg argues (and Carey picks up on this) that one can use technology to engage students more and use faculty, T.A., and staff time more efficiently.

I've talked with some thoughtful people in the college teaching-effectiveness world who are skeptical of Twigg's more extensive claims, but I'm willing to skip over those debates and say that below some level of resources, it is not possible to provide extensive one-on-one coaching, let alone individualized instruction on key topics in a course, and that Twigg's approach is most likely to be a reasonable strategy when resources are low and the material is reasonably standardized.

Remedial/developmental math courses seem to qualify on both fronts: in general it is in community colleges where resources are lowest and where there are a common set of expectations students must meet in reading and math.

But this is in the abstract -- obviously, many community colleges would need a short-term infusion of resources to transform developmental courses, and this should be tested rather than assumed to be true. Unfortunately, of the NCAT's current membership, there are only 8 community colleges (the majority in Texas), and no community college appear to have been involved in the FIPSE-funded projects in the past few years.

But this is a look from afar--those who teach or work in community colleges, please have at this idea!

Addendum: in comments, skoolboy (aka Aaron Pallas) properly takes me to task for forgetting Burton Clark. Mea culpa!

January 13, 2009

Disappointed

E-mails have been making the rounds among some educational advocates and researchers, stating rumors that Andrew Rotherham (Education Sector), Wendy Kopp (Teach for America), and Russlyn Ali (Education Trust) were being considered for sub-cabinet posts under Education Secretary-designate Arne Duncan, and asking that recipients contact the transition personnel office to object to such appointments. While I haven't seen any e-mail traffic with rumors about the posts for Rotherham or Kopp, Ali supposedly is in line for the Office of Civil Rights inside the USDOE.

First things first: I took a stand against the vitriol directed at Linda Darling-Hammond last month, explaining that on principle I am opposed to ad hominem attacks. My philosophy hasn't changed in the last two months. Has yours?

Second: not only are ad hominem attacks wrong on principle, they are also foolish politics unless you can point to a smoking gun about fitness to serve. Unless policy differences rise to the level of potential criminality (and none of those individuals come close to John Yoo on that scale), policy differences are not going to be a convincing reason to a transition personnel office. Duncan probably knows the policy positions of Rotherham, Kopp, and Ali. Does he also know that by making certain appointments he's going to upset associate and full professors at Directional State University or even Real and Not Facade Brick Private University? While I wish I had greater sway over policy, I suspect pleasing or displeasing me is not high on his concern list at the moment.

Third: the slamming of all three individuals makes little sense from a standpoint of using political capital shrewdly. Assume for a moment that Russlyn Ali is headed to the Office of Civil Rights (OCR). Is there any reason to believe that the responsibilities of an appointee at OCR would cover the high-stakes accountability issues that presumably people might disagree with Ali on? So if this rumor happens to pan out, Ali's appointment to OCR would remove her from most issues that put her name in that e-mail. 

When I began writing this entry, I wasn't not sure whether to be angry or sad at this latest sandbox fight. It's turned to disappointment. Those who look at the Obama transition years from now will admire Obama's ability to appoint people with gravitas in the headline Cabinet posts and then wonder why the Department of Education appointments were the focus of so much gossip and backstabbing.

Oversight boondoggle

Last week the Wall Street Journal lambasted Florida Governor Charlie Crist for failing to appeal a ruling that struck down the Florida Schools of Excellence Commission as an unconstitutional infringement on the powers of county school boards in Florida. The legislature wanted to set up the FSEC as a second authorizer of charter schools in case county boards were unfair and refused to let enough charter schools open. This bewildered me because Florida has no statutory cap and there are a few hundred charter schools in the state.

This afternoon, I remembered a blog entry written by St. Pete Times reporters in December: the FSEC has been spending the people's money like it was water, racking up almost half a million dollars in expenses over two fiscal years without authorizing a single charter school that has yet opened its doors. 

Isn't the Wall Street Journal supposed to have a conservative fiscal philosophy?

Brooklyn KIPP teachers unionizing

As reported in EdWize. What caught my attention was the issue of turnover:

KIPP AMP teachers believe that the high staff turnover at the school has harmed their efforts to build a positive and consistent school culture for their students. "There is a need to make the teacher position more sustainable," says [Luisa] Bonifacio, "so that teachers don't burn out, but are able to make a long-term commitment to the students and the school."

As any labor historian knows, unionization is usually driven by material and also by other considerations that motivate people to sign pledge cards: wanting to be treated decently on the job, having conditions likely to foster success, etc. Having co-chaired a card campaign, I know a touch of what a card campaign involves. If there were an Employee Free Choice Act at the federal level and parallel provisions for public employees in New York state, recognition would be automatic with a supermajority (which apparently UFT has collected from this school's teachers). Then the two sides could sit down immediately and negotiate a contract that meets the needs of teachers and students.

That's possible under current law; it just requires voluntary recognition of the UFT as the collective-bargaining agent for the schools' teachers.

Where is the bureaucracy reenactor crowd?

In the past few months, I have been struggling with how to teach a difficult topic: bureaucracy. It's not hard to enter the topic with a class; everyone experiences bureaucracy in ways that they can talk about at one level. Generally, I find that students absorb notions of street-level bureaucrats, scripts about "real school," and loosely-coupled systems. And one of the most popular books I assign is about bureaucracy: Tyack and Cuban's Tinkering toward Utopia. Especially for current school administrators, bureaucracy can be a very attractive topic.

But at another level, a bureaucracy is hard to learn. Though we experience the status games that Weber discusses, and though most adults spend months and years learning the tacit knowledge that Polanyi has described, I know relatively few friends and colleagues who can reliably describe the weird ways that bureaucracies work.

It's not that people don't theorize, but that their theories are often two-dimensional: bureaucracies always behave a certain way, at least in many of the explanations I hear. But that's not a legitimate generalization. Large organizations have repertoires of behavior, and the choices of individuals matter. The truth is somewhere between guessing the psychology of individual administrators and making cookie-cutter pictures of school bureaucracies.

There are two common errors I have observed in the lay perspective on bureaucracy, even from people who work within them. First is an inattention to the interplay of explicit and tacit knowledge, an inattention to the relationship between formal rules and the inevitable discretion in applying them. At universities, this is often played out in arguments about what an accrediting body will or will not call a university on the carpet about. Some things are no-brainers: if news reports show that an institution is the victim of massive financial fraud and mismanagement, an accrediting body will almost inevitably place the institution on probation. But the rules are often more flexible than what a reader may assume. So while my regional accrediting body requires that college teachers have a masters degree with 18 hours in the instructional area, institutions (usually department heads) can certify an individual as qualified without meeting that requirement. Too many such exceptions will raise red flags, but not the occasional one.

At other times, people confuse the discretionary authority of administrators with what is politically or financially possible. In many universities, for example, there is a political balancing act between a provost's office and departments. While in theory many a provost can overrule every department recommendation on tenure and promotion, in few cases will university administrators ignore recommendations that come from both the tenured faculty and a department chair. If the recommendation is to deny tenure, few provosts want to discourage what they perceive as higher standards. And if a provost consistently denies tenure to faculty that are recommended for approval at the department level, there will also be a political price to pay. 

A related error is inattention to institutional routines. I recently read the novel manuscript of a friend, and while I loved the plot, I winced whenever the author confused jails with prisons, swapped police and sheriffs' deputies, ignored the existence of continuances, and so forth. I do not read many mysteries these days, and when I have, I have usually enjoyed the Agatha Christie more than police procedurals. But there is something about the details of institutional behavior that matters to me.

I suppose I am the bureaucratic equivalent of a Civil War reenactor: I have an acquired instinct for institutional behavior and can spot inaccuracies faster than you can say thin slice. I have no idea where I acquired it, and I am not sure how to teach it or if one can teach it at all. But that knowledge should be teachable, because many of the problems that frustrate parents on a day-to-day basis is bureaucratic behavior. "They're just unfair" is an understandable reaction to events, but neither despair nor screaming at principals (or threatening lawsuits) will get your child the best opportunities, or at least not without considerable cost.

January 12, 2009

Deantidisestablishmentarianism in education policy rhetoric

Joel Klein and Al Sharpton wrote an open letter to Barack Obama and Arne Duncan that appeared this morning in the Wall Street Journal. And I have just a few questions about this:

  • How can the sitting chancellor and a long-time civil-rights activist claim to be railing against "the entrenched education establishment" when you could reasonably conclude that they are The Establishment?
  • Why do they think that placing a column in the WSJ establishes their anti-establishment street cred? That newspaper isn't exactly an underground pamphlet.
  • Isn't Klein the type of guy who already has Arne Duncan's cell number? They're fellow urban superintendents, they've talked at meetings, and you assume he could call Duncan up at any time, and probably get Obama's number as well. So why do they need this open letter--do they feel this deep psychological need to pose as Village Voice rebels with a cause?

Klein and Sharpton are setting up a straw-man opponent. In my masters class in the fall, one of my students argued that accountability is well-entrenched as part of the public-school policy script. Whether you want to use Tyack and Cuban's "grammar of schooling" or Mary Metz's "real school" language, I think there's a case to be made that anyone who claims that accountability is "new" is in denial and as punishment should have to watch three or four consecutive playings of an inane 1980s adolescent-rebellion film.

So someone who is less establishment than Joel Klein would be... anyone? Anyone?

Second thought: For a few years, I've had the suspicion that the public "letter to the next president" was a bit precious (in the pejorative sense). The collections of letters to the president published after the end of an administration are usually drawn from the sample of correspondence from ordinary Americans that the White House staff select for a president to read as a reality check. Even if Klein gets some credit in my book for having a salary far less than what either New York financiers or university presidents are commonly receiving these days, in no way could one call Joel Klein or Al Sharpton "ordinary Americans."

So if Joel Klein gets to write a "letter to the next president," though we all know he could call Obama up with ideas about either antitrust policy (his Clinton-era gig) or education policy (his current gig), then the gloves are off. I'm writing a letter, too! And you know from my loving hardass manifesto that I intend to bring some style to it. So here's the rule for 2009, for all of you: Staid pretentious public letters to the new president are out. Your job is to write the most outlandish letters that tell the truth. Come on: it's going to be the Obama era. You can say it.

One more update: Apparently Margaret Spellings doesn't have Arne Duncan's cell number, either! Or at least she's pretending not to. Isn't it so nice of major papers to devote part of their ever-shrinking news hole to long classified ads from major policy honchos who can't navigate their cell-phone menus? Though I think the following would have been free on Craigslist: "Arne: call me. Margaret." What? The Post may have been joking? Oh, yeah, and that's a good use of newsprint...

January 11, 2009

Columnists exist to make the rest of us look smart

The intellectual woes of major-paper columnists continue. Ken Bernstein and Kevin Carey both think Thomas Friedman is wrong in arguing for tax cuts for teachers. One calls the idea very bad, and the other thinks it's stupid. As far as I'm concerned, I'm just surprised Friedman didn't ask for a pony. In the meantime, Berkeley economist Brad DeLong is holding an Epically Bad Tom Friedman Column Contest

Update: Matthew Yglesias piles on (usefully).

January 7, 2009

Off the deep end on Griggs v. Duke Power

Before I get to the main topic this morning, my thanks to everyone who has participated in the reader survey, which will stay available through the weekend. It takes just a few minutes (or more if you want to give me lots of ideas for topics). Tell me what to do by taking the survey!

On Sunday, George Will decided to use a think-tank paper last year by Bryan O'Keefe and Richard Vedder to argue that policies have unintended consequences. Thanks, George: we never knew that without your help. But because Will accepts O'Keefe and Vedder's argument at face value, I have to correct the record.

O'Keefe and Vedder make an argument that Vedder has made repeatedly over the years: the Griggs v. Duke Power (1971) decision discouraged employers from using intelligence tests and therefore falsely magnified the credential value of college degrees as the easier way for businesses to make distinctions among applicants. In the case, 13 African American employees of Duke Power complained that after the Civil Rights Act, Duke Power changed its promotion criteria to eliminate references to race and to add a high-school credential requirement as well as specific scores on two tests. Because the combination of these disproportionately affected African American workers, the plaintiffs argued, Duke Power was using race-neutral means to maintain discriminatory outcomes. The Supreme Court accepted the reasoning of the plaintiffs, and Griggs was a landmark in disparate-impact litigation. O'Keefe and Vedder argue that because the Court said that credentials and tests had to be tied to business necessity, businesses began to turn from general IQ tests to college diplomas as the main screening device used in personnel decisions. 

There are several reasons why this argument holds little water, and let's start with the case itself. O'Keefe and Vedder are correct only if the Court discouraged IQ tests and let educational credentials alone. Without that distinction, there's no argument that businesses used college diplomas as a substitute for IQ tests. So let's peek into the crucial passage:

On the record before us, neither the high school completion requirement nor the general intelligence test is shown to bear a demonstrable relationship to successful performance of the jobs for which it was used. Both were adopted, as the Court of Appeals noted, without meaningful study of their relationship to job performance ability.

Maybe I'm misreading the case, but it looks as if the Supreme Court said both credentials and IQ tests were indefensible unless tied to job performance. I don't understand why Vedder has made this argument over the years without addressing the obvious problem with his line of reasoning.

But even if the Supreme Court had written differently, or even if HR professionals developed the same misreading that Vedder did (in which case the fault lies with them, not with the Court), it's a stretch to tie credentialism to a specific case. To believe that, we would have to believe that in the entire history of industrialization no one thought about using educational credentials as a screening tool until the 1970s and then--pow!--employers discovered that some applicants and employees had college degrees and others didn't.

In the paper, O'Keefe and Vedder do not even attempt to collect or display evidence that any industry started using college degrees after 1971 when they had used IQ tests before. And the reference they use to imply a broad historical sweep?--

In fact, according to Staffing Industry Report, a human resources newsletter, 65 percent of companies reported using some type of pre-employment screen, up from 34 percent in prior years. (p. 12)

--is from a 2008 New York Times story titled Dilbert the Inquisitor. I have no clue what "up from... in prior years" means, but it's not pre-1971. I know what business history is. I've read business history. Bryan and Richard, you are not business historians.

Keep in mind the broader uses of this argument that Vedder's shown: because college expanded in significant measure due to businesses' inability to use IQ tests, we have credential inflation and a greater use of college that is warranted strictly by human-capital needs. Ergo, we should invest a lot less in college.

Well, Richard, we already have: starting almost with the time of Griggs, states have dramatically shrunk their subsidies of undergraduate education at public colleges and universities. Students and their families have continued to see college as a good thing, even though they are having to acquire more debt as a private investment instead of a substantially public investment. Part of that is credentialism, but if so, I don't think you can blame Griggs. There are arguments to make about the problems of student debt and college waste, but O'Keefe and Vedder's argument is bad history.

January 2, 2009

Most education advocacy groups unable to borrow "Mission Accomplished" banner

Washington, DC (APOCRYPHAL PRESS)--The fundraising group Democrats for Education Reform has found itself unable to secure the Mission Accomplished banner that President George W. Bush used to declare victory in the Iraq War.

"Apparently, it's been reserved by the National Education Association," said a prominent member of the group when contacted for this story. "They went to the National Archives five minutes before we did. Damn them!"

According to sources at the National Archives, at least fifteen advocacy organizations plus another big-city superintendent tried to reserve the infamous banner for press conferences announcing their pleasure at the designation of Arne Duncan as Barack Obama's Secretary of Education. One confidential memo secured by the Apocryphal Press shows the speech that was to be used by an unidentified advocacy group:

We have successfully invaded the Obama Administration. With Arne Duncan installed in the Secretary position, the forces of educational terrorism are in retreat.

As soon as we can, we will transfer sovereignty for its policies back to the Obama White House.

No group would take responsibility for the statement, though one representative speaking on condition of anonymity said every advocacy group wanted to praise Obama for his choice "because doing otherwise would make anyone look like a jackass or a sore loser."

January 1, 2009

Creative Commons and the First Sale Doctrine

To all my readers, Happy New Year!

Over at Open Content, David Wiley has a fascinating legal puzzle for us to ponder: does the First Sale doctrine undermine the Creative Commons licensing system? I suspect the answer is no from a practical basis because the First Sale doctrine only applies to individual copies of works. But I hope the folks at the Volokh Conspiracy or Crooked Timber take it up.

December 31, 2008

Zombie jargon for the end of 2008

As this incredibly horrid, wonderful, and plain odd year limps to a close, we should say goodbye to terms that are long past their usefulness. (Snark warning: It's the end of a year, and this is one of the rare times that I will actively make fun of bad ideas.) In my view, the following terms are zombie jargon, terms that by all accounts should be dead but somehow are still walking around. Are your brains being eaten by any of them?

  • Speaking of brains, let's start out with brain-based learning. All learning is brain-based, but neuropsychology is not sufficiently advanced for anyone to say with certainty how a specific instructional method is tied to what happens in specific parts of our brains. This term is too often connected with pseudoscientic cant used to sell various products and programs.
  • Let's move from brains to classrooms: sage-on-the-stage vs. guide-on-the-side. If you can figure out a classroom where there is either no structure at all or children don't have their own ideas, go visit a real one. Until then, please don't bother me with false dichotomies. All good teachers have some structure in the classroom. All good teachers work with the fact that students are human beings with their own motives, moods, and so forth.
  • But the fact that students are different is sometimes reified into categories with little research support, such as learning styles. My educational psychology colleagues tell me that there is insufficient evidence to support claims that a particular student will inherently learn better in some presumed "mode," let alone support for the constructs of various proposed modes. And it's a good thing, too: if some of us truly were verbal or kinesthetic learners rather than being able to absorb visual information, the roads would be much more dangerous than they already are.
  • Along the lines of learning styles is multiple intelligence, which is Howard Gardner's assay of our internal baloney meter. If you've paid money to read one of his books on the topic, you failed the test. In his defense, I know that he developed the term in trying to address the king of all zombie jargon in education, intelligence. But I'm not sure if it makes problems any better if you multiply them.
  • But let's move from psychology to policy pablum. If you hear policymakers talk about world class standards, make sure to run as fast as you can before they open up your skull. I don't know if my children and their peers need to meet standards that would work in Quito, Buenos Aires, Accra, or Beijing, but I'd settle for their meeting MIT standards, Oberlin standards, and UC Irvine standards, and for their being able to make friends and work with peers from Quito, Buenos Aires, Accra, or Beijing.
  • And now that we are in the 21st century, can we stop talking about terms such as 21st century skills as if being able to read and knowing a bit of science, math, and history are somehow obsolete?
  • While all of the terms listed above should have been dead a few years ago, I want to add another one that's freshly dead (or should be): data-driven decision-making. This is not an argument against using data to make decisions but against using the term "data-driven decision-making" to avoid talking about hard decisions: what should we be teaching children, who has priority on resources, is a teacher or principle thinking that their job is working hard instead of teaching, etc. It is a term used by technocrat wannabes, the ones who would have talked about zero-based budgeting in the 1960s or time-and-motion studies in the 1920s.

If it's still 2008 where you are, please take care tonight, don't drink if you're going to drive, and have a great New Year!  Oh, yes, and don't taken in by zombies talking in management-speak:

Update: Lake Superior State University has published its new banished-words list for those who want to avoid abusing the English language when talking about other subjects.

December 29, 2008

Matt Miller's choice of a model politician on education policy is weird

Matt Miller is back with a fundamentally outlandish idea:

At a moment when we've basically nationalized the banking, mortgage and insurance industries, a little nationalization of school operating costs is in tune with the times.

There's nothing inherently wrong with a little outlandishness, and he uses funding inequalities as the basic rationale to push a combination of anticyclical stimulus, purse-strings incentives, and maybe the destruction of school boards. But like Mike Petrilli, I am a little skeptical, though for a different reason. Apart from the merits of revenue-sharing, there's something odd in his appeal to the authority of Richard Nixon:

In the end, of course, Nixon found he had bigger problems to deal with. But he left a blueprint for Mr. Obama to follow.

I don't know if Miller meant to be funny and refer to Watergate, but it's hard to figure out why Miller reached out to Nixon for an example, when Nixon's primary de facto education initiative was the relationship between his Southern strategy and civil-rights enforcement, and Nixon used local-control rhetoric frequently in his arguments against busing. There's a reason why Nixon's revenue-sharing plan was first floated and then killed: a federal funding-equalization case was rising through the courts, and any sane domestic policy adviser would have figured out tentative plans for responding to a potential blockbuster decision requiring equalized funding.

I suspect that archival documents would identify San Antonio v. Rodriguez as the primary motivation behind the plan Miller thinks was a technocratic bit of genius. But when the Supreme Court ruled in 1973 that the federal constitution did not forbid funding inequalities, there was no political reason to push the plan any further, and Nixon would have had no inclination to do so. If Miller wanted to pick a politician who was able to push funding reform without court orders, he'd be much better off writing about former Florida Governor Rubin Askew, who convinced the state's legislature to pass an equalization law in 1973 after it became clear that no court would require the state to do so.

But back to Nixon and the big picture on federal education policy. Yes, the 1972 Education Amendments had Title IX and the 1973 Rehabilitation Act had Section 504, but those clauses were inserted by Congress, and Title IX regulations did not appear until several years later. The most prominent institutional contributions to federal education policy that began inside the Nixon Administration were the creation of the National Institute of Education, which I have seen and heard generally attributed to Daniel Patrick Moynihan's inside advocacy, and NAEP, at least partially credited to Nixon's first education commissioner, James Allen, who resigned early in the Nixon administration (and died in a plane crash). Petrilli has it nailed: the idea of a huge bailout/stimulus/revenue-sharing plan is much closer to Lyndon Johnson than it is to Richard Nixon.

I've been disoriented in the past by Miller's rhetorical gambits, and so my reaction this morning fits with my response to his earlier book The Two-Percent Solution and the introduction to his forthcoming The Tyranny of Dead Ideals. The rhetoric in the intro to Tyranny is filled with a mix of technocratic rhetoric and management-guru "we must change to fit the times" nostrums, as if Miller were a genetic recombination of Marc Tucker and Spencer Johnson. 

There's nothing wrong with being technically superb, which is why I heartily approve Obama's designation of Peter Orszag as head of the Office of Management and Budget. But even if I agreed with all of Miller's policy ideas, there's something odd about his choices of arguments. Essentially, it's hard to build a case for major policy change around technocratic arguments, and I don't think you will find Obama following that path. Consistently in the campaign, he talked about his values and what he argued were shared American values; his stance was not "I'm competent, so trust me" but "Here are my values, and I'm a pretty smart dude." 

The Tyranny of Dead Ideas may be better than its introduction. It's likely to be read widely, and I just hope Miller makes better rhetorical choices in the bulk of the book than he made in the book's introduction or today's op-ed.

Sansom watch, December 29 edition

This morning, the St. Petersburg Times reported on an ethics complaint filed earlier this month with the Florida Commission on Ethics. The allegation in the complaint is that Ray Sansom "used his public position as a representative of the people to secure a $110,000 per year job as a vice president of Northwest Florida State College." The first substantive step for the commission is a review of the complaint's sufficiency: "Complaints need not be as precise as would be required by the rules of civil procedure in a court of law and shall be deemed sufficient if the complainant under oath upon knowledge or belief alleges matters which, if true, may constitute a breach of public trust." If it meets that test, the commission's executive director will request an investigation, and if the investigation finds probable cause, there'll be a hearing. But the length of this process looks indeterminate, other than required minimum periods to allow the respondent time to reply at several steps. If you are thrilled by administrative regulations, you can read the commission's full set of rules on reviewing, investigating and holding hearings on complaints.

December 27, 2008

Sansom watch, December 27 edition

Recent items in the news on Florida House Speaker Ray Sansom and his position at Northwest Florida State College:

  • Yesterday, the Lakeland Ledger published Sansom's handsome pension, which calculated how much his state pension would rise from his new position at NWFSC Vice President if he stayed at least 5 years: $4,236 more a month than if he had just been a state legislator.
  • Today, the St. Petersburg Times published capitol correspondent Steve Bousquet's column, House members stay quiet as public blasts speaker, which discusses the legislature's code of silence, and how ordinary Floridians are reacting to that silence. He quotes Marilyn Weaver's comments, aimed at Pinellas legislators: "Our household is so disgusted with the current Florida legislators for not speaking out and condemning what Speaker Sansom has done in enriching himself and bestowing favors to his college."

For those who are curious, yes, I will continue to note any news items I find on Sansom and his position.

December 26, 2008

The mixed value of GEDs

This morning in the St Pete Times, Ron Matus has a short article discussing the mixed reception GED recipients know they'll have in the world. Some earning GEDs have already faced life's curveballs interviews several adults who have received or are studying for their GED. According to one,

Samantha Fenwick knows all about the stigma. "People think if you got a GED, there must be something wrong with you, or you did something wrong," she says.

Fenwick doesn't need to refer to the GED work of James Heckman and others (though I'm a bit surprised Matus didn't): It's no longer a surprise or controversial that GED recipients benefit a little by earning the alternative diploma, but that it is not the same as a standard high school degree.

Nor should it be a surprise that the meaning of a GED lies partly beyond the credential effect. For some who take the GED test, it is a matter of respect, or sometimes family relationships, and there is almost a subgenre of articles about GED recipients showing their children (or their grandchildren) the importance of education.

The problem comes in assuming that the GED is the same as a high school diploma for public-policy purposes. If you think it's the same, then there's no problem shuttling lots of high school students into a GED program that's essentially test prep/warehousing. And that's how Florida currently measures graduation rates. If you think that the GED is better than nothing, but not the same as a high school diploma, then you let students go that route when they are far behind in credits, but you do not let high schools get credit for those students as having graduated. 

There is also the question of what is "far behind" enough in academic credits to justify that type of decision, and I do not know of a single study that looks at that. The data's there in adminsitrative records, and it would make a good project for someone who has the time and analytical skills.

December 24, 2008

Florida's corporate tax-credit vouchers and the OPPAGA fiscal impact report

Today, University of Colorado faculty member Kevin Welner has an op-ed in the Orlando Sentinel criticizing the Office of Program Policy Analysis and Government Accountability (OPPAGA) report on the state's corporate tax-credit voucher program and its fiscal impact. Welner argues that the claim of net savings to the state "is based on smoke and mirrors ... [and] concocts its numbers out of thin air." The key point: OPPAGA had to gather data or make an assumption about the proportion of voucher users who would have gone to private schools anyway. OPPAGA failed to gather data and assumed that only 10% of the students would have either paid tuition or received scholarships from the private schools. Based on that assumption, OPPAGA claimed that the state saved about $1.50 for every $1 it gave away (in the form of tax credits to corporations participating in the program).

Welner's point is important: the arguments about the fiscal impact depend on whether you think that the participating students are increases in the poor students attending private schools. As the OPPAGA report notes, if only 60% are private students who would not have attended without the program, then there is no net fiscal benefit to the state, assuming the rest of their models are correct. Since the number of participating students doubled in the past three years, it should have been simple to ask whether the new students (who otherwise were eligible to attend public schools in Florida in the prior years) were transfers from public schools. There are fewer than 1,000 participating private schools in Florida, the schools have to track students individually for audit purposes, and OPPAGA had to contact them for another part of the report, anyway. Is the failure to gather the crucial data a matter of flawed research or is it the result of an explicit directive by politicians? (More on that other part later.)

Then there's the question about the rest of the model. As one correspondent with the St. Petersburg Times noted, there is no analysis in the report about the difference between the fixed and variable costs for students. Some part of the cost of public education is scalable by student, while some of it is not scalable. The fixed costs of running a school is the reason why districts with shrinking enrollments close schools: it doesn't make sense to run a school for 50 students when the school was built for 500. If the beneficiaries of tax-credit vouchers are concentrated in a district, that district's fixed cost does not decline, and it rises in proportion to the remaining students. Since more than half of the recipients in Florida last year lived in three districts (Miami-Dade, Orange, and Duval counties), those districts had to bear a disproportionate burden of those fixed costs to serve the remaining students. Given the relative size of the voucher program and the number of students (21,000), even the most affected district (Miami-Dade) would have seen marginal impacts. But it's missing from the fiscal impact analysis, whose conclusions are distorted as a result. For similar reasons, a comprehensive fiscal-impact analysis would need to address the local costs of providing special education and other services that are not covered by the state or where the local district loses grant or categorical funding from the federal government. Depending on whether the federal government provides significant aid, a gain by the state may be balanced by a loss of federal funding. Essentially, the analysis is a simplified (and data-thin) calculation of the impact on the state government, not the impact on total revenues and services.

The big picture is important: the corporate tax-credit voucher program has been expanding rapidly, and as the actual state expenses looked like they would bump up against the $88 million ceiling, the legislature agreed to increase the ceiling and asked for the fiscal-impact report in the same bill. Because of the report's construction, I expect lawmakers who wanted to increase the ceiling further to argue for it as a matter of saving the state money. We still don't know the facts, though, either about the fiscal impact or how students are doing in the program.

And it's the last issue that was in the report but has gone unreported in Florida's newspapers. Part of the required report included a question of how to induce participating private schools to have their students take the state assessment for public-school students. The answer by private schools who received public funding for some of their students was essentially, "We don't want to participate, and you can't make us." The private schools contacted by OPPAGA told staff that they did not think the FCAT was an appropriate measure of what their students learned, that having their students take the FCAT would cause them to distort their curriculum, and that norm-referenced tests they already used would be sufficient. Now, where have I heard these arguments before?

December 21, 2008

Student debt, social investment in education and the search for a basketful of school

At the Social Science History Association conference this year, there were "author meets critic" sessions on two important books, Kathryn Neckerman's Schools Betrayed: Roots of Failure in Inner-City Education and Claudia Goldin and Laurence Katz's The Race between Education and Technology. Together, the two books represent solid new work in understanding urban education (with Neckerman) or arguments about the relationship between education and the economy (with Goldin and Katz). In particular, Goldin and Katz's argument is both a brief in favor of investment in education and a reply to skeptics such as Alison Wolf, author of Does Education Matter? Myths about Education and Economic Growth (2003). (Wolf updates the older arguments along the lines of Berg, Freeman, and