June 30, 2009

Grading reports that grade states, which have schools that grade

It's now a PR cliche in education wonkery: grade states. Issue grades, and that's a hook for reporters to write stories about the reports, because the reporters at daily metros can say, "[Your state's name here] receives 'F' in think tank report on education." But beyond the PR value of grades, it's facile, which is why I'm surprised Education Sector gave into this particular venal sin in its report on states' higher-ed accountability policies. C'mon folks: can't you figure out a more substantive way of evaluating states? At the very least, this is so 1990s.

So I'm thinking about developing a report over the next year that grades think-tank reports that issue grades for states on some matter of education, where of course schools have teachers who grade students. Among the standards will be the following:

Clear standards for grades: a year before the report is issued, does the entity that issues the report publish grading standards or criteria?

A - Entity publishes grading standards with sufficient criterion specificity that an outside observer would not be surprised at the grade a state receives the next year. (Note: this is a low bar, not requiring agreement with grades.)

B - Entity publishes standards, but standards are too vague to provide benchmarks for policy progress.

C - Entity has previously published reports issuing grades to states, but changed the standards, or described the project and the areas where states would be grade, but no standards for those areas.

D - Entity has previously published the existence of the report project, but there is no previous publication of intent to grade states in this area of policy.

F - Report appears out of the blue with no publication of intent in this area.

Okay, folks: where does today's Education Sector report fit? How about Ed Week's annual Quality Counts phonebook? Fordham's reports that issue grades?

And, yes, if I'm serious about this, that implies I have to develop some more grading criteria. After all, it would be most interesting and ironic if I created a report that contained the mechanism by which the report itself could be torn apart. Hint, hint, ...

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 2:11 PM (Permalink) |

Find the typo! and other national-stage blogging

The National Journal unveiled its new education policy blog yesterday. My first response has an embarrassing writing goof; see if you can spot it!

Listen to this article Listen to this article
Posted in Education policy at 9:31 AM (Permalink) |

No differences -> politics as usual?

While the DC vouchers debate swallowed more airtime, it's David Figlio's new study of Florida's voucher programs that will reveal the state of voucher politics. Several years ago, opponents of vouchers pointed out the lack of accountability for the programs, and in response supporters inserted a mandated study comparing achievement of students using vouchers to public-school students. Fortunately, they picked one of the best economists of education, who is careful and cautious and has done several studies of Florida's voucher programs in the past decade (including the best article on the topic, published in 2006).

Figlio's conclusion is roughly that given the data he had available, there is no evidence of differences in student achievement between those in the corporate tax-credit voucher programs and similar students in public schools. Further, the usually-cautious Figlio went out on a limb and said if additional data were available, he wouldn't expect the conclusions to change. This is not the only report I expect Figlio to produce on the corporate tax-credit voucher program, since the interesting questions for microeconomists are about how the shape of the market (the presence and size of a voucher program) changes its characteristics (esp. responses of public schools). 

But until that report is produced, and probably after it, the no-difference finding here mirrors a bunch of other studies. At this point, it looks like there is no solid evidence that students using vouchers perform better as a result, and in Florida at least, it also looks like students don't perform worse, either. So the voucher debate will not be settled by evidence of effectiveness, and we default back to questions of values embedded in public policy and the way that experiences shape the policy-relevant questions.

Those who support vouchers are spinning the no-difference findings as "vouchers do the job for less money, and choice is a positive value." Those who oppose vouchers are spinning the findings as "vouchers are no panacea, and choice can exist within the public system." And as voucher-receiving schools accumulate in the state, the ordinary politics of constituents make it hard for legislators to oppose eliminating the program. It is the last item that makes Florida (where a number of Democrats have voucher schools in their districts) different from DC (where the governing authority, Congress, has only one voting representative with constituents who use the vouchers). In the end, I think we'll see voucher programs generally stay in the states where they currently exist (primarily from the constituency-experience dynamic) but not expand much (because of the lack of evidence of great effects and because charter-school expansion in cities is an easier political sell).

Listen to this article Listen to this article
Posted in Education policy at 9:15 AM (Permalink) |

June 29, 2009

Prevent backtalk: turn on the television!

I knew it years ago, and in two studies released earlier this month and this week, I think both in peer reviewed journals, we have it confirmed: the best way to prevent teenagers from talking back to you is to turn on the television years earlier so that they don't develop the ability to talk back. So that spring day in 1996 when my wife and I decided to sell our television before moving to Tampa? A big mistake.

And there we were, deciding that we were advancing our children's interests. No, that wasn't it at all: they were 4 and 1 at the time, and we decided that since we didn't like their arguments over the television, we'd see how long we could go without one in the house.Answer: 13 years and counting. And no matter what arguments we have in our household, it's not about the channel the television's tuned to. Instead, it's about who gets the computer...

Serious side: The article released this week is more about the relationship between adult caregiver and child than about television, and it highlights the importance of one-to-one interactions at early ages. I suspect this will be followed by other analyses from the same data set.

Listen to this article Listen to this article
Posted in Personal at 11:35 PM (Permalink) |

June 28, 2009

The purpose of seminars/discussion

I'm at THATcamp this weekend and having a great deal of fun. (Check the Twitter archive for tweets with the #thatcamp tag...) But there is a lot of serious stuff here, and I was hoping that it would confirm or undermine the way I'm currently thinking about the problems of teaching online. The demography of the group doesn't quite give me enough of that reality check, since I'm in the minority as an experienced teacher; the majority of attendees are graduate students, staff members at one of the digital humanities centers in the country, or library/museum staff, but it still was a first shot at this. 

No disconfirmation in the relevant session, but it's honed the way I'm thinking about the purposes of a seminar or discussion. What many great humanities discussions share is the entree into and development of skills in a specific discourse and in "academicizing" more generally (to borrow a term from Stanley Fish). In memorable humanities discussions, teachers model analysis and establish an environment within which students can learn and practice close reading, the identification of key issues in a disciplinary or interdisciplinary context, the articulation of critical perspectives, and engagement in a dicursive community. 

Several characteristics of face-to-face classes contribute to that: the ability of a teacher to take any issue and analyze it extemporaneously, the ability to annotate material for everyone present (if verbally), the probing of assertions with either questions or counterarguments, and the capacity to revise arguments on the spot.

There are online tools for some of this, if without the immediacy. Diigo is a great social annotation tool; while it's not the type of immediacy that happens in close readings in class, I have some anecdotal evidence that it can be powerful for students. Teachers could take issues that pop up in discussion boards and expand upon them by modeling analysis and should probably be careful to construct prompts that set the stage for that. And I've been thinking about requiring weekly recorded fishbowl sessions with small numbers of students in my fall online class, as a way to generate some immediacy in the engagement.

In other words, no great insights, but the honing itself is important. And it required a bunch of people who are very comfortable online getting together face-to-face to bat around some ideas. There was an ironic moment in the session related to that fact: One staff member from the Center for History and New Media left the room just before the session to address some technical issues. I started moderating, and we generated a list of functions for seminars and discussions in general. She returned to the room, and as she started to talk a few minutes later, she said, "I'm sorry if this was mentioned before... I wasn't here at the beginning of the session."

Listen to this article Listen to this article
Posted in Teaching at 7:33 AM (Permalink) |

June 26, 2009

How to steer CYA-oriented bureaucracies, or why NCLB supporters need to think about libel law

Someone at USDOE sent me an invitation to listen to the June 14 phone conference where Arne Duncan explained how disappointed he was in Tennessee, Indiana, and other states with charter caps, let alone states such as Maine with no charter law, and how that disappointment might be reflected in the distribution (or lack of distribution) of "Race to the Top" funds (applications available in October, due in December, with the first round of funding out in February 2010). There are a few details that reporters didn't ask about (Duncan's somewhat surprising statement that a good state charter law would set some barriers for entry rather than establish a "Wild West of charter schools," and the way that small charter schools and charter schools with grade configurations outside state testing programs can stay off the radar for accountability purposes), but I was not surprised that two Tennessee reporters were called on for questions.

But apart from the selection of reporters for questions, the phone presser and other DOE moves made me think about the various uses of power in education-policy federalism. In limited ways, explicit mandates can be effective, if there is a sustained willingness within the USDOE (and esp. OCR) to make painful examples of the nastier school systems that try to evade those mandates. Offering technical assistance is another method, and despite the massive conflict-of-interest problems in Reading First, I agree with one of the researchers in the field who thinks that Reading First did improve primary-grade reading instruction, on balance. (Thumbnail version: hourslong scripts, ugh; explicit instruction in phonemic awareness and some other fluency components, obviously necessary.)


But neither heavyhanded mandates nor technical assistance can do everything, and neither works with the greatest motivation for both defensive and hubris-oriented bureaucracies: risk management. If you are a public school teacher or administrator, my guess is that you can identify some fairly silly action by your district that was motivated almost entirely by CYA motives, and if you can marry those CYA activities to pedagogy, you've been lucky or have a black belt in administrative maneuvering. (If you have such victories, please describe them in comments! Otherwise, we'll all wallow in the shared misery of observing defensive administering and the all-too-frequent ensuing train wreck.)

I think the federal government can shape bureaucratic behavior to the good by using that risk management and structuring accountability policies around that. And here's the lesson I take from my high-school journalism class in ninth grade 30 years ago: libel law in the U.S. generally recognizes the truth as a positive defense agaist libel allegations. That seems like a backwards way to frame the legal issue -- after all, isn't it common sense that a publication is libelous only if it's false? -- but the notion of a legal positive defense gives an individual or organization a way to organize behavior in a way that is both professionally appropriate and also make a legal defense aligned with professional expectations. Because the truth is a positive defense against libel claims, even an idiotic general counsel for a newspaper or publisher looks to the professionally-appropriate standard: is there documentation that the published work is true?

Sometimes a positive defense is not explicitly part of jurisprudence but evolves as a practical guidance for clinical legal work and internal advice for school systems. Observing procedural and professional niceties create exactly that type of positive defense in special education law. There is nothing in federal special education law to carve out an explicit positive defense for school system behavior, but many articles written by Mitchell Yell over the past few decades constitute a convincing case that school systems now have a de facto positive defense: professional documentation of decisionmaing and scrupulous adherence to procedural requirements are a positive defense against a broad range of allegations by parents of and advocates for students with disabilities.

Yell has argued (persuasively) that due-process hearing officers and judges use procedural adherence and professional documentation as a filter in special education cases. If a school district can document that it has paid attention to procedural mandates and has met professional standards for documenting decision-making, then hearing officers and judges are extremely reluctant to look at the substantive merits of those decisions. But if a school district has ignored standard procedural expectations that most districts meet, or if a school district has kept no or inadequate documentation of its decision-making rationale, then all bets are off and a hearing officer or judge will be much less likely to defer to the school district on professional judgments.

In essence, Yell implies, school districts can avoid adverse judgments if they pay attention to timelines and other procedural niceties and if they keep teachers and principals on their toes about current "best practices" as well as deadlines, notices, etc.  Not all districts are aware of this positive defense, or I suspect that some enterprising special education researchers could make a mint running seminars, "How never to get sued again." 

More broadly, I'm beginning to think that the construction of a positive defense against charges of incompetence would be healthy for school systems and state policies. The devil would definitely be in the details, but instead of being frustrated by a consistently observed school system behavior, maybe we should take advantage of that consistency.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 12:38 PM (Permalink) |

The right kind of infection

The Powell et al. article on cultural complexity 90,00 years ago, published in the June 5 issue of Science, has some interesting consequences for education policy, though it's an archaeology article. The argument the authors make is that one needs a certain population density before one can find surviving signs of cultural complexity (archaeological evidence of more sophisticated used of symbolism and technology). Sub-Saharan Africa had both those population densities and archaeological evidence from 90,000 years ago, as did Eurasia 45,000 years ago.

Powell et al. are arguing that the development of the earliest human cultural skills may have depended on nothing other than density. This is an appealing story: get enough humans living in proximity, and whatever culture is developed will be maintained while the various subpopulations (clans, etc.) interact and teach each other, keeping the ideas floating around the population in a way that would not happen in a sparse population with little interaction between subgroups.


I suppose that as someone without an archaeology background, I have no insider knowledge of the contribution this paper makes to studies of human evolution. The authors are portraying the issue as an explanation of how human culture could appear suddenly (on the eon-scale) without resorting to changes in biology (esp. cognitive capacity). We'll see what other researchers of human evolution say about that, but there's something important there for education.

The article suggests that one can categorize various cultural characteristics by the extent of continuity across time. Isolated behaviors and skills may not survive unless they spread beyond the individuals who may exhibit/learn them for a time. With enough contact among people, knowledge, skills, and behaviors can become continuous; that continuity is the subject of the article. But one can look at knowledge, skills, and behaviors that are more than continuously existing. Are they common (maybe the experience of a minority but where everyone knows several people who have that experience)? Are they the normative expectation? Are they ubiqitous (universal or nearly so)? That's a five-category, ordinal variable for the extent of cultural behavior in a population: isolated, continuous, common, normative, ubiquitous.Okay, there's a sixth category, absent. 

Many of the debates over education policy are about shoving the national population from a common experience of X to a normative expectation of X, or from a normative expectation to the ubiquity of X. In the space of 70 years, high school graduation moved from a continuous population behavior to a normative experience; that's the story in my first book. But the rhetoric surrounding a national population's experiences often obscures variations. As Claudia Goldin has pointed out, high school graduation became normative in the midwest and northeast by 1940, while it moved much more slowly in the South (for Southerners of all races/ethnic backgrounds). And today, while approximately a quarter of teenagers leave high school without a standard academic diploma, there are many high schools where graduation is common but not the majority experience, and probably a few high schools where graduation exists every year but is not common.While the latter should be alarming to anyone, in reality the majority of high schools in deep crisis fall in the former category, schools where graduation is common but not the majority experience. 

There is an argument that the Powell et al. article suggests: if culture "spreads" once there is a sufficient number of "carriers," maybe we should look at education as akin to a disease process that we want to propagate. This is close to the contamination theory Geoffrey Canada has (or had when Paul Tough followed him around while writing his book about the Harlem Children's Zone). There are both ways in which that argument is interesting (esp. in communities where half or more teenagers drop out without a high school diploma) and others in which it is disturbing (assuming that students can be "carriers" of culture in way that adults can manipulate, though they can't shape adolescent experiences directly.. uh, no).

How do you move a behavior from a common-but-minority experience to a normative expectation? That's essentially the question we have in a large number of high schools in the country and with regard to baccalaureate degrees for the entire country. At least in my understanding, there are two requirements, involving both the spread of an idea and set of habits (habitus, in Bourdieau's language of cultural capital) and also institutional infrastructure. Attending high school became the normative experience for teenagers when they could no longer enter the full-time labor market with ease, when people began to think of high school as an experience that could be useful, and when there were enough high schools for majority attendance to be physically possible.

I do not think that there are exact parallels for all circumstances, just a combination of population behavior and institutional behavior. They go together. And, yes, there are cases where the extent of cultural experiences can reverse: working-class attendance at Shakespeare in the late 19th century, if you believe Lawrence Levine, or girls' primary education in Afghanistan from 1995 to 2001.

Listen to this article Listen to this article
Posted in History at 12:29 PM (Permalink) |

June 25, 2009

Nothing fuzzy about fuzzy math

George W. Bush is probably responsible for people calling constructivist math advocacy "fuzzy math". There is a field called fuzzy logic, and while I know very little about it, I'm irritated that the former president's maladroit use of English is messing up technical terminology. Fuzzy logic is a useful tool in engineering, and while Lotfi Zadeh's original term may not be perfectly descriptive, math teachers should be the last to misuse a term that's in their own discipline.

Listen to this article Listen to this article
Posted in Education policy at 3:26 PM (Permalink) |

See-no-knowledge in education policy?

I seem to be reading several "we don't know anything so let's plow ahead" arguments in education think-tankery, from Mike Petrilli's argument that because we don't currently have a solid research base about how to turn schools around, we shouldn't try, to Kevin Carey's consistent argument in Education Sector's blog that because there is no research consensus about predictors of good teaching (and considerable research suggesting that there is not a link between effectiveness and countable items like years of experience beyond the first few or graduate degrees), it makes better sense to let people into teaching and then evaluate their effectiveness.

Fortunately, that's not the approach of the Institute of Education Sciences under John Easton, which has just announced a large research initiative on turning around schools. I suspect that both Petrilli and Carey would acknowledge that research in difficult topics is a good thing and argue that IES initiatives are different from policy, because sometimes you have to make decisions based on the state of knowledge you have, not the ... oh, shoot, there's Donald Rumsfeld phrasing again. But you probably know what I mean: Petrilli and Carey's stances are policy stances based on topic-specific agnosticism, not opposition to research.

But there's a serious question buried here: on big questions of policy, where you have to make choices, and the research is nondirective, how do you make decisions? I think the answer has to be incrementally, to allow research to catch up and influence policy later. If you make a huge political and institutional commitment to a policy path that has no research support and no ethical/legal obligation, then you're committing millions of children and hundreds of thousands of educators to a path that is very hard to change later. 

For that reason, while I think Arne Duncan's four-choice speech earlier this week is not based on research, and Petrilli is correct that there is no particular reason to believe that charter schools will somehow rescue the education of students otherwise stuck in horrible circumstances, the policy itself is good largely because it doesn't make hard and fast commitments to a particular path. The good thing about a charter is that it can be revoked, and in states such as Florida where there is a single authorizer for a geographic area (here, the county school boards), authorizers can be reasonably aggressive in shutting down shady or incompetent operations. So I share Petrilli's skepticism, but precisely because I am skeptical of any particular approach to schools in crisis, and because Duncan is being wishy-washy, I will applaud the Secretary for being wishy-washy. 

Update: I first used the term "know-nothingism" in the title. Ugh. Bad move for an historian. Petrilli and Carey are not members of the 19th century anti-immigrant party. Mea culpa.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 8:49 AM (Permalink) |

June 23, 2009

One more reason not to use Elsevier

If fake journals created at the whim of pharmaceutical companies weren't bad enough, how about paying people to assign five stars to textbooks in Amazon's review system?

Listen to this article Listen to this article
Posted in The academic life at 8:53 AM (Permalink) |

Being paid to sit in a/c

As Notorious Ph.D. wrote at the end of last week, most nine-month faculty do not get a vacation in summer but are just unemployed and often still have to work. Some such folks have plenty of resources to tide them over until the fall, others have a salary that is stretched out over twelve months, and yet others are paid fairly pitifully. I'm in none of those categories, having a salary that's well below the average for rank and discipline but higher than the median U.S. salary and considerably higher than the average wages through human history. I also have a paid teaching gig this summer, one course, and for most of yesterday from about 8 am until 9 pm, I was reading papers or engaged in various class logistics. Last week, any time not in class was spent on union work or a teaching workshop for high-school history teachers on the Spanish Civil War and American involvement in it. The latter is a far cry from a Teaching American History project, but it gives me a taste of what the best of workshops can be like.

This morning is one of those days when I had a substantial incentive to get to campus early: when I woke up, the temperatures were already in the mid-80s (F.). Right now, the weather station at Tampa International is recording 85 F. with 82% relative humidity. I keep telling myself that at least the sauna is free, and the driver of the car parked next to mine this morning added, "In the north, you have to shovel stuff." For the record, 82% humidity in mid-80s temperatures is darned close to shovelworthy, but not yet.

So I'm in my office, and with luck I'll be able to grade some straggler student papers before class. Because of last week's workshop and a whole set of other things, I'm behinder than usual on other matters. And if you think the third-to-last word in the previous sentence is not in fact a word, you may not have been reading a slew of student papers recently, and you might be one of those language mavens who would like to bury the student body, preferably next to Jimmy Hoffa (apologies to Strunk and White).

Listen to this article Listen to this article
Posted in Teaching at 7:48 AM (Permalink) |

June 21, 2009

Still mis-remembering Title IX

Two years ago, I pointed out that our modern discussion of Title IX focuses too much on athletics and ignores the broad sweep of changes brought by Title IX. So what does the White House blog mention when previewing its Title IX celebration? Athletics (in an entry written by United Nations Ambassador Susan Rice).

June 19, 2009

Conversation often works ... where it's tried

Today, ACTA's Anne Neal thanked the AAUP and AACU for welcoming her outreach efforts.Towards the end of the blog entry, she writes,

ACTA also shares many faculty members' legitimate concern about administrative bloat and about trustees who lack a sensitive understanding of the special protocols and values that underwrite the unique enterprise of higher education. That said, we also believe that it is the professoriate's job to reach out to trustees. Faculty should understand that presidents and trustees are engaged in enormously complex, vital, and often urgent fiduciary endeavors. They should also understand that, going forward, trustees must be included among academia's primary stakeholders, alongside faculty and administrators.

I hope that's possible; that depends both on faculty and on trustees not accepting upper-level administrators as gatekeepers. My experience in Florida is that trustees often accept the role of administrators as gatekeepers of information, so that a president can essentially filter out quite a bit. I know of one UFF chapter at a community college that was able to meet with the chair of the trustees and establish a good working relationship, but that's rare. Far more common is a fairly uncomfortable and unproductive divide between trustees and most faculty, with a handful of administrators controlling the interaction.

I suspect that there's a pretty easy way to prevent greater access from becoming a vehicle for cranks and sophists (who will get their word in, anyway): err... asking faculty to provide the reality-check filter.

For those readers outside Florida, what is your experience with the extent of interaction between governing-board members and faculty?

Listen to this article Listen to this article
Posted in Academic freedom at 5:29 PM (Permalink) |

June 18, 2009

The world is complicated, part 752

So the Center for Research on Education Outcomes has a report on charter-school performance, the Center on Education Policy has released a report on student achievement trends, NAEP released art-education data, and the spin has begun. Missing from almost all the reporting: Statements about the extent of peer reviewing for any of these reports. I'm not too worried about the professionalism of these reports,  since I know that the Department of Education always has an internal review process, CEP usually asks researchers in the area to review draft reports, and I would be surprised if CREDO did not have a pre-publication review process. However, the failure to report on the extent of peer review is a continuing and glaring omission in the reporting of education research.

In terms of the substance of the reports, I'm up to my eyeballs in prior commitments, but it's clear from the brief reading I have been able to do that the findings for all three reports are more complicated than the spin emanating for many of The Usual Suspects.* That's not news, I know, but I am the King of Things That Are Obvious Once He States Them, and I have a job to do.

* a great name for an a cappella group, if you happen to be starting one up.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 9:18 AM (Permalink) |

June 16, 2009

Iran's university students and faculty under threat

According to a Chronicle of Higher Education report late today, as well as Twitter reports and images/videos from bloggers who have been able to post, Iran's universities are under attack, either physically (with property destroyed and some reports of student deaths) or political pressures. If the clerical authorities gave orders for the Revolutionary Guard or other forces to attack universities, they are willing to throw overboard civic institutions as well as electoral politics to preserve their power. I'm not surprised. I'm very saddened, but I'm not surprised.

Listen to this article Listen to this article
Posted in Higher education at 6:49 PM (Permalink) |

June 13, 2009

On graduation rates and auditing state databases

I sympathize with Florida's Deputy Commissioner of Education Jeff Sellers, finding himself defending the state's official graduation rate the week that Education Week published its Swanson-index issue and pointed to Florida as a low-graduation state, using numbers far below the state's official numbers.

Some perspective: Florida's official graduation rate is inflated, but it's still better than Swanson's. Florida's graduation rate does more than Swanson (i.e., does anything) to adjust for student transfers and the fact that ninth-grade enrollment numbers overestimate the number of first-time ninth graders. 

Because of Florida's state-level database and the programming/routine that already exists, Florida is much closer to the new federal regulatory definition of a graduation rate than many other states, and Commissioner Eric Smith has been preparing the state board and other interested parties for the likely effect of the change on the official published rate -- i.e., that the rate will be a visible quantum lower than the currently-published rates (and largely for the reasons I have explained in the 2006 paper linked above). So in a few years we'll get a closer estimate of graduation from a lay understanding (the proportion of 9th graders who graduate 4, 5, or 6 years later).

The point in the St Pete Times interview where I winced was Sellers's answer to the question of how the state (and the general public) knows that the exit codes entered for a student are accurate: Sellers said that his department conducts an "audit from a data perspective."

That statement is misleading. It is technically true that there is an audit in two senses: each school district is required to check its data for accuracy before sending the data to the state's servers, and the state conducts a search of students reported as withdrawn in one county to see if they entered another county system before labeling them dropouts. But while I have seen reference to checking that the withdrawal codes are correct, I have not seen any evidence that such checks have actually occurred, and I have been unable to find that evidence anywhere on the Florida Department of Education website. That doesn't mean that it doesn't happen, but call me a touch skeptical. Without random checks, there is no guarantee that a 16-year-old coded as a transfer to another school actually was a transfer.

Given Florida's long experience with a state-managed education database, the lack of published audits of this process should caution us about the magic of state databases. They are important, but they need to be done properly. It makes sense to talk about the internal and external checks that should happen as other states construct databases and all states start to conform to the mandated longitudinal graduation rate:

  • Districts will need to be the first party to check accuracy, both in terms of preventing mistakes/fraud but also conducting consistency checks--are there any records which claim that a 45-year-old is attending kindergarten, for example? The first is supposed to happen in Florida, and I suspect that counties catch the low-hanging fruit in terms of errors. But the accuracy check on withdrawal code is the type of check that requires extensive follow-up to document whether a student identified as a transfer did in fact enroll in another school.
  • States will also need to conduct accuracy and consistency checks, though a state will necessarily be far less likely than school districts to catch outright fraud in claiming students transferred when they did not. 
  • States will also have to conduct the cross-checking that Florida currently performs every year and that I describe above: which students move between districts in the same state, but are counted as dropouts because a county only looks at its own students.
  • Finally, the auditing of transfer records would be MUCH easier if there is a standard way for school districts and individual schools to request the transfer of a student record and simultaneously use that authenticated request as verification that a transfer code is appropriate.

This is an incomplete list, but it's a start.

Listen to this article Listen to this article
Posted in Education policy at 8:55 PM (Permalink) |

Mostly useless... except for blogging

I don't know if any other bloggers has the experience I occasionally do, of wondering if people ever read my blog to see if I'm skipping work that I owe them. At almost any time, I owe scads of work to others, and I suppose they might all think I'm shirking my work when I blog. I suspect that is now true for Twitter and Facebook as well.  It's probably more of a suspicion with the social-networking sites, in part because there are a lot more people at any time twittering and puttering on Facebook than are writing blog entries.

I may post a few entries tonight, primarily because I can't concentrate on reading this evening. What do you do when distractible to the point when a gripping war memoir starts to put you to sleep? In my case, it'll probably be mopping the kitchen and blogging. And wondering who wonders why I'm blogging instead of working on a Saturday night.

Oh, yeah. Saturday night. That explains it, too.

Listen to this article Listen to this article
Posted in The academic life at 8:48 PM (Permalink) |

June 12, 2009

Stopping by the office on a steamy morning

Whose words these are you darned well know.
My house is not on campus, though
I've stumbled in by instinct. Guts
have steered me through the traffic flow.

My teenagers must think me nuts.
I work when they lie on their... beds;
you smile upon their summer ease?
Through morning steam the concrete juts

and swallows cars and spits out keys
and drivers all who scent the breeze
and recognize their own mistake.
It's not relief but just a tease.

I give my workday yoke a shake,
the methyltheobromine take.
They tell us we must be the best.
But if no mug, I'm just a flake.

The summer's lovely, hot and blessed,
but Friday morn finds me all stressed.
No beach for me, not half-undressed;
it's hours to go before I rest.

Listen to this article Listen to this article
Posted in The academic life at 8:46 AM (Permalink) |

June 10, 2009

Someone left me out of the conspiracy

Dear Rev. Jeremiah "Them Jews aren't going to let him talk to me" Wright,

Apart from fact that you spoke on the day a white supremacist shot a security guard at the U.S. Holocaust Museum (once again, your timing is impeccable), there are only a few things I have to say to you:

First, I have yet to receive my Protocols of the Elders of Zion Secret Decoder Ring, despite sending in the required two cereal box tops, the completed, photocopied application to be a Junior Elder, and proof that my mother is Jewish. Personally, I thought that her affidavit about being forced to attend Jacob Neusner's piano recitals when they were growing up in Hartford was both ingenious and obviously genuine, but either the envelope got lost in the mail or someone intercepted it. I have been thinking of blaming someone, but I have yet to narrow down the possible list of suspects.

Second, to hurl this invective a week after it's becoming pretty clear that President Obama is, in fact, pressuring Israel to stop expanding or creating new settlements? Yep, that's a sign of the Victorious Conservative Jewish Conspiracy if I ever saw one. Or maybe you were talking about the J Street Conspiracy? Please make sure you identify the correct Jewish conspiracy, because they have different addresses and phone numbers, though I'm fairly sure the phone numbers both end in "666." 

Third, I haven't heard personally from the president in over three weeks, and I suspect he's starting to listen to others at this point. To be honest, he hasn't called me since I suggested that Peter Sagal become the Official White House Jester. Well, actually, if you want to know the truth, he hasn't called me since the election, the ingrate after all I've done for him. Okay: he has never called me. But I suspect that's beside the point. If we were good buddies, despite my being a lousy point guard, the president would probably follow his own counsel. That's probably been true for most of the presidents we've had, and since he and Michelle have steadfastly refused to get a bearded dragon, he's also ignoring my wife as well as me.

Fourth: you want the two pink pills in the morning and the green one at night. Don't forget to take a glass of warm milk with the green one, since you can get an upset stomach if you forget.

Yours very agnostically Jewishly,

Sherman

Listen to this article Listen to this article
Posted in Random comments at 3:17 PM (Permalink) |

Teachers and school demographics

A few weeks ago, the Journal of Labor Economics published C. Kirabo Jackson's study of teacher moves away from schools in Charlotte that were moving towards single-race, segregated status (see lay description here; subscription-required article here).

Today, the Education Policy Analysis Archives publishes Kitae Sohn's article, Teacher Turnover: An Issue of Workgroup Racial Diversity (secondary site), which focuses on the potential attrition associations with teacher demographics rather than the student demographics. The punchline from the abstract: beyond a relatively small threshold of racial diversity among the teaching staff, "young White teachers are more likely to stay in their original schools when the proportion of minority teachers is smaller." The article was accepted well before I knew of Jackson's study, and there are a few small (and disturbing) nuggets apart from the main findings.

I suspect that for both of these studies, there will be replications, criticisms, and debates, and that's absolutely appropriate. Both articles focus on what is an important issue for policy (how do teachers make choices about where to work), and the conclusions are fairly disturbing. For that reason alone, I hope that they are the start of more work in this area.

Listen to this article Listen to this article
Posted in Education policy at 3:04 PM (Permalink) |

June 9, 2009

Drug education is NOT working... at least not at ED in '08

Strong American Schools' ED in '08 campaign became one of the most successful independent advocacy initiatives of the 2008 election season and has helped turn the need for education reform from a low-priority campaign issue into one of the Obama Administration's top policy priorities.
--"Final Report from Strong American Schools" e-mail from Roy Romer and Marc Lampkin, June 9, 2009

ED in '08 spent millions of dollars with almost nothing to show for it. That's not a shame in and of itself, because loads of policy and political experiments fail (and that's why we call them experiments). But when you flop on a big stage and then claim an Academy Award? Sheesh. Education was one of the lowest-visibility issues in the campaign, it's hard to see how education is trumping the economy or health care as a focus of the White House's attention, and it's even harder to see how ED in '08 is responsible for whatever attention is being paid to education policy. 

I don't know what Roy Romer and Marc Lampkin are smoking, but I'm tempted to ask.

Listen to this article Listen to this article
Posted in Education policy at 3:49 PM (Permalink) |

"Manufactured Controversy" the paper

Last week, Free Exchange on Campus published the latest thing other than student work to go on my to-read list: Manufactured Controversy, on the lessons to be learned from the attempt to politicize universities by David Horowitz and his allies. 

No, I haven't read it yet. But that's good news: this is a chance for you to get ahead of me in yet one more thing!

Listen to this article Listen to this article
Posted in Academic freedom at 12:37 PM (Permalink) |

June 8, 2009

No one ever accused Arne Duncan of impersonating an education researcher

Hopefully some day we can track kids from pre-school to high-school and from high school to college and college to career. Hopefully we can track good kids to good teachers and good teachers to good colleges of education.

This was an excerpt from a speech Duncan gave today to IES staff about the need to use data warehouses to link individual teachers and test scores and then use that linkage to evaluate teachers (hat tip). Oh, yes, and do it based on research. Some day, Secretary Duncan, but tying an individual teacher to student performance is not something that you can assert is based on research available today. It is more wishful thinking than anything else. The best apparent on-the-ground research of this type with teacher education is nonetheless full of caveats. And that's on a program-level scale, not on the level of the teacher. 

I'd accuse Duncan of spouting fuzzy logic, but fuzzy logic (the real stuff, research-wise, using fuzzy sets) may be one tool we use to get out of this dilemma.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 12:51 PM (Permalink) |

June 6, 2009

Sifting priorities, micro and macro

I had such good intentions this morning. After dropping off my daughter at the High School o' SATs, I figured I'd sit in the local Starbucks and read student work while she was wearing down No. 2 pencils. So there I was at about 7:45 in the morning, listening to slightly-too-loud Sinatra and reading drafts of one section of the major paper for the class I'm teaching this summer. After about a third of the batch, I bailed on both student reading and the environment of too-loud soft music and too-loud jovial fellow customers. I listened to Scott Simon's interview of Naturally 7 while driving a few blocks to the library branch that just opened up, and I'll sit here for the meantime, trying to figure out what to do for the rest of the weekend. As usual, I have Too Much to do, and I have to do some of it and not the rest. May I make the choices wisely, but more importantly, may I make the choices consciously.


In many ways, education policy and policy debates are about the same types of choices: you can't do everything at once, you can't fix everything at once, and being ambitious requires being selective about where you spend energy. It also requires a big-picture perspective. That's part of why I shook my head at Norm Scott's confectionary history of UFT. There's an important role for internal debates inside unions, and I have respect for UFT activists who are willing to go toe-to-toe with the most powerful teachers union leader in the country, but there are huge leaps of logic in Scott's thumbnail history and a failure to see a crucial big-picture issue.

Scott assumed that there was an overarching "sellout strategy" that Al Shanker consistently used after spring 1968, and that the sellout strategy was based on a circumscribed realpolitik vision of unions:

After the brutal '68 strike Albert Shanker knew the UFT could never again win much more than salary increases for teachers, and at some point only those at the expense of selling out. Thus over the next 15 years was born the "new unionism" where the union no longer is an antagonist but a cooperative partner with management.

The problem with this argument is not that it has no basis in fact but that it gives far too much credit to a single individual for the direction of the UFT (and AFT). Shanker was certainly a forceful unionist, and both the UFT and AFT were shaped by his leadership, but the general dilemmas facing UFT in 1968 were not new or unique, Shanker would never have been able to take the UFT on strike without the agreement of hundreds of UFT leaders, and there is something odd about the obsession of union dissenters with a single leader.

It's the last that's the most surprising to me on both intellectual and political grounds. If I were a member of ICE (a dissenting caucus within UFT), I would not be obsessing about Randi Weingarten. While focusing on individual targets can be useful for energizing one's base, it's useless for public discussion and the nuts and bolts of organizing and campaigning. To put it bluntly, it's following the reasoning template offered by the New York Post, whose editorial board loves to focus on personalities and the imagined virtues and vices of key figures. Imagine for a second that Shanker had died fifteen years earlier than he did, in 1982 rather than 1997. How would the history of the AFT have been different?

Oh, wait. We don't have to speculate. We can look at what's happened to the AFT in the past 12 years, since his death. There have certainly been stylistic differences, and the AFT has a far less closed culture (and is thus healthier) than it was at Shanker's death. But many of the strategic decisions taken in the late 1990s and early part of this decade would probably have been taken if Shanker had been alive, and it wasn't because anyone at AFT held seances to figure out "what Al would think" (despite the jokes made about Richard Kahlenberg's attempt to channel Shanker and probably some debates framed in that way). 

Consider the debates about mayoral control in New York City. I don't pretend to know the inside politics, but anyone looking at the picture three months ago could have predicted a few things:

  • Mayoral control would not be extended precisely as is, but neither would it end, and whatever came out would be a political compromise.
  • There would be test scores released that would be spun by multiple sides, and almost surely inaccurately on multiple sides.
  • Weingarten would have to make choices about where to push for change in mayoral control.
  • Someone would accuse Weingarten of being a sellout no matter what position she took, because she would be presumed to have given her okay for whatever came out.

I can't see either the logic in Scott's understanding of his own local or how Scott thinks teachers unions should behave in public debates such as over mayoral control. He either is using Shanker as a synecdoche for the strategic choices many UFT leaders have made over the decades or truly thinks that the key problem is that the wrong charismatic leader is in charge. Okay: Weingarten will be gone from the active UFT leadership in some months, so who's going to be the next target? I suspect that Scott knows deep down that his fight is with a very large group of fellow unionists who just disagree with his desire for more open conflict. 

One of the dilemmas with collective bargaining is the fact that the act of collective bargaining channels an adversarial conflict into a pattern of routines that then circumscribes relationships between union and management. Sit down and bargain, ratify, enforce agreements, picket and strike, lobby publicly for your members' interests and values: these are the public tools of power for a recognized union. A skilled union leadership knows how to use more than one of the tools at any time and if both wise and lucky will use the right tools more often than the wrong tools. An unskilled union leadership relies on a narrow set of tools in a predictable and increasingly less effective way until its members have essentially lost all the advantages of representation. But as several labor historians have pointed out (and my apologies for forgetting the names right now), there is no way to avoid the fact that if you buy into the legal authority of a union, you then buy into the set of tools that gives you.

Buying into that set of tools is not the only choice, of course; there's the historical example of the Wobblies who disdained contracts and collective discipline. I don't mean to suggest that the alternative is to match the violence by some Wobblies, but suppose for a moment that a union's leadership essentially ignored contracts, contract enforcement, and the like, and instead let the union culture evolve into wildcat direct action much of the time. There are two problems with arguments that unions should look more like the Wobblies (absent violence) than the UFT. First, I don't think it's a very smart political move. Because this country has 70 years of at least putative legal protection/recognition of union organizing and close to 40 years of effective public-employee organizing, most of the general public would conclude that anarchic direct-action participants over the age of 22 are trying to eat their cake and have it, too -- have the benefits of legal recognition without trying to take on any responsibility to follow the consequences of that recognition. In addition, in the internet age, glaring inconsistencies in the explanations of direct-action participants will make a union look like its members are less in touch with reality than George W. Bush, more manipulative than Dick Cheney, or both.

Perhaps more importantly, a lack of collective discipline and strategic choice is a path that is going to lose more often than win. Direct action does work where it's organized and lucky. It does not always work, and as one observer noted about the United Teachers of Los Angeles one-day strike fizzle, if it's intended as a public show without a broader strategy around it, it's nothing but street theater, perhaps entertaining and good enough for the evening news, but not enough to shape policy.

Maybe Weingarten needed to drive a harder bargain (and I think that's a reasonable position to take, that she made her peace too early), but you are making an implicit argument against collective discipline if you pretend that a union doesn't have to make strategic choices, make bargains with adversaries, or decide what is a reasonable settlement.

Listen to this article Listen to this article
Posted in The academic life at 10:23 AM (Permalink) |

June 4, 2009

My brother, the health-care policy wonk

Time to kvell: my brother Stan Dorn is currently a staff member at the Urban Institute, the latest step in a long career in health policy. Yesterday, he went toe-to-toe with Douglas Holtz-Eakin, former Congressional Budget Office director, and Dennis Kneale, apparently one of the designated Yelling Heads at CNBC who asserted that 20,000 dead each year from lack of health insurance is a "rounding error." Right. 

Listen to this article Listen to this article
Posted in Personal at 4:42 PM (Permalink) |

Clemson, prestige, and reputation

Despite its attempt to claw back from an unintentional statement of truth, Clemson's apparent manipulation/gaming of the U.S. News rankings system should give people one more reason to read Brewer, Gates, and Goldman's In Pursuit of Prestige (library copies), about the difference between colleges and universities that try to move up the rankings, on the one hand, and those that try to serve their students, on the other hand.

As I've stated before in a few contexts, few governing boards will hire a university president applicant who says, "Yep, you've got it just about right. I'm not making any changes and have no further ambitions for this place." That's just not the nature of the beast, and U.S. News rankings are often part of the discussion of institutional ambitions. So what to do to forestall this type of corruption or battle against the subtler forms, such as when universities want to raise the average SAT score of incoming first-time-in-college students? One way inside a university is to push for the inclusion of measures that focus on the service to the public. On the education side, that includes things such as the percentage of undergraduates receiving Pell grants and the percentage of students who are in the first generation of their families to attend college. Those links are to my own institution's strategic-plan data system, and they show that we're headed in the wrong direction on these important indicators, though the change on the Pell-grant proportion is small. I know from the development of this strategic plan that one of the measures was in there to begin with (for which the university administration deserves credit) and another was pushed by a faculty member (for which both the faculty and the university administration deserves credit).

Now here's the frustrating part: no one is holding us accountable for this. In the abstract, there are writers such as Peter Sacks who can uncover the shenanigans Clemson's administration apparently is engaged in and explain the connections to the college opportunities for children from poor and moderate-income families. But that's pretty abstract. In the din surrounding education budgets, together with shrinking news holes in your nearest metropolitan daily, there's little chance for the type of accountability that matters: discussion in a community about the public value of a college or university and where the institution should be headed to increase that public value. 

And, yes, that includes private institutions: you and I are indirectly paying for them with tax deductions to their donors.

Listen to this article Listen to this article
Posted in Higher education at 7:47 AM (Permalink) |

June 1, 2009

The Procrustean bed of teacher tests

Mike Petrilli's stab at the Sonia Sotomayor nomination via the Massachusetts teacher tests is a little askew, and I'm surprised he didn't look at an obvious dilemma that's deeper than the politics of a judicial nomination. Several former teachers have sued the state (and Pearson) for what they claim is a discriminatory impact of teacher tests given the disproportionate failure rate of minority teachers. This is the employee side of impact-analysis law that most school lawyers probably know better under the graduation-exam cases in Florida and Texas.

The landmark case here is Debra P. v. Turlington, which led to a number of federal decisions that guide the use of tests that have disparate impact in schools. To wit, tests with disparate impact by protected classes are acceptable if...

  • There is a rational state purpose for imposing them (guarantee graduate skills, in the Debra P. case)
  • There is sufficient notice to those affected
  • Those affected have a reasonable opportunity to learn the material on the test (the key reason for delaying graduation test applications in Florida, where federal judges did not want to hold the victims of segregation responsible for the unconstitutional behavior of schools)
  • The application of the test is professionally done (I'm bundling together several separate issues, including the composition of the test, defensible setting of cut scores, multiple opportunities to retake the tests, etc.)
  • There is no better way to meet the state's purpose that also reduces the disparate impact.

In the employment context, Petrilli is probably correct that the translation of the first item is essentially whether the test is a reasonable proxy for necessary teacher qualifications. But there is almost no way for anyone engaged in the current debate over teacher qualifications can defend these tests or defend the teachers' lawsuit without having some fairly severe inconsistencies.

Consider first the folks who have the approach that we should not care who enters teaching as long as we measure student achievement and make personnel decisions as a result. Several (whom I will not name to protect the guilty) have accused the High Quality Teacher standards in NCLB of obsessing about inputs (i.e., what teachers know) in contrast to outputs (what students learn). Anyone in this camp should abhor the Massachusetts teacher tests (and all teacher tests) because they continue the "let's look at the teacher qualifications absent the kids" approach, and we should be moving away from proxies for teacher effectiveness.

But the lawyers for the teachers and their supporters are not in much better shape, logic-wise. It is going to be very difficult to knock the legs out from the state's teacher testing program. They have to argue that the tests are a poor proxy for teacher skill, or that the tests were poorly constructed, or that there is a better option with a reduced disparate impact. If they cannot convince a judge that the tests were constructed and administered unprofessionally, the lawyers are going to be in the uncomfortable spot of arguing that the testing is an inferior proxy for judging teacher quality, in contrast to ... [The conclusion is left as an exercise for the reader.]

Summary: If you are in favor of judging teachers by student learning, then content-testing knowledge is a poor proxy by your own arguments. If you are against the content-based testing, then you have to come up with a better standard that will hold up in court. No, I don't think there's a way out of this for anyone with skin in the game, but if there is no summary dismissal and no evidence of rank incompetence in test construction, the fireworks will be interesting to watch.

Listen to this article Listen to this article
Posted in Education policy at 5:58 PM (Permalink) |

Texas, South Carolina, Missouri, and Alaska

I know that the reports of the common-standards agreement shepherded by the Council of Chief State School Officers and the National Governors Association describe a few different reasons for why four states have not joined in a standards framework that is probably going to be about as close to a less-is-more approach as one can get in a bureaucratic standards document. Yes, I know Texas has just drafted standards (as has Florida, which is joining), that Missouri is searching for a new state superintendent (my guess is others are as well), that South Carolina has Mark Sanford (which is enough for any state to deal with), and that we haven't heard from Alaska. But here are my imaginary real reasons for why these states have opted out (thus far):

  • Other states refused to agree that everyone in the country would have to pronounce Harry Truman's state as mizZURah.
  • Texas would have to admit that bidness is not a word.
  • South Carolina did not get its way that there would be history standards with the required benchmark, "All six-year-olds will understand that each state is required to have at least one completely nutty elected official at all times, and this is a heritage of the Founders." 
  • There was a riot, not when Alaska insisted that NAEP math exams all use the Iditarod as an example of measure, rate, and general all-round toughness (other states just wanted to add their own events), but instead fisticuffs broke out when the Alaska rep. insisted that the current accepted size of the Earth was incorrect because if it was as large as most people thought, then you couldn't see Russia from your house.

Unfortunately, I suspect that the truth is far less entertaining. That's okay. We still have Joe Biden and George Will to mangle the facts in an interesting way.

Addendum: Lest anyone think I am making fun of other states, I should be very clear: I grew up in California in the 1970s, and I now live in Florida. That's enough ridiculous states to live in for a lifetime!

Listen to this article Listen to this article
Posted in Education policy at 9:54 AM (Permalink) |