So let me try to address the substantive policy issues. No Child Left Behind created a large program (Reading First) to give money to states that promised to adopt early-reading programs with significant research support. This came on the heels of a National Reading Panel report that emphasized the importance of phonemic awareness and phonics instruction to early reading, among the focused questions it addressed.

For now, let me skip the question of the NRP report, since I'm not a reading research specialist (see completely ambiguous disclosure at the end of this entry). Instead of looking at the reading research base, I'm going to make the point that at least the implementation was bollixed up. The Department of Education's appointees to various pieces tied to Reading First were often tied to people at or from one institution (the University of Oregon), and the Inspector General's report was concerned about both conflicts of interest and also the way that many states felt pressured to adopt a specific curriculum/reading program.

I don't have much experience reading program audit reports, but from the few I have, there's an understated quality to most of the language, and it's not clear from the outside whether the muted tones necessarily mean, "Well, someone complained, and there are minor problems," or whether they mean, "I'm going to be very polite, but at least one person screwed up massively, and the only reason why no one's being prosecuted here is because there's no covering statute or the threshold for conviction is pretty high--but since I'm an auditor and not a prosecutor, I'm staying well out of that territory." I'm on the outside, so I have no clue which is which with the Reading First report, though I looks like it shaded into at least minimal corruption.

So it's possible that the Congressional bristling at appropriating funds for Reading First may reflect some informal briefings about the extent of problems. But it's not that simple, either, since Reading First appropriations may also be the way that Congressional Democrats can exercise limited authority over the Bush administration scandals: it may be possible that since Democrats can't punish the DoD or key administration figures over Halliburton the way they'd like, they're going to Make Damn Sure that other shenanigans are shut down (or programs they perceive to be shenanigans). Whether that shades into partisan battles probably depends on your partisan leanings.

... or it may be the standard legislative Scandal Fatigue: "We're not sure exactly what the problem is, but something's wrong, the program evaluation doesn't appear to look good, and maybe just wiping the slate clean is best."

... and wiping the slate clean may be best, both for state officials who want funding for reading programs and also for children. There will probably be a new reading program, with several new statutory requirements to prevent a repeat of what the IG found (or what Congressional leaders think the IG found or are concerned about because of the report or what their staffers think is a good idea in response to the audit report or...).

Whatever federal program comes out of the ashes of Reading First may be as closely related to phonemic awareness and phonics as Reading First, but it may not. The evaluation cracks open the debate over teaching reading that the NRP never really closed. I'm not sure it's that controversial that fluency is important but not sufficient to guarantee comprehension. But Big Bucks are involved, so everything gets magnified. The corruption in Reading First hasn't helped that, either.

(And now the disclosure: My experiences are firmly on the side of phonemic awareness's importance: I was a postdoc with a fellow postdoc who was a firm advocate of Direct Instruction (with capital letters), and I've seen similar stuff work with struggling young readers. And one of my children clearly learned to read relying first on phonics and classic blending instruction (together with individualized picture mnemonics to learn the ball-and-stem letters' sounds). But my DI friend's roommate was a comprehension researcher who teased her friend, "So after your kids learn to sound out words fluently, they need to come to me to learn what the stuff means!" The struggling readers I mentioned earlier also had the benefit of engaging text. And my other child clearly was a print-convention person whose learning of reading didn't appear to need phonics instruction, as far as I can recall. Go figure, but if you can find an ax I'm grinding here, you're pretty creative.)

• What Should Happen in Our Houses of Learning? (Ravitch)


• A Marshall Plan for Teaching (Meier)


• Is Finland the Answer? (Ravitch)


• Let Schools and Districts Defend Their Solutions (Meier)

Here we see two articulate educators defend very different views of the Good Education--not philosophical questions but the policy question of whether a centralized curriculum is appropriate for a state or country. Ravitch favors a centralized curriculum that is less prone to what she sees as the weaknesses of localism. Meier favors local choices that can reap commitments greater than central control.

I think we can take the strengths of each position for granted: Meier has run several schools with very local missions, and she has done so remarkably well. Ravitch points to situations where a centralized curriculum has strengths (such as Finland).

What neither addresses very well are the weaknesses of their own positions. I think Ravitch has a point with the weaknesses of localism: while Meier and many other educators can and have constructed unique curricula that serve students and the community well, there are plenty of cases where localism led to low expectations or just nutty ideas (my phrase, not Ravitch's). And Meier and other critiques of a centralized curriculum have a point: there are plenty of centralized curricula that fetishize knowledge and discourage in-depth probing of key questions.

But those same weaknesses are also often true of each advocate's preferred choice: institutional inertia can easily turn centralized curricula into whatever was gong on in the status quo ante, and local curricula can fetishize factoids as easily as a centralized curriculum.(And don't tell me that a national test will do much to discourage either problem: no test does more than lightly sample any curriculum, and the most easily testable parts of that curriculum.)

What Ravitch and Meier show is that the debate over the curriculum is not just over the stuff but also the how and who and all sorts of meta-issues that focus on control: should a state or country's political leadership (or bureaucratic leadership) decide what children learn, or should teachers and communities (the local bureaucratic and political leadership)?

If you're wincing at that expression, I've made my point: this is the wrong debate to have. Yes, control is important, but whatever level of government/institutions make curriculum decisions, there needs to be regular discussions about what children should be doing and learning. To be honest, neither world-class standards nor community needs cut the mustard with me, because they're shortcut jargon. Here's a challenge: start with a single student's work and go from there. Since history is my discipline, we can use an essay by any student in middle or high school and ask the question: Is this what students should be learning about history and doing in a history class?

Then get a batch of student work with ranges in skills and purposes (of the assignments). What looks "right" to you? Are most students at that point, and what would be necessary to get more students there?

Then look at the discipline more broadly: how much of the thousands of books written in U.S. history in the past few decades is enough stuff to learn in secondary school? What would be embarrassing for students to graduate without knowing? Then, a second look: how many winces can we stand on that point, because adolescents are remarkably forgetful about history?

Then a third look: how would the teaching of history have to change to get the minimum number of winces? How would professional development look? Can the state or country pay for that? Can we afford not to pay for it?

In history, I don't believe anyone's gone through this type of iterative process for K-12. Some parts of it, certainly, but not all of it, except for higher ed. And with due respect for one of my national affiliates I'm embarrassed that the AFT gave Virginia's standards a "100%" for its standards. (That curricular micromanagement is an embarrassment to the discipline.)

I am tempted to comment on a bunch of items in the news, and I may in the next few hours, as relaxation, distraction, and celebration. I also need to work on the next English EPAA article, write a few disposition e-mails, herd a few cats (reviewers), finalize the summer class I'm teaching in June and July, schedule a talk with a coauthor, decide if I'll write the book prospectus I was speaking with a series editor about in the last month, write a note to a research group I'm facilitating, Do Union Stuff,... and have a life.

But today is a bit of an in-between moment, with some shepherding of my children to events and small stuff. Too bad blood donation rooms are closed on Sunday, because I haven't given in a while. (I'm eligible, and I have veins the size of superhighways, relatively speaking.)

In both cases, there was a broad array of opponents, though the bill to undermine science standards was far closer to passage. In the case of university governance, the state's university faculty were joined by the editorial boards of major daily newspapers, the state's Chamber of Commerce, a business development group called the Council of 100, and a former private-university president who is now a state-house representative. Everyone who opposed the proposal deserves credit for killing it.

* The pronunciation is commonly "sigh-nee die," though purists would probably prefer "sin-ay dee-ay." I still like "sins die," but maybe that's because I'm now completing my 12th year in Florida.

... when UFT President Randi Weingarten blew the whistle on the DoE's wasting of taxpayer funds at City Council hearings, the DoE retaliated by publishing the New Teacher Report it had been holding for this moment...

So part of this is the question of substantive policy, but another piece is the allegation that the NY DoE was being manipulative, essentially making policy by press strategy.

Incidentally, we'll now be able to judge the UFT's details by the city Department of Education's response. Here, remember the adage about what lawyers do: If you have the facts, pound the facts. If you have the law, pound the law. If you have neither the facts nor the law, pound the table.

Like the bulk of previous research, including Belfield and Levin [and several other studies], ... the new study finds that student achievement in public schools improves as vouchers expand the set of private options.

Greene is referring to a 2002 review by Belfield and Levin in Review of Educational Research. I remembered it differently and went to the source, where the abstract says the majority of studies show positive effects. So far, so accurate. But here's the next sentence:

The positive gains from competition are modest in scope with respect to realistic changes in levels of competition. The review also notes several methodological challenges and recommends caution in reasoning from point estimates to public policy.

Was Greene's link appropriate in that context? I give leeway on blogs, but Belfield and Levin is far more cautious about voucher programs than Greene is, or rather Belfield and Levin's article has far more cautious conclusions than Greene implies.

This comment focuses entirely on aspects of 34 CFR 200.19 concerning graduation rates (esp. 200.19(a)). There are several strengths of the proposed definition and two weaknesses in the proposed regulations (one in the definition of a permanent standard rate and one in the proposed transition option of the Averaged Freshmen Graduation Rate).

In addition, the proposed regulations would require additional discussion around minimum cohort and subgroup sizes (and alternatives for very small cohorts/subgroups), they also would require technical assistance around documentation of transfers, and there should be a follow-up technical study on differences in measures depending on whether a cohort is grade- or age-based.

Strengths of proposed changes to 34 CFR 200.19:

Strength 1) The move to a longitudinal rate based on ninth-grade entering cohorts uses the most accessible and publicly understandable option of several valid ways to measure graduation. (An alternative would be to use an age marker, such as students' 14th birthdays--for several reasons, using age cohorts would be superior to a grade cohort, but the technical improvement probably does not justify changing the regulations, at least until we see what differences arise.)

Strength 2) The definition of a cohort eliminates several loopholes that states have been documented to use (e.g., students who drop out to join a GED program are removed from a cohort, and students who are retained in grade are moved to a later cohort--both of which are events that would not trigger removals from the cohort under the new 200.19(a) definition). Of particular note is the definition of a confirmed transfer as requiring official documentation that the student has moved to a program that ends in a standard academic diploma.

Strength 3) The disaggregation of graduation rates (in 200.19(e)) is absolutely appropriate.

Weaknesses of proposed changes to 34 CFR 200.19:

Weakness 1) The definition of a four-year graduation measure is intended as a proxy of an "on-time" graduation rate. It is not entirely clear what the difference between graduation rates means, with a single indicator: does the fact that one school has a 70% on-time graduation rate and another school has a 73% on-time graduation rate means that the second school has higher overall graduation or that its students tend to graduate earlier... or even that they are more likely to be retained in 8th grade (something not accounted for in the draft regulation's definition)?

While there is no formal analysis I am aware of on this point, I strongly suspect that an on-time rate will be more sensitive to on-time/lateness issues than the overall level of graduation. Large, 20-30 point differences with large cohorts are going to be the result of substantial differences in the overall level of graduation, but smaller differences with large cohorts, or larger differences with smaller cohorts, might well reflect a slight delay in graduation rather than differences in overall graduation rates if one were able to look at completed-cohort experiences.

For policy reasons, I strongly advise against using such a restrictive definition with subgroups and smaller cohorts. In a cohort of 30, if all graduate but 10 graduate in their fifth year of high school, the four-year graduation rate will be 67%. Is that an accurate picture of the cohort experience if the rest graduate in the next year? Moreover, a four-year-or-bust measure gives schools no incentive under NCLB to keep students in school past the fourth year, and our national experience of the past 5 years has shown that many schools respond to the mechanical parts of NCLB in perverse ways.

A better regulation would require reporting of the four-year graduation rate but the calculation of three different measures and the ability of the state to craft an index that combines the three different measures:

a) A four-year cohort graduation rate
b) A five-year cohort graduation rate
c) A longer-term cohort graduation rate

If the regulation were to define a legitimate graduation index as "a weighted combination of the three cohort graduation rates where the four-year cohort graduation rate has no less than a 70% weight," that would serve the public interest in emphasizing on-time graduation without unduly penalizing schools that graduate high proportions of students, if some of them are not "on time." (While there are ways to model such measures with two years' worth of data using synthetic cohorts, there are a variety of ways of constructing the longer-term rates.)

Weakness 2) The Averaged Freshman Graduation Rate is an inadequate substitute for a true longitudinal rate. It is NOT true that "It has been shown to be a reliable, accurate estimate of the high school graduation rate" (Federal Register, p. 22025). Seastrom et al. 2006 (the reference used for AFGR) was written and submitted to the internal USDOE review process before the publication of John Robert Warren's 2005 article in Education Policy Analysis [Archives], which demonstrated several technical flaws in AFGR and other commonly-used graduation measures. (A note in Seastrom et al. acknowledges that it does not include or respond to Warren's analysis.)

There are two primary weaknesses in AFGR, in practicality. One is the tendency for any measure relying on aggregate data to be vulnerable to unmeasured migration/transfers. Without auditing, this first weakness is very difficult to remedy. The second is the lack of any explicit modeling to connect the average of 8th, 9th, and 10th grade enrollment in successive years to first-time 9th grade enrollment in the middle year. For a variety of reasons detailed in Warren 2005, the better alternative is to use the prior year's 8th grade enrollment as a proxy for the next year's first-time 9th grade enrollment. This does not eliminate the problem of unmeasured migration/transfers, but it is a more sound measure from a modeling perspective, and its elements are as readily available to calculate as AFGR.

Additional considerations:

1) Minimum cohort and subgroup sizes (and alternatives for very small cohorts/subgroups). In the Federal Register notice, there is no discussion of the numbers in a cohort that is a minimum to be reported for graduation rates (either as a subgroup or as a cohort size). Because graduation rates are highly vulnerable to misspecified transfers/migrations, it is especially important that there be alternative measures available for small group sizes and graduation rates. The Federal Register notice does not indicate whether the graduation rate changes come under the same averaging rule as other measures for AYP. If so, it may be necessary to issue nonregulatory documents on whether such averaging (or smoothing) has to be either weighted or unweighted. I would recommend that such changes be left open to states, since it is unlikely that either weighted or unweighted smoothing would clearly give an advantage to states in "gaming" the system. (I would recommend some simulations, though, to check.)

2) The USDOE will need to provide technical assistance around documentation of transfers. SIFA and the CCSSO EIMAC projects are currently in (what I think are) the formative stages of providing useful technical guidelines, and the requirement to confirm transfers for cohort-adjustment purposes should push those types of projects to the front burner on both funding and also USDOE support to resolve potential political issues among states and between different sectors of education (public and private).

3) The need for a follow-up technical study on differences in measures depending on whether a cohort is grade- or age-based. The proposed graduation rate definition for 200.19(a) is based on first-time-in-ninth-grade cohorts. This comports with common understandings of a high school graduation rate and with several existing measures. It is not entirely clear, however, that this is appropriate, both because of the numbers of students who are well over the age 14 when entering high school (because of grade retention below high school) and also because of the ninth-grade "bump" that has led Warren and others to recommend 8th-grade enrollment as the base number for cohorts in aggregate-data estimates.

One alternative would be to use an age rather than a grade as the starting point -- thus, one could use birth years as the natural cohort, with the students in a school/district as of their 14th birthdays as the start of the cohort (with adjustments after age 14). This age-based cohort measure would fit better with standard demographic analytical models. But it is not clear how much difference the two measures would make. For this reason, I recommend that USDOE or NIST call a workgroup together no later than 2010 or 2011 to examine the practical and theoretical differences between age- and grade-based cohorts.

Our findings suggest that giving college students multiple concrete examples may not be the most efficient means of promoting transfer of knowledge. [emphasis added]

To outsiders, our bucket may seem full, but at Harvard, we so often see aspirations we hope to fulfill that we can never have too many resources.

• Good: The proposed regs propose a longitudinal graduation rate as the long term, preferred measure of graduation.
  
• Bad: The proposed regs allow the "averaged freshman graduation rate" as a transitional measure until 2012. AFGR has little empirical basis for its estimate of the ninth-grade cohort size.
  
• Good: The proposed regs eliminate several loopholes I've seen states use to inflate graduation rates, including shifting a student's cohort when the student is retained in grade, removing the student from any cohort if they drop out to enroll in GED programs, and so forth.
  
• Odd: the only graduation measure proposed is a four-year cohort rate. While I disagree with Leo Casey's claim that the cure is worse than the disease, the four-year-only rate fails to acknowledge or credit schools for promoting graduation on any schedule other than a strict four-year schedule. It makes much better sense to report a four-year rate, a five-year rate, and an any-time rate.

In addition to the perennial issues with special education, there was somewhat more focus on two topics: the Responsiveness to Intervention (RTI) policy initiative and concerns several presenters had about personnel preparation. There was an interesting range of views on RTI, with some disagreement about where special education fits on the levels/tiers and what proportion of students would be in the chronic non-responder category. Mary Brownell effectively made the point that no matter what teacher education and personnel preparation models you want to use, there are uncomfortable dilemmas. There was plenty of other discussion, and there will be follow-up to turn the papers into something more.

And I received some nice comments on the style of my slides as well as the substance. When I get home (I'm working in the grad-student office at the U. Delaware school of education), I'll upload a few of them.