June 20, 2006

Graduation rates, redux

Today, Education Week published Chris Swanson's new round of estimates of graduation. Lawrence Mishel also posted a new set of comments in a discussion of rates that's now considerably longer than the original entry. Andrew Rotherham points to my own state, Florida (which according to Swanson ranks as fifth worst), and Ron Matus's quick story in the St. Pete Times notes the differences between Florida's official calculation and Swanson's.

For those who need a scorecard for the grad-rate "players" (i.e., newly-coined methods of calculating graduation)...

  • The Boston-area researchers (Walt Haney, Gary Orfield, Jing Miao, etc.) use a straight diploma:9th-grade or diploma:8th-grade quasi-longitudinal rate (going from graduation back in a pseudo-cohort line to 9th grade 4 falls before or 8th grade 5 falls before). The diploma:9th-grade measure conflates grade-retention issues with graduation.
  • Warren uses the diploma:8th-grade rate plus a migration/mortality correction (using smoothed Census bureau state population estimates by age). In my opinion, this is the best-justified method using administrative records (such as the Common Core of Data). It's also useless at the local level because of the need for some data for the migration/mortality adjustment. (Mortality is so low for teens that it's not a serious concern, but I mention it for completeness.)
  • The US DOE has an "average freshmen graduation rate" that is akin to the Boston-area uncorrected quasi-longitudinal rate except averaging the pseudo-cohort's 8th, 9th, and 10th grade enrollments. This is an attempt to address 9th-grade retention, but Mishel and Roy are correct that it's jerry-built rather than having a theoretically-justified basis.
  • Greene and Winters use the USDOE averaged-freshmen rate plus a migration/mortality adjustment that is almost identical to Warren's (which Warren proposed in a 2003 paper).
  • Swanson's (and now Ed Week's) method chains together proportions of nth to (n-1)th graders in two successive years to get a quasi-period measure. It's entirely uncorrected for grade-retention and migration/mortality issues.

Swanson's production of numbers down to the county and district level will get large play in the broadcast and print media in the next few days, even though it has some serious technical problems. Those technical problems in some places are swamped by large differences in graduation in some places (South Carolina probably does have close to the lowest graduation rate if not the lowest, as Swanson claims), but the actual numbers are going to be inaccurate, especially in cases with significant net migration or 9th-grade retention. And it's at the local level where you are likely to see such cases. For example, consider Detroit, which Swanson says had 22% graduation for 2003. Detroit's PK-12 enrollment also shrank from 173,742 (in 2002-03) to 150,604 (2003-04), a net outmigration that would lead to an artificial deflation of Swanson's measure. That doesn't mean that Detroit is a great school system. It means that Swanson's measure is largely useless for Detroit.

Migration is the great problem with trying to estimate graduation at the local level. Without an audited trail of transfers in and out, there is no conceivable way of calculating an accurate graduation rate for a school or district.

Listen to this article
Posted in Research on June 20, 2006 6:12 PM |