June 8, 2006

How graduation estimates vary by migration

It's a bit late tonight for me to continue with some of the obligations I haven't gotten to yet this week (several manuscripts to read/evaluate for EPAA, an EPAA article to begin preparing for publication, a student paper from an incomplete a while ago, a book review, a commentary on a lecture, and probably a few other things), so I'll write a bit about the stuff that's almost ready to put in article form, about graduation. You can see some details of my approach, more formally, but the crucial bit in this piece is that the demographic approach I'm using allows for seeing how estimates of graduation change depending on what migration conditions hold.

My example here is from Virginia statistical reports from the late 90s through 2003-04, because the state gives generally plausible estimates of enrollment by age and grade as well as total numbers for graduates in different categories (which I have collapsed into standard academic diplomas, special-education diplomas, and miscellaneous other certificates—including GEDs and certificates of completion). Given the data provided, one can see how graduation estimates (for this entry, the likelihood of earning a standard diploma from age 14 up) change as assumptions about net migration change. If I had much more time and programming skills on my hands, I could see how the estimates change as one changed the migration estimates age-by-age (the model is that detailed), but I've taken the simple road and seen what happened if one assumes constant migration and changes that hypothetical migration rate. If there's a high net in-migration rate, for example, then the straight-up (zero-migration) estimates of migration will be biased up because the algorithm wouldn't make a distinction between cohort-size changes and migration. But the question is how much does migration affect estimates of graduation?


The answer is: quite a bit! The figure above shows biennial estimates of standard-diploma graduation rates for Virginia between 1996 and 2004 (each set of estimates is a different curve), showing how the estimates (y-axis) changes depending on the hypothetical migration rate (x-axis). As explained above, net in-migration drops the estimate to account for the (hypothetical) bias, and net out-migration raises the estimate.

If one looks at the 1997-98—1998-99 estimate (and one needs two years of data for this calculation), a change of migration of just 0.01 is fairly dramatic. A zero-migration assumption leads to a rate of 74.0%; a migration rate of 0.01, 70.6%; a migration rate of 0.02, 67.5%. Now, there's a world of difference between zero migration and a 0.02 rate (which is substantial if not world-changing). But the fact that the graduation-rate estimate changes about 3% for every 1% change in the net migration rate is rather amazing. I'd be hesitant to claim significant changes in Virginia's standard-diploma graduation rate without pretty good evidence about real net-migration (and not just unaudited transfer statistics from administrative records). And this raises even more questions about whether any graduation measure could be sufficiently robust to rate individual schools based on improvement in graduation. Or, rather, you could, but the costs of auditing transfer stats might be on the same order of magnitude as the consequences of the statistics.

Update (6/17/06): Via Eduwonk comes the link to a USDOE Inspector General report on South Dakota's graduation-rate method (such as it is).

Listen to this article
Posted in Research on June 8, 2006 10:14 PM |