June 17, 2007

A natural experiment with third-grade retention in Florida

Thus far, in the various news stories about the errors in Florida's 2006 third-grade reading FCAT scores, no one has thought of the obvious study that should flow from it: a comparison between students retained in 2005 based on FCAT reading scores and students who were promoted but whose corrected scores in 2006 would have retained them.

My impression is that students who would probably have been retained without the incorrect scoring will not be held back in fourth grade. We thus have a fascinating contrast: two cohorts of students with low reading scores in third grade whose retention decision was a matter of sheer accident.

Not only that, but Florida's environment is different from that of previous studies, where the comparison really was a question of retention and praying that the student did better, on the one hand, and promotion and praying that the student did better, on the other hand. For years, former Governor Jeb Bush claimed that the placement of reading coaches and massive technical assistance would make third-grade retention policies different: targeted, effective, and helpful. We can test this claim now!

The study design would be fairly obvious, I think: match third-graders in the two cohorts using the standard propensity-score method (include age!!) and then look at measures that would not be biased by the simple fact of retention. FCAT scale scores in fourth grade would be inappropriate because you're comparing promoted students to students who had one more year of exposure to the intermediate-grade curriculum. I am hesitant to suggest using the FCAT's supposedly vertical scale (called a "developmental score"), but in terms of achievement it might be best, unless one can identify individual children in a large district and assess them in the same exams a year apart. If you're a graduate student who cannot engage in data gathering over 15 months, I would choose the developmental score and choose other indicators as well.

I don't have the time to do this study, but someone should: go to it, folks!

Listen to this article
Posted in Education policy on June 17, 2007 1:08 AM |