April 17, 2009

Are GPAs dirty while the SAT-I score is clean?

Before I dive into a minor patch of weeds, some basic issues: Above all else, the vast majority of college and university admission slots are not at selective institutions, so the debate over SAT use for deciding admissions should largely be tangential to policy concerns about postsecondary attainment. This is akin to spending all one's time thinking about the undergraduate curriculum at Harvard or the civic engagement of students at Oberlin. Even if you look at the institutions that require SATs, I suspect the vast majority of slots are at selective institutinos only in the barest sense of rejecting some applicants. But the use of SAT scores is a political hot topic because it stabs into our ideas about meritocracy (as Nicholas Lehmann has written) and also because it has been used for status purposes by institutions or pushed down on institutions (either by state politicians or U.S. News & World Report).

So when Michael Kirst gives us a heads-up that a forthcoming book will argue that SAT I scores have no added predictive value for finishing a degree (not first-year grades, but finishing a degree), I am not surprised one whit. I will wait for the book to see if the evidence is convincing, but I don't think that it will change either the use or the dominant themes in the debate. When SAT scores are used for things it was never intended to and for which there is no documented validity (as a placement tool in college, or for use in judging high schools), you're talking about culture rather than rationality and evidence. A case in point is Chad Aldeman's recent discussion of the SAT debate:

It may or may not be biased against minorities and low-income youth, and kids can be coached on how to improve their score. But, what else do we have that's better, that elite colleges and universities would trust as a replacement? High school GPAs are tarnished by grade inflation and high schools themselves are yoked to reputations. Personal statements are no less coachable than SATs, and extracurricular activities favor the children of parents with time and money. Even worse, none of these things are objective; a student in Abilene, TX cannot be compared to a student from Anchorage, AL on these things. The SAT, on the other hand, is a national test.

Since Aldeman had previously argued that selective institutions should set a basic "we think you can do the work" threshhold and then run a lottery, this is a fascinating defense of a largely defenseless practice. Here's the gist: plenty of research documents that despite all of its problems, a high school GPA is (roughly) at least as good as the SAT in predicting first-year grades. But while many people understand that imperfect data can still be useful (and I suspect that would be Aldeman's defense of the SAT), there is a theme in the excerpt above that appears commonly in debates about admissions standards: GPAs are dirty, SATs are clean.

The argument is almost always laid out the way that Aldeman does: high school GPAs are inconsistent from place to place. Even course titles don't mean the same thing; first-year algebra in one place can be remarkably different from algebra in another. Grades are often a reward of students' putting up with seat-time rather than a demonstration of accomplishments. In contrast, the SAT is a nationally-normed test, and whatever weaknesses it has, it more than makes up for that in its being objective.

One practical problem with this argument is that college is not a set of SAT-like tests. College is messy in all sorts of ways, and for all its flaws, there is something in a high school transcript that has more information about a student than an SAT score. We'll have to wait for the book to come out to see more, but there's a reason why a regular diploma is a more valuable credential than a GED, and the GED is also a nationally-developed test.

A second problem with the "GPA dirty, SAT clean" argument is that the use of the SAT can most harm the chances of students who come from high schools with the lowest graduation rates, schools where one could argue a relatively high GPA says a great deal about relative persistence. As Ted Sizer argued almost a quarter-century ago in Horace's Compromise, suburban schools are filled with the types of classroom treaties that result in grade inflation. But in a school where roughly half of the students never graduate, grades tell you a great deal. They may not tell you if someone who finished algebra I with an A can derive the standard binomial-equation solution (the SAT-I doesn't tell you that, either), but they tell you how much a student has persistence, guts, bureaucratic navigation skills, etc. And if someone from such a school writes an essay (we're talking about selective institutions, again), I suspect it would be far less likely to be coached or professionally edited than the essay of a student in a comfortable suburb. 

As an historian, my professional judgment is that the debate over the SATs has almost nothing to do with whether there is a rational justification for its use in admissions. Instead, the public debate is almost entirely over our ideas of merit, and the framing by one side of the debate as a claim that the high-school GPA is dirty while the SAT is clean is confirmation of that judgment.

Listen to this article
Tags: college admissions tests, high school transcripts
Posted in Higher education on April 17, 2009 9:15 AM |