November 9, 2007

Janie's mother endorses cliff-diving

"You're too young for make-up, Sweetie. Wait 'til you're sixteen."
"I'm not Janie's mother. I don't do this to be mean."
"If those clothes fit any tighter, you would bust out every seam!"
When did my mother slip inside of me?
--- Brenda Sutton, Mama's Hands

For those of you who truly wanted a test of the famous parental Socratic question--"and if Janie jumped off a cliff, would you do it, too?"--we now have a natural experiment. The University of Wisconsin system has committed to the Voluntary System of Accountability, including standardized testing of learning outcomes (hat tip: Zach Blattner).

The Voluntary System of Accountability is a joint effort by the American Association of State Colleges and Universities and the National Association of State Universities and Land-Grant Colleges to respond to pressures for accountability in higher education. Much of it makes sense except for a rather premature (even nuttily premature) inclusion of standardized testing as a proxy for learning outcomes. Only one of the VSA "learning outcomes" tests has been reviewed by the Buros Mental Measurements Yearbook, and the one that was reviewed (Collegiate Assessment of Academic Proficiency) had a fair assessment ($$) from the standpoint of the VSA:

The validity section of the technical manual is quite brief, and the data provided are not particularly encouraging. There is no information with regard to content validity except the suggestion that each institution should conduct its own content validity assessment.... A major concern regarding content validity of the CAAP relates to the coverage of the CAAP to what is taught in college.... There are skills measures that are certainly important to the social sciences, but the work and tools of the social scientist (hypothesis generation and testing, interpretation of statistical data, the search for alternative explanations of findings, etc.) are fundamentally absent from the assessment.

Less than a few weeks after Miami Dade College's internally-developed portfolio system received positive attention from Margaret Spellings, Wisconsin is essentially drinking the Kool-Aid of poorly-constructed standardized testing as a proxy for accountability. When a young friend of mine had to choose between two schools where she was interested in a performing-arts major, she visited the schools, sat in classes, talked with students, and watched performances. Despite Kevin Carey's desire that she and her family use someone else's ranking to make decisions on college, she used the criterion that made sense: see what students are doing in the field she intends to study. AASCU and NASULGC have made a poor choice that risks the waste of millions of dollars poured into the companies that produce those tests and do little to bring serious accountability to higher education.

Listen to this article
Posted in Accountability Frankenstein on November 9, 2007 2:53 PM |