February 20, 2009

Technology and assessment

Education Sector's new report Beyond the Bubble is shorter than I had expected, so I finished it while watching the end of my son's tae kwondo class last night. It looks to be a decent summary of the optimistic side of technology-and-assessment literature. Its tone is, "Yes, we can dramatically change and improve assessment with technology that is either just about to come online or that deserves some investment." And I think that for some things, that's absolutely right: an online/computerized science exam could have color images of tissue slides, videos of animal behavior, and so forth. But, while author Bill Tucker bowed his head in the direction of friendly technoskeptic Larry Cuban, there are some flies in the ointment:

  • Students with disabilities. This is true for pencil-and-paper tests as well, but when you only have black ink, there are a few other issues you don't have to worry about that on-screen designers have to: red-green color blindness, epilepsy and screen movement, etc. The half-page on universal design is good, and any CFP will need to specify (and budget for) disability/accessibility awareness.
  • Code creep. I don't mean internet safety but the fact that programming languages grow up and die. We've gone from perl to python, from HTML to XML, and languages and interfaces will continue to evolve. I wonder how many of the cases pointed to in the report are essentially one-off projects that will die at some point because the platform no longer exists. (Any readers remember Infocom's text games?)
  • Holy Grail syndrome, also known as a belief in "the leap in cognitive science that will allow perfect, automatic scoring of essays is just around the corner." Same with the great and brilliant analysis of hundreds of microstate data that a single student can generate in a simulation environment. I trust colleagues who work in cognitive psychology to do some great things in the next decade, but this seems a bit utopian. Okay, more than a bit.

All of this doesn't say we shouldn't be engaged in using technology, but maybe we should work along two tracks: encourage the fast, frequent, and flexible for now and also invest in the medium- and long-term projects.

There is something that the paper never addresses: intellectual-property rights. Part of the imprisonment of assessment in an oligopoly is the ownership of assessment materials, backed up by the fear of security problems. (Here's reality for you: the day after a state test is given, assume NO security for that test. None. Despite all the laws. Just give that idea up, folks, unless you believe in the tooth fairy, have never heard of BitTorrent, and don't think college students ever cheat.) I am curious what the position of various folks are on open-source assessment. I am not entirely sure what it would consist of, or how it would meet adequate technical standards, but it's tough to argue that despite the testing industry's oligopoly status, we should suddenly think that a brand-new investment will erase both the proprietary rights of the major firms or the start-up threshhold for the creation of commercially-viable products.

Listen to this article
Tags: assessment, technology
Posted in Education policy on February 20, 2009 1:26 PM |