August 23, 2004

What students know (or think they do)

This semester is my first teaching one of the frosh courses in USF's Honors College, and I enjoyed meeting my students this morning, though there was one disconcerting minute or two, when I was trying to encourage discussion of real-life high-stakes examples of trying to discern truth. (The course, titled "Acquisition of Knowledge," is really an epistemology course.) I had discussed the controversies over Michael Bellesiles' now-discredited book (Arming America) that had won a prestigious historians' prize and controversies over what educational research is and is not, but those are fairly arcane matters for the first morning of the semester. Time to let them start a heated debate. So I asked how many had read or heard of the Swift Boat veterans ads or the controversy over Bush's National Guard service. The group looked sheepish while a few raised their hands. I was flummoxed. Oh, well. So much for students' keeping up with current events. Time to try another tack, I figured. I asked how many remembered the O.J. Simpson trial. I don't know why, really—it was the first thing that came to mind in terms of pop culture and discerning the truth.

Every hand went up.

Listen to this article Listen to this article
Posted in Teaching at 7:07 PM (Permalink) |

August 20, 2004

Comments from NIH!

The NIH program director sent me the comments of the study section on the net-flow grant proposal. On the whole, they were very encouraging, in two ways. First, they generally agreed that the approach I had in January (when I submitted the proposal) was sound and interesting, and there was nothing inherently problematic. Second, their criticisms were all about mildness—it seemed a mild innovation, and with a few mild weaknesses. I had applied for an R03 grant, which is for pilot projects with some innovative promise, so obviously that plays an important role in the evaluation of projects within the study group.

What to do? With NIH, I can revise and resubmit two times, so I will. My first instinct is to change the followng:

  • Update the methods section to what I have now
  • Address whether or how I'd estimate the net flows for specific schools, and how this might be accomplished. (This was a specific weakness addressed by the study-group comments.) If I decide it's impractical at this point, explain why the research is still valuable without it. (Why is it okay to look at districts? Many small districts only have one high school. So then I can discuss large-district issues.) Or discuss the hope that modeling retention rates might allow the choice of retention rates for a high school based on an aggregate figure (which is often available). This last will be acceptable for demographers, who often must choose a set of model mortality rates when estimating population parameters or projecting populations.
  • Change the focus of the proposal to something that emphasizes the innovation and the immediate intellectual results, which may compensate for the innovation. The historical materials here might be very useful, since a number of schools have either age-grade tables (Delaware) or have both age tables and retention data for grades (Boston, from 1884 through the 1950s). Thus, there might be some real hope for establishing relationships between the age-derived estimates and grade-derived estimates.

The trick, I suspect, is not to overpromise. I can't promise to look at all districts for multiple years in Texas, Florida, North Carolina, and Massachusetts, make school-level estimates for a large state, estimate many series of data from historical records, and also conduct the analysis I've proposed to NSF. I've embarked on a potentially long-term series of research projects stemming from this method. So the question is how to frame it as a good pilot study.

I sometimes plan articles and other pieces from the reaction I want to get from a reader. Usually it's "I hadn't thought of that and, with a few seconds' reflection, it makes a lot of sense." Here, I need something different: "Wow, he's continued to work on this project, he's addressed our concerns, and I'd give my next sabbatical to see the results." Well, not really the last one, but I do want to give them the impression that this is an incredibly promising idea.

Listen to this article Listen to this article
Posted in Research at 5:50 AM (Permalink) |

August 12, 2004

Grant proposal away!

The proposal to NSF based on the net-flow estimates is now submitted. We'll see what happens. (I asked for a primary review by the Sociology program.) Thanks to a staff member in my college, I now know how to get the parallel data (not by grade but in toto) to estimate net flows for graduate programs at USF, divided into advanced and other (mostly masters) programs. Hmmn...

Listen to this article Listen to this article
Posted in Research at 6:52 PM (Permalink) |

August 11, 2004

More net-flow SAS files

More test SAS files for state-level net flow from North Carolina in 2000-01, Massachusetts in 2000-01, and Massachusetts from the mid-1990s. As I've seen before I got the algorithm right, there's a significant downtick for Mass. 2000-01 for 10th grade (or a higher-magnitude negative flow). You need several years of data after that point to know precisely how to interpret that datum. Is it a response to the implementation of either the high-stakes system with MCAS or the exit exam in Massachusetts? Hard to tell, exactly. Having seen anecdotal evidence of school-system responses to high-stakes testing, it might be evidence of a massive burp of triage/purging or of a student response. It's the broader pattern that's really necessary to tease things out.

One methods issue: there are two ways to think of smoothing data for small systems. One is to take annual data and smooth it. The other is to estimate two-year net-flows. Have to work on appropriate SAS statements for that.

Listen to this article Listen to this article
Posted in Research at 5:35 AM (Permalink) |