March 31, 2008

Jim Anderson retrospective, part 1

Last Tuesday in New York, a roundtable panel presented a 20-year retrospective on Jim Anderson's The Education of Blacks in the South, 1865-1935, chaired by Joy Williamson (U. Washington). In this entry, I'll summarize some of the perspectives of the panelists. In the next entry (tomorrow, I hope), I'll present my personal engagement with the book.

Rubén Donato (University of Colorado) noted that students who read the book for the first time generally have bimodal responses. Many of them react to Anderson's argument as if they've never heard such a radical idea. Donato called these the "Oh, wow" reactions. But he also said a slice of students react with a little more cynicism: "Of course," because of their experiences or their friends' and families' experiences with schooling.

Donato also noted that most readers of Anderson's book come to it as graduate students, and he is worried that its arguments rarely filter into undergraduate history courses. While I think he's wrong in terms of Anderson's broader arguments (I'll explain this in my more personal post), he is correct about the readership for the book.

John Rury (Kansas) noted first that Anderson's book represents the maturation of revisionist education historiography. The book is sophisticated, nuanced, and detailed, and it carries an argument more successfully than almost any other challenging/radical history of education published in the prior 20 years. (This last weekend, Anderson and a whole bunch of new scholars were at Penn for the 40th-year anniversary of Michael Katz's Irony of Early School Reform/non-Festschrift conference.)

Rury also noted that for much of the book, Anderson was writing an elite history, a "reform by imposition" story that focused on the network of foundations active in the late 19th and early 20th century. While he shifted later in the book to discuss community efforts (see below on Vanessa Siddle Walker's comments), Anderson's book was remarkable in its focus.

Finally (at least in my notes), Rury noted that Anderson's book left a huge agenda in its wake, and scholars have been either riding that wake or trying to catch up to it since.

Eileen Tamura (Hawaii) focused on the part of the book Rury avoided, where Anderson discusses human agency. Tamura pointed out that while the elites involved in foundation work discussed how education could tame the political and economic aspirations of African Americans, Black communities were willing to tax themselves a second time through voluntary contributions to raise buildings and pay for operating expenses of schools. In this way, Tamura argues, Anderson pointed out how there were multiple discourses, with the local discourse undercutting the foundations' efforts to impose a tame sort of education. Tamura suggested that one of the chunks on the agenda left by the book was the way that cultural capital works in networks (my awkwardness not hers).

Vanessa Siddle Walker (Emory) also focused on the actions of Black communities and community members, and Siddle Walker focused on several historiographical points: First, Anderson identified the undercurrents in a way that would not have been possible with a single storyline. Her language was that Anderson identified a "story within a story" rather than just recycling old ideas. Second, Siddle Walker pointed out how Anderson was patient with his work ("lingered with an idea"), something you can identify if you look at the 42 newspaper series, 63 government publication series, and 30+ pages of bibliographic references in the book, as well as the acknowledgments that note the broad professional network Anderson used in working on the book.

Third, Siddle Walker argued that Anderson's work showed the importance of believing in the value of community perspective. She argued that this is the ethic of good oral historians, and couched a warning as well: often enough, historians are confronted with relatively little response, which does not mean that there isn't a story so much as the fact that the historian may not have gained entree to the community's trust.

Finally, Siddle Walker argued that Anderson's book made the story (stories?) accessible, readable and free of jargon that some others indulge in. (Was she referring here to Aronowitz or Giroux?)

Anderson then responded, gave credit to David Tyack for raising questions he had not considered when looking at Southern education after the Civil War, and then made four general points. First, he said that he should have paid more attention to the members of those communities who were still living in the 1970s and 1980s. He said that his own mother read the book and then told him, "You should have talked to me before you wrote it." If he had the chance to do it over, he said, he'd include oral history (which he did not).

Second, Anderson explained the background behind his book's not receiving the History of Education Society Outstanding Book Award for that year: he was on the award committee, and when his book was mentioned, he said that it wasn't that worthy a book and he'd have to recuse himself anyway. So the committee chose another book (I forget which...). The larger point here is that Anderson had no idea how positive the reception would be over time.

Third, Anderson was skeptical that his book was as definitive as some have implied. It was a broad overview, he said, and there is so much more to be done and so much that has been since, from Sieglinde Lim's work on Chinese immigrants in the Mississippi Delta to Siddle Walker's work, Adam Fairclough, David Cecelski, and so forth.

Finally, Anderson pointed out that Rury was essentially correct in terms of the origins of the book as a top-down perspective, and only in the middle of the work did he discover community culture, and then had to revise his views.

The discussion afterwards was mostly warm and fuzzy recollections, but there was one sharp question by Tyrone Freeman, asking Anderson's views on today's education foundations, from Gates to Broad, and whether they, too, were trying to impose a specific view that might be as pernicious as what he described. Anderson demurred, saying that the key to his work was discovering that the public representation of white philanthropists' work was dramatically different from the private work that was detailed in their papers. Since we don't know that private conversation going on today, he said, he couldn't comment.

For the most part, I sat back, took sketchy notes, and just enjoyed the conversation. Now, don't you wish someone was taking better notes, or that you were there yourself?

Listen to this article Listen to this article
Posted in History at 10:44 PM (Permalink) |

Tacit knowledge and the AERA program hustle

Eduwonkette has commented on the heterogeneous quality of sessions at the American Educational Research Association annual meeting, quoted someone saying it was a tenure hustle, and suggested that the IES-funded Society for Research on Educational Effectiveness is a rival to AERA. (Oh, yes, and her friend skoolboy is right in recommending Topaz Thai.) I've commented on the oversized aspect of the conference, but I waited until after AERA to gripe about one feature of AERA that is fundamentally inequitable:

AERA's reviewing system provides structured advantages to groups of researchers who collaborate on submitted proposals. Researchers from disciplines with solitary traditions face inherent disadvantages in such a system.

Because of its size, AERA has for years rationed session slots to divisions and SIGS by the number of submissions in prior meetings. (I don't know the formula, but I suspect it includes the number of prior submissions, number of prior panels, total membership in the division/SIG, phase of the moon at AERA two years ago, together with a cosine function tied to the inverse proportional pressure wave created by a size-10 shoe dropped from the AERA executive director's office to the street below.) So there is a huge incentive for division leaders to encourage large numbers of submissions, which produces a low acceptance rate.

Theoretically, that should mean a better overall quality of sessions, but it doesn't turn out that way, because with a large number of submissions (which are heterogeneous in quality), you also need a large number of reviewers, and program folks literally go begging for reviewers in the second half of the year, after submissions are in.

If you think the quality of reviewing is significantly more consistent than the quality of submissions, I have a swamp or bridge to sell you. If you're looking for reviewers, you don't have much of a choice. And AERA's reviewing system has a one-size-fits-all quantitative rating scheme (rubric), regardless of the methodological or epistemological traditions of the scholar. "Data Sources" are irrelevant to the philosopher, but it's a required criterion for all reviews. And the quality of feedback varies as well. Here are the (quite positive) reviews my coauthor and I received for the proposal that was accepted:

Choice of Problem/Topic    4 / 5
Theoretical Framework    4 / 5
Methods    4 / 5
Data Sources    4 / 5
Conclusions/Interpretations    4 / 5
Quality of Writing/Organization    5 / 5
Contribution to Field    4 / 5
Membership Appeal    4 / 5
Would You Attend This Session?    4 / 5
Overall Recommendation    5 / 5

Comments to the Author
This is a well-written, clear, and very focused proposal. It offers new perspectives on the oft-talked about teacher shortage problem, providing new evidence from data on re-entry into the profession and analyzing entry and exit by age. The data, methods and conclusions all appear to be solid. However, I do feel that more critical issues related to teacher shortages emerge if we consider the distribution of teacher shortages--for example, shortages of teachers willing to teach in urban areas, and subject-specific teacher shortages in math and science. Nevertheless, this paper makes an important contribution to the overall teacher shortage debate.

Choice of Problem/Topic    5 / 5
Theoretical Framework    4 / 5
Methods    5 / 5
Data Sources    5 / 5
Conclusions/Interpretations    5 / 5
Quality of Writing/Organization    4 / 5
Contribution to Field    4 / 5
Membership Appeal    4 / 5
Would You Attend This Session?    3 / 5
Overall Recommendation    4 / 5

Comments to the Author
A strong, well-designed proposal on a clearly important topic.

I'm not sure if the second reviewer was exhausted from 17 prior reviews (my hats off to her or his service in that case) or just had little to say, but I've had reviews that are all over the map in terms of ratings and amount/quality/relevance of comments. I pity the poor program volunteer who has to sort the reviews and figure out what to do with submissions that receive disparate splits (4s and 5s from one reviewer, 1s and 2s from another, with either or both reviews having either many or no comments). But there's one conclusion I take as a member of AERA who submits proposals:

Whether your AERA proposal is accepted is substantially a game of craps. This conclusion doesn't mean that horrid proposals are accepted but that plenty of very decent proposals are shot down because there is no way to create a consistent system of reviewing, and there is probably no way to predict which good proposals are accepted and which good proposals are rejected. (I wonder if anyone has asked permission to look at a set of proposal ratings to calculate reliability...)

I suppose I could make money by having a side bet system (but I don't live in Vegas or Atlantic City), but there's a more pragmatic consequence that some researchers use to increase their odds of being placed on the program (often a requirement for getting travel funds from your institution): Agree with colleagues or graduate students to collaborate on submissions. The more submissions your name is attached to (either as main presenter or coauthor), the greater your chances of having a proposal accepted and thus being on the program (see the "tenure hustle" comment above).

This consequence is obvious to some but the type of tacit knowledge that isn't told to others as part of their grad-school socialization. Many of us work in relatively solitary fields (philosophy, history, etc.), where being on the margins of someone else's work doesn't seem to deserve the intellectual credit of being a coauthor. So someone coming from that field would probably not be told by her or his advisor that to maximize one's chances of appearing on an AERA program, you need to network and increase the number of submissions your name is attached to. In my subfield, the usual advice is to collaborate with others to propose a coherent panel, which is supposed to have a higher chance of acceptance because of the higher quality and relevance for the complete panel. That works in some conferences where there are advantages to complete panels, but in most divisions at AERA, that is unlikely to be true.

I'm not griping about the system this year, since I did the logical thing and both collaborated with a colleague where we could ethically submit two proposals (one emphasizing my side of the work and another emphasizing his side of the work) and also agreed to be put on as a panelist by a third colleague, and in another role by a fourth person. The proposal where I was the presenter happened to get accepted. Was that because my proposal was the most qualified? Not likely. Just having several proposals with my name on it helped, and the odds worked in my favor. But if you haven't learned this and your single proposal to AERA was rejected, you now know what you need to do: get your name on multiple submissions to the next AERA program. The submission deadline is in the summer, so it's time now to start networking for next year. Don't be unethical: network where you really can be a contributor. But if your tenure depends on AERA appearances, it's (sadly) in your interest to play this game.

Ultimately, I suppose AERA could be overwhelmed if groups of researchers decide they'll band together in 100-person units, each of whom submits 2 paper proposals as a primary presenter and 99 coauthors. That is probably not likely to happen, but the ad absurdum thought experiment should make my point clear: increased numbers of submissions do not inherently improve the quality of accepted panels at AERA, even with lower acceptance rates, and those who work in large research groups have an inherent advantage in a metastasized conference like AERA.

There are some potential fixes I can imagine:

  • Divide single-authored proposals from multiple-authored proposals in the reviewing process, so single-authored proposals are compared only to single-authored proposals, and likewise with multiple-authored proposals.
  • Have some metric of reviewer trust within AERA. No, I have no clue how this could be done feasibly.
  • Subdivide the reviewing process so program volunteers only have a limited number of submissions to work with and can read and filter the reviews without going bonkers.

There are two reasons why AERA should care about this problem. First, it's an issue of equity. AERA's annual meeting is already designed in a way that benefit faculty who work in better-funded institutions who can support travel to and several nights' stay in expensive hotels in New York, San Diego, Chicago, etc., and those are the same faculty who are likely to have research groups (i.e., grad students) that foster a multiple-submission system and increase odds of appearing on the program.

Second, it's also an issue of quality with the program. AERA is the best evidence I know that a high rejection rate does not increase one's program quality. That rejection rate is only meaningful if it reliably includes stronger proposals and filters out weaker proposals. Apart from disciplinary and other differences on what you or I may think are stronger or weaker proposals, I just don't think the system is working at AERA. That doesn't mean I'm going to abandon AERA entirely (I've reviewed dozens of proposals over the years, regardless of my own participation), but I am one of those on the margins of AERA in large part because the current annual-meeting structure is dysfunctional.

Listen to this article Listen to this article
Posted in Reading at 2:46 PM (Permalink) |

March 26, 2008

Biology is constructed, but there's only one version of history

Today the Florida Senate PreK-12 committee approved Ronda Storms's bill (SB 2692) that would undercut the new state science standards that include evolution as the basic model of biology. It's called the "Academic Freedom Act," but it only applies to biology teachers who want to teach Intelligent Design (or the Flying Spaghetti Monster theory of creation). If you're a history teacher, you're on your own with a set of rigid legal prescriptions that tell you what history is:

American history shall be viewed as factual, not as constructed, shall be viewed as knowable, teachable, and testable, and shall be defined as the creation of a new nation based largely on the universal principles stated in the Declaration of Independence.

So someone who thinks that the Declaration of Independence was undercut by a more conservative Constitution is forbidden from teaching that. Since history is factual, you can't debate arguments about whether the founders were more influenced by their views of Enlightenment debates in Europe, Renaissance political theories, or classical history. And you definitely can't read David W. Noble's Historians against History (1965), which critiques precisely this claim that the history of the U.S. represents an entirely fresh start, a dramatic break from the past.

To mash up W.C. Fields and Elbert Hubbard, biology is a rich stew of diverse views, but history is one damned fact after another.

Listen to this article Listen to this article
Posted in Education policy at 11:44 PM (Permalink) |, students' intellectual property, and fair use

Eric Goldman has the latest news and commentary on the high school students' lawsuit (and the suit's dismissal) against having to submit papers to (hat tip). It's a fascinating and complicated issue, and Goldman's discussion explains at least some of the tangles involved (though I wouldn't be surprised if there were more).

Listen to this article Listen to this article
Posted in Academic freedom at 8:44 AM (Permalink) |

March 25, 2008

AERA brief note

I'm in the Delta terminal of JFK, waiting to go home to Tampa. Presented. Listened. Laughed. Bought things in both the AERA exhibit hall and also the Juliiard School bookstore (which was having a 30%-off sale on a bunch of CDs). I will be blogging later this week on a NYC nutty policy and on the 20th retrospective session on Jim Anderson's 1988 book, The Education of Blacks in the South, 1865-1935.

Listen to this article Listen to this article
Posted in Education policy at 6:29 PM (Permalink) |

March 23, 2008

In Mid-town Manhattan

I'm at the corner of Broadway and 54th right now, with relatively minimal fuss except that in the rush to pack last night, I grabbed my daughter's jacket instead of mine, and I installed a package to my Linux laptop without pinning, so a whole bunch of things disappeared and I had to wipe the entire system clean. As you can see, that didn't turn out to be horribly onerous, and I even figured out how to get a new Firefox add-on to meet and greet my blog back-end.

I'll only be here for two days or so of AERA, but this is going to beat last year, when I was at AERA for only one day, and it turned out to be The Day of Sideways Sleet (aka The Day of Frozen Humiliation and Pain). But it's sunny right now, and well into the 40s, so I'm happy on that end.

Oh, yes, and a colleague and I have a paper to present tomorrow. And colleague to meet.

Listen to this article Listen to this article
Posted in The academic life at 4:15 PM (Permalink) |

March 22, 2008

I'm feeling better... I think I'm going to go for a walk

I know it's been a super-busy week, but I was taken aback when one of the searches leading people to this website turned out to be "Dorn of the Dead." Lal Bihari and the Association of Dead People aside (they do great work and won the 2003 Ig Nobel prize for peace), I'm still here, am still recognized as alive by my family and coworkers, am not about to talk anyone to death about eating their brains, and just need to nail down a few things before I head off to New York for the American Educational Research Association annual meeting.

Listen to this article Listen to this article
Posted in The academic life at 11:15 AM (Permalink) |

March 20, 2008

Who ever expected Florida legislators to be enmeshed in ...

Following a few weeks after revelations that a small university up the road from me hired a state legislator as a lecturer for close to six figures, there's a story in IHE with the title A State Senator's Sweet Deal, about one of the universities in our state capitol.

This type of news item appears occasionally in almost any state, but my concern is that if a proposal to gut the powers of our state governing board goes forward, we'll see a lot more of it.

Listen to this article Listen to this article
Posted in Higher education at 9:06 AM (Permalink) |

March 19, 2008

Life The tenure process can work out

She may have been doubting it at various points, but many of who have read her blog over the past few years never did: Profgrrrrl got tenure.

Listen to this article Listen to this article
Posted in The academic life at 1:18 PM (Permalink) |

"Differentiated accountability"

Alexander Russo links to news coverage of the Margaret Spellings announcement yesterday that maybe not all AYP failures are the same. Here's some blog coverage:

Spellings went to growth pilots, waivers (or turning the other cheek) to allow tutoring before choice, and now differing judgments on failure to meet AYP after others talked about the ideas for years. I think Spellings is just channeling Adlai Stevenson, who once quipped that leadership is seeing where the crowd is heading and getting in front of it.

(Does anyone know the exact wording or source for that?)

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 10:03 AM (Permalink) |

Florida ed policy and politics

The legislative session is in full swing (or a more colorful noun), and a bunch of things are in the air either in Tallahassee or elsewhere:

1. Both houses of the state legislature are considering bills to change the role of state testing (FCAT), either by adding other information to the labeling of high schools (the senate's approach) or by a compromise bill that discourages test-prep and sets more specific grade-level standards (the proposal in the house).

2. The ACLU sues Palm Beach County for its low high school graduation. Superintendent Art Johnson suggests it's the state's fault for not providing enough money (scroll down for "But the superintendent..."). (Disclosure: A 2006 paper of mine is mentioned in both stories.)

3. Something that wasn't covered in my local papers in January: Holmes County administrators have banned students from displaying anything related to gay pride. The ACLU of Florida sued. I suspect this one's a no-brainer in a bench trial: in the majority opinion in Morse v. Frederick, Chief Justice Roberts made a distinction between what he thought of as the political speech of Tinker and the display of "Bong Hits 4 Jesus."

The only interest the Court discerned underlying the school's actions [in Tinker] was the "mere desire to avoid the discomfort and unpleasantness that always accompany an unpopular viewpoint," or "an urgent wish to avoid the controversy which might result from the expression." Tinker, 393 U. S., at 509, 510. That interest was not enough to justify banning "a silent, passive expression of opinion, unaccompanied by any disorder or disturbance." Id., at 508.

I think that reasoning clearly applies in this case.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 9:35 AM (Permalink) |

When you make speeches the issue...

I don't generally talk about electoral politics in this blog, but yesterday's speech by Barack Obama strikes me as historic, whether or not Obama wins the Democratic party's nomination or the general election. Historians are often wrong in their predictions (we earn a rear-view mirror with that degree, not a crystal ball), but this is about a judgment involving historical perspective, and I'm reasonably comfortable in that.

Moreover, I think that anyone who thought that the Jeremiah Wright controversy would inevitably and permanently damage the Obama campaign missed the way it created an opportunity for the best orator on the American political stage. Put bluntly, here's how the conversation went, in meta-narrative style:

Look at what your pastor said!

He was wrong.

You haven't denounced him enough.

He's no longer connected with my campaign.

Explain why you haven't left your congregation!

Okay, if you really want me to talk some more...

If there's one thing that his rivals don't want to demand of Barack Obama, it's that he give a speech.

Listen to this article Listen to this article
Posted in History at 9:11 AM (Permalink) |

March 17, 2008

Complex object creation tools: review needed

With the recent release of new versions for both Omeka and Sophie, I'd love to see some comparative review from both institutional users (e.g., the perspective of someone in charge of a project team) and also individual users (e.g., teachers trying to create content for specific courses or modules).

I'm not saying I'm going to (no time!), but I'd love to see the reviews from both perspectives.  Oh, yes, and while we're at it, how about a review of Inform 7?

Listen to this article Listen to this article
Posted in Higher education at 9:05 PM (Permalink) |

Grad rates of the fanciful and research kind

While Erin Dillon and Inside Higher Ed are creating mash-ups of the NCAA hoops brackets and college graduation rates, I'm working on reframing the CPI when I can't handle fighting fires any more, except this evening I'm working with the 2900 counties in the U.S. with upper-grades enrollments reported for 2001-02 through 2005-06 and regular diplomas reported for 2002-03 through 2004-05 (reported with the 2003-04 through 2005-06 enrollment data, thanks to CCD lag). That omits 249 counties (about 8%), including all of Alabama, Hawaii, New York, and Wisconsin, whose county reporting for the Common Core of Data had a gap for at least one year and variable.

predicted v estimate Swanson counties.JPG

r2 = .84 for both the log equation (not shown) and the transformation back to predicted vs. estimated smoothed CPIs. Ignore that incredible outlier in the upper right-hand corner. I'll hunt down that outlier, remove it, and redo, and it'll end up almost the same except for a slightly lower r2 and a more evenly-distributed graph. Two points here:

  • Again, this is not to suggest one can accurately estimate graduation rates from the cross-sectional formula I'm working with. Rather, the biases one should be worried about with the CPI are mostly captured in any biases that show up in the cross-sectional data.
  • This graph was produced by the open-source R Project for Statistical Computing package. Despite what others have reported, the learning curve for R is not that steep. If you have to pay for SAS or SPSS, use moderate sized data sets, and expect to work in quantitative fields for the next few decades, it'll probably pay off in the long term to switch now.

I did not sign up to be a firefighter

... so why do I feel like I've spent the last 5 hours playing the role?

Oh, yeah: my university is back from spring break; the first department peer review committee meeting (annual evaluations) is tomorrow morning; I'm out of town most of next week.

I want a week to work all by myself, and then I'll take the week I scheduled. Aiiiieeee!

Listen to this article Listen to this article
Posted in The academic life at 12:58 PM (Permalink) |

Busy next week

March 24, 2008
Teacher Attrition: Age-Specific Transition Rates (Sherman Dorn and Stephen Provasnik)
In panel Sociological Perspectives on Teachers and Teaching
American Educational Research Association (SIG - Sociology of Education)
New York Marriott Marquis Times Square, Soho Complex, Soho Room, 7th Floor

March 29, 2008
Defending Effective Accountability and Assessment Practices
Joint AFT/NEA Higher Education Conference
Hilton Washington

Ora James Bouey, Health Sciences, Stony Brook University, SUNY
John Hammang, Director, Special Projects, American Association of State Colleges and Universities,
Debra Humphreys, VP Communications and Public Affairs, American Association of Colleges and Universities
Sherman Dorn

Listen to this article Listen to this article
Posted in The academic life at 9:41 AM (Permalink) |

March 13, 2008

They're for the children... or the adults. Well, which one is it, and where's the scorecard when you need one?

There's a deeper dynamic in urban school politics that reporters are missing, even the work of good beat professionals such as those who work for the St. Petersburg Times: When they talk about African American politicians who support vouchers, whether it's members of the Florida legislature, Barack Obama, or David Paterson, they often forget that in communities that are residentially segregated, in charter and private schools that serve primarily African American student populations, the proprietors of and teachers will themselves be more likely to be African American than other private-school educators... or public-school educators. So the perceived interests of adults and children are once again blurred, though no one aware of education history should be surprised that opportunities for and perspectives of African American educators are part of the issue. I also wonder how many who have started charter schools or voucher schools in Florida are African American teachers who retired from public schools. Oh, the headache: will someone come and tell me once and for all who is For The Children?

Teasing of DFER's resident blogger aside, I suspect I'm not the first one to think about this connection (and if a commenter points to a relevant article, I'll update this with a link). But it's one more point against the more simplistic efforts to tally up who's on which side in the adults v. kids wrestling match, when you're not sure who's what's side's hidden side.

Listen to this article Listen to this article
Posted in Education policy at 10:57 PM (Permalink) |

A rescue for Swanson's CPI?

(Note: I've changed a few things in the first graph, and someone pointed out to me that I had fallen prey to one of the many Excel glitches, but I'll show the changes below...)

I've written informally here on graduation measures, expressing my concern about Chris Swanson's Cumulative Promotion Index, known among the grad-rate numerati as the CPI (e.g., 4/16/06, 6/21/06, 6/12/07). One of the CPI's weaknesses is its reliance on the annual numbers reported to the Common Core of Data. Another is the assumption that the ratio of enrollment in 10th grade in 2008 to 9th grade enrollment in 2007 is a meaningful gauge of cohort retention/promotion. In 2005, Rob Warren explained (PDF, iPaper) why that is problematic: 9th grade retention and student transfers pollute the measure. Transfers pollute all of the components of the CPI (net in-transfers artificially inflate CPI), but 9th grade retention is particularly problematic (deflating CPI).

Smoothing down the 9th-grade bump and year-to-year jiggles

I've been returning to the issue of measuring attainment (my Holy Grail, I suppose), and there are a few ways I've thought to improve on the CPI. Two obvious ones are to smooth the data and remove 9th grade as an issue:

  • Use three years of enrollment and diploma data, to smooth over single-year bumps and reporting problems.
  • Start in 8th grade and use a two-year 10th-to-8th grade enrollment ratio as the first term in CPI.

I tried that on state-level data from the beginning of the Common Core of Data to the latest year (1986-2005), and the effect of smoothing is what I had hoped: the measures of central tendency for each state are similar, but the "bumpiness" of the data is dramatically reduced. And if one looks at data within each state, over the entire time series state medians for the Swanson CPI is highly correlated to state medians for the smoothed measure (r2=.92, N=51). By starting with 8th grade, state medians for CPI tend to rise between 3% and 8%. That's not surprising.

Reframing CPI

Then I had another thought: what if we looked at grade level not as a proxy for the time in high school but as a set of gateways, requirements to meet on the path to graduation? Then the concepts behind the CPI terms could be thought of as a standard probability problem. Through a few razzle-dazzle maneuvers, I snatched some cross-sectional data from CCD, took the natural log of everything, and tossed it through regression to see if the cross-sectional data could predict the smoothed CPI, at least with the state-level data. Here's the result, with the regression prediction on the X axis and the smoothed, skip-9th-grade CPI estimate on the Y axis):


The previous graph has the same points but a more ambiguous indication of r2. For the geeks, I ran the regression on the logs of everything (there's a clear reason tied to the background for all this), and the top r2 refers to that regression. But you can also look at the translation back into percentages/ratios, and the second r2 is for the plotted graph. In this case, they're virtually identical. N=867 (17 x 51). Pretty snazzy, eh? No, I'm not releasing the details. Not until it's been highly vetted...

And I'm not going to break out the champagne, either (especially after a bit of embarrassment with the Excel glitch). At the lower end of the range for states, the prediction underestimates the smoothed CPI, and there are no guarantees how it'll perform at that low range. The different points for each "year" are not truly independent within a state, since we're working with multiple years of data (closely related to a moving average). And, as noted above, student migration can easily bias CPI, leading to CPIs above 100% with substantial migration.

States don't hit extremely low levels of graduation or high levels of migration/transfers, but districts do. I took California districts with average enrollment in grades 8-12 over several recent years of at least 3,333, removed a few elementary- or high-school-only districts (California has that odd combination) as well as others with some data anomalies, and ended up with 127 districts that make up 59% of the 8th-12th enrollment over the years in question earlier this decade. Snagged the same cross-sectional elements from the CCD.  How well does that idea work (the following graph was before fixing the spreadsheet formula error)?


That was clearly not nearly as nice as the state-level picture. A few things are important to note, here: larger aggregations tend to look different in any statistical analysis, and you'll see here a broader range of smoothed CPI predictions and estimates, including the dreaded and improbable over-100% measure.

But there was an error in the spreadsheet (caused by copying a column instead of a formula). Here's the new graph:



Much better, no? Again, I've noted both the r2 for the regression model and r2 for the translated figure, a little lower than with states and not quite as close. As before, smaller entities are going to have broader variation, but I'm a little more encouraged. Yes, I have a few ideas on how to attack that over-100% CPI, but I think that's enough for saving the world today. I have chaffeuring and journal editing to do in the next few hours... (the driving is done but I'll see if I have a bit more energy tonight)

One last point: The implications of this are a bit subtle, apart from the utility of smoothing the data with multiple years and starting with 8th grade. The ability to connect a few cross-sectional data elements tightly to the synthetic CPI does not mean that CPI is without flaws but rather that the roots of any bias in CPI are at least parallel to and probably identical to the biases in the cross-sectional data. If you slide up and down the potential biases in the cross-sectional elements, you also slide up and down the biases for the CPI.

Listen to this article Listen to this article
Posted in Education policy at 3:11 PM (Permalink) |

March 11, 2008

Playing chicken with a Mack truck and living to tell the tale

This year I've occasionally likened Florida politics and policymaking to playing chicken with a Mack truck. Usually, the truck wins. But twice in the past few months, the minority leader in the Florida House (Dan Gelber), has won the game, most recently by getting out in front of a mail-in redo of the Democratic presidential preference primary. While the Miami Herald story doesn't mention Gelber by name, there is no doubt that Gelber's mid-February blog entry was the first trial balloon.

A disclosure here: I've worked with the House Democratic caucus a few times in the past on education issues. I think I've observed Gelber enough to see a connection between this presidential election politics story and state education policy, since I've seen two other times when Gelber's made an impact by being out front on an issue, and in the other two cases, it was education. One was a presentation to the Florida Board of Education that the Department of Education responded to with a broad statement about accountability philosophy (not yet up on the FCAT external advisory group webpage), and the other was on end-of-course exams as a replacement for the high school FCATs. In the first case, the statement was the most reasonable thing I've seen out of the department on accountability in the 12 years I've been in Florida. On the end-of-course exams, he may have jumped in front of a crowd heading down the street anyway (was it Adlai Stevenson who defined that as political leadership?), but his timing was dead on. In both cases, he took a few risks that I haven't seen from other Democratic caucus leaders in the past decade. I suspect he's had a few run-ins with that truck which I haven't observed, but evidently he keeps getting up and going on.

Gelber is running for a state senate seat being vacated in the middle of the term, so if he keeps that seat, he could be in the state senate for a decade (two years for the remainder of the current term plus two full terms). If he keeps avoiding the Mack truck, he could have a long-term impact on the state's education policy.

Update: Splat. That's egg on my face, if not the Mack truck. Now you all know why I'm keeping my day job...

Listen to this article Listen to this article
Posted in Education policy at 7:37 PM (Permalink) |

Defending Effective Accountability and Assessment Practices

Saturday, March 29, 2008
Hilton Washington

Defending Effective Accountability and Assessment Practices is the title of the session I'm a participant in at the NEA/AFT Higher Education Joint Conference.

From what I understand, the tentatively-slated participants include staff members of two institutional associations as well as us faculty. As soon as I have permission to post those names, I'll do that.

Listen to this article Listen to this article
Posted in Accountability Frankenstein at 12:18 PM (Permalink) |

March 10, 2008

Essayist as Puck?

If we shadows have offended,
Think but this, and all is mended,
That you have but slumber'd here
While these visions did appear.
-Puck, A Midsummer Night's Dream, V, i

In trying to explain Why I Write These Columns, Stanley Fish argues that his goal as an essayist is to probe the logic of an issue, and that he can remain agnostic on the larger issue while probing that logic. Thus, he says he could be atheist while criticizing Richard Dawkins et al., against identity politics while grasping one possible rationale, against the Iraq war while seeing advantages for John McCain in a McCain-Obama matchup, etc.:

[W]ere I to address myself to those matters, I would be entering the realm of moral and political (as opposed to analytical) judgment.

Fish has a point here: One can talk about aspects of an issue without taking a position on other aspects. On the other hand, I am surprised with how he did so. Fish's tone came across as whiny, or that's how I read it. The indirection of the first few sentences nailed it for me, with my comments in brackets:

Every once in a while [honestly, Fish, I don't care how often you do this] I feel that [glad to know you have feelings, but could you get to the point?]it might be helpful to readers if I explained [does anyone else think this phrase talks down to the reader?] what it is I am trying to do in these columns [Ah: we finally get to the point, which is that you're going to tell, not show]. It is easier to state the negative [you know that you should be stating the positive instead]: For the most part, it is not my purpose in this space to urge positions, or come down on one side or the other of a controversial question ["I'm not going to carry any reader's water"].

This is the worst argument for academicizing a subject I've ever read from Stanley Fish. Instead of pointing out how removing oneself from the instant issue can give one a broader perspective, he's being remarkably self-indulgent, focusing on how people have responded to prior columns. Who cares that comments on his prior columns misunderstood his point? Or, rather who cares about those specific misunderstandings?

I'd be slaughtered on end-of-semester surveys if I tried this approach with students: You're misunderstanding everything I say. That may be true, but maybe it's my fault, or maybe I could try explaining it in a different way. Implying that your immediate audience is stupid isn't endearing, even in the Gray Lady's blogs.

Listen to this article Listen to this article
Posted in Higher education at 6:55 AM (Permalink) |

March 9, 2008

Eating okra at the carnivore's table

I'm in Tallahassee this evening, giving up a day and a half to convince legislators that a proposal for a diminished university Board of Governors would be a bad idea. This evening, I asked the hotel clerk for a restaurant. The one she directed me to had one car in the parking lot: not a sign of confidence for me in the restaurant's popularity in town. Instead, I went to a good ol' Southern restaurant, full of ham and other meats. A buffet, so I figured I could get something, though I'm a vegetarian. This is the South, so even vegetables like green beans have ham in them. One has to be careful.

Fried okra. That was the solution. When we moved to Nashville in 1993, I discovered that I loved okra. I figured out how to make baked and breaded okra (with cayenne pepper!), and while most of my okra in Florida is now in soups, I still like the crispy kind. I didn't ask what else went into the fry bin, but I figure that's not my ethical problem. Everyone around me was eating meat, while I was eating okra. We got along. We each got what we needed. I suspect my fellow diners were as sated as I was when each of us left.

So I'm going to try to eat some okra tomorrow, of the conversational sort. Legislators have their interests, and I'm fine with that as long as my interests are met. We talk, we see where our common interests lie, and we try to eat at the same table.

Listen to this article Listen to this article
Posted in Higher education at 9:42 PM (Permalink) |

March 8, 2008

Embarrassments and education politics

Does anyone else think that Florida state senators waited to criticize the university system's chancellor's bonus until they wanted to decapitate the Board of Governors? I am no fan of huge bonuses for academic administrators, but surely the legislators knew of this for several years, as they've known about large bonuses for university presidents (which they are not criticizing). For what it's worth, Mark Rosenberg is head and shoulders above every other SUS chancellor we've had in Florida. That doesn't mean bonuses of this size are a good idea, but his salary is still less than the salary of several major-sports coaches in Florida, and on principle academic administrators should be paid more than football coaches.

This is the second time this week when senators I normally think of as temperate have clearly lashed out at Rosenberg in personal ways.

Listen to this article Listen to this article
Posted in Education policy at 9:05 AM (Permalink) |

Why I love reading blogs

Reason #421: when I have a headache and have to work on something tedious, Timothy Burke nails why I dislike Richard Dawkins. To be honest, it's why Burke dislikes Dawkins, but as is often the case, he is more articulate than I would be making a similar argument.

Incidentally, Burke would still be more articulate even if I didn't have a headache.

Listen to this article Listen to this article
Posted in The academic life at 8:56 AM (Permalink) |

March 5, 2008


I wish I had time to write something pithy today, but I have a communications bungle in one direction, a crisis in another, several overdue items mating in this corner, and a Sword of Damocles overhead, and to boot I'm mixing all of this up with a swamp metaphor, so Margaret Soltan (aka University Diarist) is going to kill me with an SOS entry.

No, please don't be tempted to shift the river. I have to do all this stuff, still.

Listen to this article Listen to this article
Posted in The academic life at 12:22 PM (Permalink) |

March 4, 2008

Setting the agenda or setting my teeth on edge?

I'm beginning to think that the "the candidates won't talk about education!" complaints about the campaign are a bit whiny, or at least they're striking my ear that way for a few reasons:

  • The complaints are pretty vague. "Talk about education" could mean a variety of things, and I always wonder what questions the complainers would want to ask or perhaps who they want to be writing the questions. Would Ed in '08 want Jonathan Kozol to ask questions at a debate? Would Alfie Kohn want Kati Haycock to ask the questions? (Eduwonkette, if you're reading, I challenge you to nominate the most interesting and eclectic panel of questioners at a hypothetical fall education debate for the candidates.)
  • Sometimes the oxygen is just sucked up by other things. You don't change an agenda by complaining about the agenda but by ... making the issue so visible that it's difficult to ignore it. And I know from observing a few things close up that whether an issue acquires headline status is often luck.
  • Because the luck may change, and because the campaign dynamics will inevitably be different in the fall, there will be opportunities for the priorities or at least the headlines to switch around. Make your own opportunities by being ready, folks...

Listen to this article Listen to this article
Posted in Education policy at 7:40 PM (Permalink) |

March 2, 2008

Ethnography of seminar discussions?

This is a stab in the dark, because my normal routes for searching aren't leading me anywhere, and I want to get this query out there before I forget (and I don't have a "fleeting thoughts" category for the blog): If you know of a good ethnography of seminar discussions in higher ed (either undergrad or graduate), please let me know. Equivalent studies in other disciplines are also relevant (so something in communications that looks like micro-ethnography would certainly qualify).

Listen to this article Listen to this article
Posted in Teaching at 11:10 AM (Permalink) |

March 1, 2008

You can write a very nice article describing train wrecks

The budget situation for Florida is pitiful and deteriorating. I'm on the Florida Education Association's governance board, and we're meeting this weekend. I think the students in the Florida Student Education Association and the occasional younger teacher were probably among the few who were truly partying last night at the reception. Part of it is addiction: As I told one activist who's on the NEA national board, what the heck were we doing talking shop at 10 pm? But part is being disheartened at the emerging picture in the state.

At one level, it's my emotions that are engaged, in part because I represent over 1700 faculty and professional employees at USF, and the idea of any one of them receiving a layoff notice is upsetting. Someone not being reappointed or failing to make tenure is a different issue; in principle, those should be merit-based decisions. But with a layoff, you're telling someone who's worked hard and met the institution's standards that they're gone. I hate that, and a large part of my time and energies in the last few months has gone towards addressing that.

And yet there's a part of me that knows that a budget crisis is a remarkable opportunity for studying organizations. Almost a quarter century ago, David Tyack, Robert Lowe, and Elizabeth Hansot wrote Public Schools in Hard Times, looking at how the Depression changed public education. Some in my institution look instead to Naomi Klein's The Shock Doctrine, which argues that restructuring ideas float out in the political ether, and people who advocate those ideas use crises as opportunities to push dramatic change that would never be considered otherwise. I haven't read Klein, but the representation of her argument strikes me as a more conspiratorial version of John Kingdon.

The world is more complicated, at least with regard to education. Several years ago, Iowa's plan for performance pay got knocked for a loop when a budget crisis led the state to cut those dollars, and given the realities of budgeting in most states, innovative programs funded with discretionary dollars are often the first on the chopping block. That's the dynamic whether the programs represent good, bad, or ugly ideas.

But this is clearly an area where I'm relatively ignorant. Putting school and budget crisis into my favorite academic search hopper gives us a few pieces to examine, including the following ones that look promising:

You can then snowball outward from those first entries by looking for who cites Glassberg and others. These are two of the essential tools of the academic researcher: leveraging one's interest/passion in a topic to begin crafting questions and discovering what others have already written. And I suppose this is all to say that someone else can write some fine articles on what is currently giving me nightmares.

Listen to this article Listen to this article
Posted in Education policy at 7:10 AM (Permalink) |