May 27, 2010

Evil Academic Overlords for Peer-Review Reform

As I've started copyediting the last batch of accepted manuscripts for Education Policy Analysis Archives (EPAA) from my editorial tenure, I've been thinking of John Willinsky's and Kathleen Fitzpatrick's comments about academic publishing, open access, the peer review process, and academic credentialing in general. In his incrementalist "let's push any move towards more open access" view, Willinsky pointed to Gene Glass's founding of EPAA as an example of one route to access, what Willinsky called the "zero-budget" journal. And Fitzpatrick's discussion of peer review (in Chapter 1 of the draft for Planned Obsolescence) pointed out the dilemmas of trying to generate a sustainable model of review that's new. As I'm seeing the end of my duties coming up (you really thought an editor's duties ended strictly at the end of the editorial tenure?), it's given me a chance to think about the trajectory away from subscription-based print journals. I don't know where academic publishing is headed, precisely, but I know what has happened in the recent past.


EPAA is a refereed journal, and I tried to run the English-language review process as close as I could to existing models, with double-blind reviews for the most part. But EPAA was and remains published completely open-access, free to anyone who can download the articles. So it moved one giant step away from the model of academic journals that dominated several decades after World War II, within a prepublication peer-review model. When Gene began the journal in the early 1990s, it was distributed through an e-mail list. This was only one of Gene's projects to broaden the discussion of education research through email lists, and he set up a number of lists for the various divisions of the American Educational Research Association.

He also set up a generic list on education policy, which is how we met in the mid-1990s. In a postdoctoral position at Vanderbilt, I started exploring lists and this new thing called the Mosaic browser. I subscribed to John Lloyd's spedtalk list on special education. Then I found edpolyan, which Gene had created, and I became deeply enmeshed in a vigorous 1995 debate about the Tennessee Value Added Assessment System. Eventually I started submitting articles, joined the editorial board, and was encouraged to apply for the editorial position in 2004.

In the past almost-six years, I have learned a number of things most social-science and humanities journal editors learn: how institutional support gives you some time, but never enough; how odd it is that submitted pieces can both fit the mission of the journal and leave you scratching your head on who can competently review them; how hard it is to get ad hoc reviewers to respond to requests; how review logistics are like cat-herding, only without the organization; how uneven your colleagues' research and writing skills are; how uneven your own are in comparison with some fabulous new scholars; how you never really knew how much you were avoiding learning the intricacies of a particular journal/citation style, and how much more successful some of your journal authors had been at avoiding that; how wonderful many new scholars are, and what a joy it is to give them a venue side-by-side with well-known scholars; what a great feeling it is to organize reviews so you can give coherent advice for revision; how you can be both absolutely on-target and completely off-base in predicting what articles get read, commented on, and cited; and how much you wish you could clone yourself so you could devote enough time to the journal, devote enough time to teaching, devote enough time to your own scholarship, and still have a life.

Running an open-access journal on something close to a zero-dollar budget (the college gave me a little break on teaching, and I had a wonderful graduate assistant for one year to help out), I learned quite a bit more: take the last clause in the sentence above and multiply it several times. A zero-budget operation is not an easily sustainable model to accomplish all the tasks required for a refereed journal. It requires a certain supply of surplus time, and there are no guarantees that an editor (or editorial team) will have the surplus time on a continuing basis for the central tasks, or that a reviewing pool will have the surplus time for refereeing.

Fitzpatrick addresses the reviewing part of the question, or at least the question of what would need to happen with a shift to post-publication review. She is on-target when she points out that the critical element is the evaluation of reviewing. In a standard pre-publication referee process, the editor (or editorial team) filters the referee reports, and any replacement would have to satisfy the discursive element of academic (meta-?)evaluation that Lamont described.

I understand Fitzpatrick's leaning towards an algorithm, carefully constructed, again because I worry about the time required for thoughtful moderation. My experience with the mass-reviewing process at one of my scholarly societies is not positive: I regularly receive reviewer comments for American Educational Research Association meeting proposals that are widely divergent and often enough show that the reviewer either did not read my proposal or had no clue what the standards of a discipline were. Because of the algorithm AERA uses to apportion session slots to divisions, there is a perverse incentive for divisions to encourage oversubmissions (and I've seen that operate in at least one division). That leaves program committee members the distasteful task of looking at an inflated number of submissions with divergent and sometimes irrational ratings by reviewers within a narrow window before recommendations on acceptances are forwarded from the division volunteers to the central processors of submissions. The result is that I frequently see at least reasonable proposals (both mine and others) that are not accepted, while the program has hundreds of sessions each year that are remarkably frugal in their use of scholarship. The frequent ridicule of AERA has its origins in a self-defeating program-development structure.

Maybe a more anarchic approach would work: scholars who have surplus time could become ad-hoc reviewers of working papers that appear online. I occasionally write brief blog entries on papers that are likely to gain attention from newspaper reporters, and I could as easily write entries on working papers that appear online in other areas of interest. The advantage: no one has to organize this, it would be transparent, and readers could judge the work in the context of what I write in other entries (as well as my published scholarship). It would also feed into Google's pagerank algorithm by linking to the working paper. The disadvantage: it's anarchic, so idiosyncratic public reviewing of working papers will not satisfy the scholarly credentialing process Fitzpatrick discusses. And though my blog has an ISSN, it would probably not feed into Google Scholar's algorithm. On the other hand, if more scholars are likely to read and cite someone else's work because I write about it on my blog, maybe that's not a bad thing. On the third hand, I don't really want to be a kingmaker in my subfield. On the fourth hand, maybe the fears of Sherman Dorn as Sole Public Reviewer for a certain area will push others to become more active either on their own or in creating the type of post-publication reviewing/endorsement organization that Fitzpatrick advocates.

I suspect I'm not nearly as fearsome as necessary to spur people to create such a system, but one can always dream of being an Evil Academic Overlord. Organize post-publication review or I shall destroy you!

August 31, 2009

EPAA acceptance rate, 2007 and 2008 (English language articles): 12-13%

I have been doing some tidying up with EPAA, and looked at the stats for 2008. I don't check the statistics often (and especially not early in a calendar year when manuscripts from the prior year may well be still pending), but authors occasionally need the information for tenure and promotion purposes. While there are still a few revise-and-resubmits out, it looks like the acceptance rate for the English-language section was fairly steady for the last few years, 12-13% in both 2007 and 2008.

June 10, 2009

Teachers and school demographics

A few weeks ago, the Journal of Labor Economics published C. Kirabo Jackson's study of teacher moves away from schools in Charlotte that were moving towards single-race, segregated status (see lay description here; subscription-required article here).

Today, the Education Policy Analysis Archives publishes Kitae Sohn's article, Teacher Turnover: An Issue of Workgroup Racial Diversity (secondary site), which focuses on the potential attrition associations with teacher demographics rather than the student demographics. The punchline from the abstract: beyond a relatively small threshold of racial diversity among the teaching staff, "young White teachers are more likely to stay in their original schools when the proportion of minority teachers is smaller." The article was accepted well before I knew of Jackson's study, and there are a few small (and disturbing) nuggets apart from the main findings.

I suspect that for both of these studies, there will be replications, criticisms, and debates, and that's absolutely appropriate. Both articles focus on what is an important issue for policy (how do teachers make choices about where to work), and the conclusions are fairly disturbing. For that reason alone, I hope that they are the start of more work in this area.

April 17, 2009

Four hours

It took me four hours today to write and polish a disposition e-mail to the author(s) of a submission. This was a manuscript that I enjoyed very much but for a variety of reasons could not publish, and I owed the author(s) some good advice, or at least the best advice I could give. When I began this editing gig, I'd frequently spend that much time on a disposition e-mail for almost any manuscript, and I'm still spending lots of time on R&R (revise-and-resubmit) e-mails.

And while I spend some time on rejection letters, they usually don't require as much time as an R&R request (would that be an R&R R?), because the key thing about R&R is to provide as focused feedback as I can -- that's both fair to authors and sanity-saving for me. Rejections require an explanation of why a work is either not appropriate for the journal or technically weak, but there is less of the social-contract obligation of an R&R. After all, if the author(s) revise and submit to another journal, they will (quite properly) have an entirely different set of eyes looking at the material.

The next few e-mails are for R&Rs, and I think they'll be shoved back to sometime in the weekend, because they may take some time, too!

December 24, 2008

E-book readers and faculty workflow

I agree with novelist Charlie Stress (hat tip): electronic book readers are enormously useful for people who have to read enormous amounts of text. Stress's context is the group of acquisition editors ("slushpile" readers) he knows, but it is also true for faculty. I bought a Sony PRS-505 earlier this year in hopes of becoming more efficient as an editor, and that's finally happening now that I have the right tools and habits to fit with it. In the past week or so, I've been able to make decisions and send off e-mails on a bunch of manuscripts by going to a quiet location with the following:

  • The reader stuffed with manuscripts and reviewer compilations
  • A printed sheet with a bunch of prompts for me to guide my thinking and take notes on
  • A clipboard with separate compartments for the sheets of paper and pens
  • My reading glasses
No internet, no distraction. On occasion, statistical tables are difficult to read without a printout or a full computer screen, but I'm getting used to the formatting. If I'm very lucky, I can read, think about, and prepare notes for e-mails on 4 or 5 manuscripts. But it also gives me a greater chance to get some work done when I only have a few minutes.

A similar process works for reading student work of some types. I am a teacher who often writes far more comments than other people tell me are commonly read/absorbed by students. That's fine with me, but I turn student work back more quickly if I first read through a batch without commenting (and then insert comments in the files the next time I have a few hours with a computer). 

I haven't been successful yet with the reader and published journal articles (often downloadable via PDFs). But it will work with nontechnical papers I sometimes download. I suspect that if I commuted by train, I'd do a lot more work this way. But I drive, so much of my commuting time is spent with podcasts rather than electronic texts.

December 15, 2008

Grading highs

I have a bunch of odds and ends to finish before submitting grades, but I read two papers this morning that earned very high grades in different ways. I'll explain more after I'm done, but it was a great way to start the week. The rest of this week is for grading, EPAA stuff (yes, authors, I know I owe you e-mail!), and a bunch of loose ends to tie up on the new collective bargaining agreement. Next week, the university is closed, which is both good and bad, but no one can complain about the state of my office next week. As of now, I hope everyone understands that finishing the semester is the most urgent task.

June 8, 2008

Feeling lazy at 9:30 am

I must be too-well socialized: I'm reading my daughter's book manuscript this morning, but I'll probably leave the house shortly to get some work done on Education Policy Analysis Archives and then reading and grading papers. Yesterday's class documented two things for me: I had planned enough for an entire day of teaching, and it's been 9 years since I've done that. That is, students grumbled a bit as we went a few minutes overtime (I'll figure out how to make up for that), and I collapsed after I came home. And my spouse had no sympathy for me, since she's taught full days for 13 years of her life, the last 6 in special education.

This morning, I woke up to find that our finches were mad (short, sharp vocalizing instead of singing). My wife and I had no clue why they were mad since they had fresh water and millet, and she had opened "their" sliding-glass door so they could hear the birds outside. That usually sparks some active singing, but it didn't. (The finches also like opera, especially coloratura arias.) Apparently, the younger finch had been mad at us last night. So I said hello and started exercising, which apparently satisfied the older guy, since he started singing. But the younger one was still grumpy (I think from lack of sleep, though I'm sure I anthropomorphize). He cheered up only when I did some shoulder rolls (as in whole-body rolls over a shoulder). Since birds are in very bad shape when lying on the ground, watching human exercises such as crunches or rolls must be the bird equivalent of going to horror movies: "He's on the ground and still alive: it's a giant food-bringer zombie!!!" And the nice thing about rolls is that if you do five or six of them in a row, you get much the same feeling as if you had been on a turn-you-upside-down roller coaster, all without the sunburn or entrance fees of theme parks.

On second thought, maybe the finches are just laughing at us when we exercise, an entertaining break from their efforts at redecorating. For now, I'll finish reading this chapter and then head off to work.

June 4, 2008

Can we count graduation??

Ed Week has its annual graduation special issue online, Diplomas Count 2008. Joydeep Roy and Larry Mishel have a brand-new article out today criticizing Swanson's measures as well as those of others, available at the ASU server or the epaa.info server for Education Policy Analysis Archives. (Disclosure: I'm the editor. And I've done some research on graduation myself.)

June 1, 2008

Brain... dripping... out... editor's... ears...

I've done about all I can on editing today. I still have e-mails from last week I need to respond to, reviewers to nag, disposition e-mails to write, ... but I've been in my office for over 8 hours, mostly alone, and if you are waiting for me to send you something, please understand that you don't want me to do that right now. Trust me on that one.

February 17, 2008

On eprints at Harvard and Full Monty open-access

I'm still trying to figure out the consequences of Harvard's Arts and Science faculty voting last week to push open-access publication of faculty work. This is fundamentally different from the occasional individual boycott of subscription-based journals. Harvard's faculty move is closer to Congress's push for a mandate that all grant-funded articles etc. be accessible to the public within a year of original publication. It is from these institutional moves that the publishing world will change. There is a simple, digestible explanation for the open-access moves related to grants (the public pays, so the public should be able to read) and the Harvard A&S faculty (we're established enough not to have to worry about the reputational economy of subscription journals). What flows from that is not necessarily clear, but we can reasonably assume that something will flow.

Reputational economies and the refereeing process

There are two broader issues here that need to be untangled. One is the reputational economy of academe, which is partly tied to the referee process and partly to post-publication reputational measures, such as citations. As physics has shown with arXiv, a discipline can survive quite nicely with a much fuzzier boundary between working paper and publication. But maybe that's because of the established reputation of physics. Similarly, I think history, classics, math, and other disciplines that have relatively high intellectual status (if not in resources) have nothing to fear from loosening up the refereeing process.

But what about other disciplines, including education? Education research already has a number of unrefereed publications that receive a lot of attention, largely because of differential access to publicity. Unlike medicine, where the top-reputed journals have publicists that distribute press releases (and you will see those regularly reported in the press), education has a different distribution of publicity. If you look at the indispensable Fritzwire, you'll see oodles of announcements for think-tank-based research symposia, and the ability to hire publicity folks does have an impact on what gets reported. As one colleague in another institution explained, when I asked why his work received far less attention in his area than the think-tank-based work of X and Y, which I thought was of lower quality, "Sociology departments don't usually hire publicists."

This is not to say that all think-tank-funded research is of poor quality, or that articles in refereed journals is of high quality: you don't know until you read the stuff. Nor am I suggesting that think tanks fire their publicists or stop doing the legwork to get attention. My point is rather that given the existing visibility of nonrefereed work in education, in addition to the status issues in education already, I suspect that faculty in education will be far more reluctant to let go of a peer-refereed model. Even though the notion of peer refereeing is historically and geographically bounded (see Einstein versus the Physical Review for one example), it is wrapped up in status issues. For Harvard's A&S faculty to vote for an open-access preference is one thing. For even Harvard's education faculty to go the same route? We'll see.

Economic models for open access

Since EPAA is described by John Willinsky as a "zero-budget journal," I'm living the tensions involved in open-access.  We don't charge either readers or authors for anything, though I have no compunction about asking authors to review other manuscripts as part of a reviewing ecology, and I've shifted the submission checkoff to alert authors that very long manuscripts or manuscripts with a number of tables may involve some paid preparation of an article post-acceptance. (I haven't yet asked authors to pay for such preparation, but it's a recent move.) Apart from the administrative issues involved, I am not philosophically inclined towards allowing advertising on EPAA. Maybe I should, but I and many editorial board members would be uncomfortable with that. But as a result, the burden of making the journal work is largely on volunteer labor, or labor borrowed from other tasks. Even if I were to accept advertising into EPAA, I suspect that we would not receive much revenue from it, and it may not be worth the headaches involved.

The most visible open-access journal system, the Public Library of Science, relies on publication fees charged to authors, starting right now at $1250. Here is the PLoS explanation of publication fees:

It costs money to produce a peer-reviewed, edited, and formatted article that is ready for online publication, and to host it on a server that is accessible around the clock. Prior to that, a public or private funding agency has already paid a great deal more money for the research to be undertaken in the interest of the public. This real cost of "producing" a paper can be calculated by dividing your laboratory's annual budget by the number of papers published. We ask that-as a small part of the cost of doing the research-the author, institution, or funding agency pays a fee, to help cover the actual cost of the essential final step, the publication. (As it stands, authors now often pay for publication in the form of page or color charges.) Many funding agencies now support this view.

For largely grant-funded disciplines, that's doable. For others? Not possible, either because an institution will not pay publication fees or because an author may be an independent scholar.

Here's the bottom-line concern: For journals in non-grant fields that are currently subscription-based and where there is paid staff who work on the journal, the transition to subscription-free work is fraught with risk, and I suspect that forcing all currently-operating journals to go subscription-free would result in the closure of hundreds of journals. I don't think anyone wants that to happen, but there is no secure economic model for open-access journals right now. We'll see the development of hybrids for some time (such as the Teachers College Record in education research), and that will work to some extent. And my guess is that a number of journals would have no problem with open-access for a substantial number of country-specific domains, to help scholars in countries that do not generally have institutional subscriptions to expensive journals. But that's different from the "Full Monty" open-access journal.

Where to go from here

Of the two issues, my guess is that the reputational-economy question is easier to answer. I suspect citation harvesting will be the basis of future reputation economies in academic publication. Google Scholar is incomplete and inaccurate, but so is ISI's Web of Science, and as long as academics don't treat bibliometrics as carved in stone, things should work out (or at least the problems are of a much lower magnitude than other problems we face). Unlike David Rothman, I do not see online comment forums and rating algorithms working, in part because few researchers can afford the time to invest in such forums or devices. For institutions that care about research, they will still use external reviews at promotion gates, and that will supplement other information.

The economic model of "full Monty open-access" is going to be harder to achieve. Maybe I should state what I would love, as an editor: for someone to figure out how to provide me great copyediting and compositing. Make it so I don't have the headaches of economic administration and post-acceptance detail work, and I'll probably swing towards accepting advertising or a sliding-scale manuscript-processing fee. That's going to be a bit of a challenge, since I have very particular ideas about how an article should look. But a clearinghouse that manages advertising, moderate manuscript-processing and publication fees, copyeditors and compositors, and has a quality-control mechanism for the copyeditors and compositors would do me a huge favor. And if this finicky editor will accept it, and if you can make it work economically, you just might make open-access work on a sustainable basis.

February 16, 2008

Publication stats for EPAA

Logistical chugging along tonight: I'm afraid I didn't get to any serious manuscript thought in the last few hours, but I did figure out how to analyze about 10 months' worth of manuscripts, after excluding repeated submissions and other errors:

  • 86 total manuscripts
  • 18 still in review (about 21% of this batch)
  • 55 declined (64% of all manuscripts, 81% of manuscripts with decisions)
  • 13 manuscripts either accepted or in revise-and-resubmit cycle (15% of all manuscripts, 19% of manuscripts with at least initial decisions)

In general, I only request revisions if I am confident that the manuscript is likely to be accepted at some point, after rewriting. (I do not request a revision if one of the problems is that the data collection is insufficient.) I am not pleased right now with the speed of reviewing, but that's a combination of three bottlenecks, and since one of them is my own time crunch from last semester, it's beginning to ease, and another bottleneck is getting some lubrication thanks to some tricks in the new OJS software (the submission package we use). This will never be perfect, in part because we are what John Willinsky calls a "zero-budget journal," but it should get better... at least until a tree falls on me and interrupts the flow again.

(The last is a reference to the editor of the weekly physics- and general science-politics e-mail newsletter What's New, Bob Park, whose public-education work really was interrupted when a tree fell on him. He's doing much better, now, and the tree never got a shot as his sharp tongue, which remains.)

February 3, 2008

Pacing: when adrenaline is not enough

Bad head cold time today, involving sinuses and other stuff you don't need to know about. I'm heading to western Washington state on Tuesday for most of a week, so I have to get some things done before then. Yesterday was shot, so today requires getting stuff done (priority: editor stuff).

I'm sitting here in ChainCafe, having reread referee reports and getting ready to reread the submitted manuscript. It's a revise-and-resubmit decision, so I need to go through this carefully to identify priorities for the author(s) to spend time on. (Rejections are much faster to write in practice, though in theory the need to provide feedback in a sensitive way could require more time.)

The problem is that I have just run out of energy to immerse myself in a manuscript (which is how I operate in friendly-criticism mode). Normally, I can somehow access adrenaline or some other internal chemistry to focus, but I'm wiped out. Time to rest and write an inconsequential blog entry.

Maybe I should have chosen something caffeinated instead of African Red Bush, but for me caffeine affects attention, not energy.

So this is going to be a long day of paced work. The most important and urgent tasks will happen, but nothing else.

February 1, 2008

At least Timothy Leary chose to drop out...

I think I understand Leary's choices, or at least the temptation: It's the end of two very tiring days, when I had a chance to talk for a few hours with one of the folks who tore down Florida's old Pork Chop Gang. Short story: an undergraduate I've been mentoring for a few semesters had an internship with the law firm of this Florida political hero, and after e-mailing back and forth, he needed some questions answered about the background of his senior thesis. So he proposed a joint meeting, first scheduled at the law firm and then moved to my office. I was expecting it to go about 90 minutes. It lasted 150 minutes instead. So we got off on various tangents, since he had the personal experience and I had the history, but the student said it was worth it. I had several meetings today (some planned, some impromptu, some deferred). Lots of things delayed, which is my life these days.

But even if deferred for a few days, the new English-language article of EPAA is out: Avoidable Losses: High-Stakes Accountability and the Dropout Crisis. Its authors combined interview work with following students in Texas as they were left behind in 9th grade and then dropped out. This is very difficult work to do, and the findings are provocative. Two stand out for me: that principals know that they are choosing between education and satisfying the test-score gods, and they reluctantly choose to satisfy the gods; and that to students, there is no distinction between accountability and all the practices that alienate many of them from high school. To the students in this Texas school district in the late 1990s and early 200s, there is a single massive bureaucracy that held them back, denied them opportunities in part to game the system, and never told them that their education was being sacrificed in the name of pressure whose putative goal was to ensure that they were not denied educational opportunities.

Whether you agree with the article's authors or not, I suspect it will be discussed vigorously, which is all to the good. A few years after Jennifer Booher-Jennings' article on triage in Texas, one of the models for NCLB continues to be a focus of criticism and debate.

(No, I've never taken illegal drugs, nor have I ever been tempted to, in reality. But I live on antihistamines when I have a cold...)

January 12, 2008

How do I get behind so quickly?

I'm preparing an article for EPAA right now, working on a Saturday night after just a week of the semester. (To answer those who'd like to nitpick: No, I'm not working on that article right now. I'm taking a break between formatting most of the text and inserting and fiddling with tables.) So much for my resolution to keep on top of everything.

In reality, I'm not that behind or haven't lost much ground in just 5 days, but I've had e-mails flying at me from about 50 directions, and there were at least three major things that happened this week that each took at least 3/4s of a day but then needed to be squeezed into half a day. (And this last paragraph was typed after the first bit of fast-editing magic on 4 tables. I'm going to need a stiff drink for the next part, which requires some tedious formatting decisions. By stiff drink I mean a double-caff split-tall decaf grande nowhip mini-short sugar-free cinnamon hot-chile double-shot espresso, skinny with extra whip, two raw-sugar packets, iced, and a spoonful of chocolate-covered grounds.*)

With another table behind me, I know that despite my crumbling patience and withering self-esteem (a joke, folks: no need to call emergency services), the medium-term picture is more important. In reality I had a short week given that my beautiful adolescent children weren't in school until Tuesday. So in four days I did ... lots of e-mail and other short writing tasks. But it's important to keep my eyes on the prize, which is ...

Yeah, what is the prize?

Time to move to the next table, I think.

* - I don't remember the Steve Martin movie where his character satirizes Starbucks ordering, so I had to make up my own impossible drink order. Add your own below!

August 28, 2007

Today's priorities

On the agenda today: working on my promotion packet and on Education Policy Analysis Archives.

August 12, 2007

One manuscript submission, four hours

I have just changed the projected submission turnaround time for EPAA to reflect my reality, and here's a reason for that reality: I just spent four hours crafting a response to a manuscript submission.

In this particular case, I think the manuscript is publishable with work, but it does need work, and I needed to reread the reviewers' comments a few times, reread the manuscript one more time, revise my notes on the manuscript, and write the letter. I think the resulting letter gives specific recommendations to the authors that should allow them to revise successfully (and make reader reception much better), but it's a time-intensive process.

Incidentally, I enjoy this part of editing a great deal, especially if an author replies with a much stronger paper. Maybe that's an incentive to spend a lot of time crafting revise-and-resubmit directions, but I think it's wise for the journal.

August 6, 2007

Framing NCLB debates

Matthew Yglesias has a point about the the details of NEA's No Contractor Left Behind flyer passed out liberally at YearlyKos this weekend. Yglesias notes that the message of the flyer relies on sloppy reasoning and is more sensationalist than sensible.

I'm worried by something else about the flyer: it's irrelevant to NCLB policy debates. As I've argued before, you can agree with the conflict-of-interest argument 100% and decide that the appropriate response is to build in more procedural safeguards against such dealings, not change the structure of NCLB. Fundamentally, it's a waste of NEA's resources to push this, and as a member, I'm ashamed at the poor decision-making.

But I think I understand why NEA staff have still diverted it: it holds a certain appeal for those of us angry with the Bush shenanigans. Mike Klonsky's entry on the matter demonstrates the appeal that the flyer holds for some.

(Incidentally, for those who know of Yglesias's relationship with Sara Mead, this isn't a devious insider plan to discredit the NEA. If I were really devious and wanted NCLB to be reauthorized intact, I'd encourage the NEA to waste even more resources on this nonsense. There are real conflicts of interest, but that's not a wise political focus if you want to change policy structures.)

And now, back to editing a 104-page manuscript for EPAA. It's a good one, but as I've discovered the efficiency of giving suggestions on accepting a manuscript, it's labor-intensive. I need to take breaks from the close reading/editing, and the blog will get the benefit of that.

June 4, 2007

Whew!

Just got done with hours of close editing of an article manuscript that I am accepting. I have discovered that giving editorial advice to authors of accepted manuscripts with the acceptance saves considerable time on the other end, since the final manuscripts that come in are usually coming in with fewer problems. (There are still the issues of formatting tables appropriately, but I'll slowly craft better advice.)

In this particular case, the manuscript required extensive editing, literally hundreds of recommended changes from the nature of the article. Yes, I'm confident I made the right choice, but I think I'm going to head to the library to reward myself before heading back to journal editing work.

May 6, 2007

Pithy wisdom will wait

Last night as I was driving around on a few errands, I listened to part of This American Life's 10 Commandments episode. This morning, a clever idea from that popped into my head, but I'm afraid I've been spending time on journal editing tasks--reading reviewer reports and sending out some disposition letters, starting with the easy ones (easy decisions in either direction or revise-and-resubmits where the reviewer remarks converge and where I remember the manuscript clearly enough to write them without a painstaking rereading). No, I'm not afraid at all: It's necessary and pleasant.

So I have other things to do today, both professional and personal, and clever lists will have to wait.

March 19, 2007

Reporters do hard work: School consolidation isn't the golden goose

Kudos to Scott Elliott and William Hershey for doing their homework for yesterday's article on school Consolidation: Savings may be fleeting in the Dayton Daily News. The article was about budgets, but there is also some nice research on The Influence of Scale on School Performance by Bob Bickel and Craig Howley from 1998 in Education Policy Analysis Archives.

November 21, 2006

One more study of accountability using aggregate NAEP scores

Yesterday, EPAA published Relationships between High-Stakes Testing Policies and Student Achievement after Controlling for Demographic Factors in Aggregated Data, by Gregory J. Marchant, Sharon E. Paulson, and Adam Shunk of Ball State University. Its conclusions:

The few relationships between high-stakes testing and achievement or improvement in reading, writing, or science tended to appear only when demographic data were missing; and the minimal relationships with math achievement were consistent with findings in previous research. Considering the cost and potential unintended negative consequences, high-stakes testing policies seem to provide a questionable means of improving student learning.

Marchant et al. worked very hard on this study (go read it!), and it tends to reinforce what we've read elsewhere (e.g., Nichols, Glass, and Berliner's work earlier this year). I think, though, that given the availability of individual-level NAEP data, it would be wonderful to see people start to use that, and so today, I have my first editorial, No More Aggregate NAEP Studies? It is not a criticism of the work that people have already done using aggregate data to want researchers to use the finer detail now available. The board members were wonderful in providing honest feedback when I circulated a draft a few weeks ago, and the short editorial is better for their input.

November 18, 2006

One small task for an editor, and one article waiting for a tiny thing

It took me about a day and a half to scrounge together enough time to whip the next article into shape. When you see it, you'll understand: more than 20 tables. The authors did a very nice job of getting the tables about 90% of the way, but I'm finicky on some things. But that's done, as well as a disposition letter that's been hanging for half a week. Online editorial-board conferencing the week after Thanksgiving.

And next week is carving-out-time week. I hope!

Added: To answer profgrrrrl's question (in commnts) about what an editor does, my situation is a bit unusual (though growing more common), since Education Policy Analysis Archives is an online, open-access (i.e., "free") journal. It's free to those who read it, and since Gene Glass founded it in 1993, it's operated without page charges, either. As John Willinsky put it in The Access Principle, it's a zero-budget journal, at least in terms of cash changing hands.

That's not true, of course. The colleges of education at Arizona State University and the University of South Florida have given Gene, Gustavo Fischman, Chris Murrell, and me time to edit and do the technical stuff on the journal, and I assume Pablo Gentili's institution (Universidade do Estado do Rio de Janeiro) has as well. But while there's a little available to do some translations of abstracts, there is no "publisher" outside the institutions. So we do everything.

When the weeks go well, I can devote at least Monday to editing duties. Here's an incomplete list of activities, in a rough workflow order for article processing:

  1. Read incoming manuscripts (to see if they're good enough to criculate to reviewers)
  2. Identify potential reviewers (look through the existing pool in the editorial board and other volunteers, then scrounge through my memory and other resources for volunteers)
  3. Send reviewers a request and correspond as necessary to follow up
  4. If reviewers decline, go through steps 2-3 again if necessary
  5. Thank reviewers who have completed reviews
  6. Read reviews. Reread manuscript. Make a decision
  7. Write letter to author explaining decision
  8. Check webpage for revisions
  9. Read revisions ... (this can be a recursive process, but I'll skip the graphic on that)
  10. Send authors of accepted manuscripts a note explaining what they need to do to prepare a final copy
  11. Send the title and manuscript out for translation
  12. Take the authors' prepared copy and prepare it for publication (a process that can be painless with manuscripts that are polished and more involved depending on the number of tables and the quality of the writing and citation mechanics).
  13. Send authors the 'galley' and any specific requests for clarification as necessary
  14. Make corrections as necessary
  15. Send file to Chris Murrell for uploading. Send notice out in various ways.

The task that can bog me down, because it takes 30-60 minutes per submission and because I have to focus about as hard as I ever have to, is finding reviewers. For example, if a paper comes in on funding prekindergarten Montessori programs in Tasmania, I need to find folks who are familiar with Australian education financing, or maybe Australian preschool programs, or ... and that can be a fun scavenger hunt, and I usually learn a great deal about scholarship by having to find reviewers, but not if 4 manuscripts come in simultaneously (or close to it).

This list does not address the other issues involved in editing, such as nagging various indexers to figure out whether they'll carry the journal, setting up various projects (e.g., the applications and review process to create the new-scholar board early in my editorial tenure, or a few other things in the works), etc.

September 24, 2006

An editor's joys include organized authors

The next article going up at Education Policy Analysis Archives (early in the week) came to me almost perfectly prepared (in terms of the post-acceptance process). Given my crazy week, this counts as a mechiah. (See an online Yiddish reference source such as this glossary if you don't know what it means.)

Of course, the readers should appreciate it for other qualities.

August 20, 2006

Smooth work satisfies

Took a little bit of time to go out of the house to ChainCaf√© this afternoon to finish preparing an EPAA article for this week. Smooth... a few minor editorial changes, then the usual format issues (change hyphens to en-dashes where needed, expand the italicized text a bit) and send it off to the author for vetting.  There are no questions that he needs to answer, just check the 'galley' for goofs.

And last night, I did the first check on the galleys for Education Reform in Florida. I found a few goofs but not many. The index is virtually all the work left.

Just saying, I like when things go smoothly. I could use smooth more often in my professional life.

August 18, 2006

EPAA manuscript stats

A few months ago, I switched the intake and review process for EPAA to a web-based system, and here are the interim stats from the point at which I switched over (since the system can calculate statistics for me in about two clicks):

Declined: 33%
Revision requested: 25%
In the review process: 33%
Not yet assigned to reviewers: 8% (a few manuscripts I've received and either have not yet evaluated or need advice for reviewers because of the unusual nature of the expertise needed).

I haven't yet accepted a manuscript that's been submitted through the web system, but that will change shortly: it will just take some minor revisions for a few manuscripts. I accepted a few revisions recently from the older e-mail method, and there are a few revisions I'm still waiting on from before the switch.

August 14, 2006

Editorial marker/to-do list

If it's a Monday morning during the academic year (and, yes, I'm already on contract), it'll be journal management stuff—engaging in the type of detail that can drive an editor to drink be glad that an online journal system exists. We're behind in publishing English-language articles right now because authors who need to answer queries to help get an article up are often on vacation and out of e-mail range in the summer. I heard from one author today and hope to receive at least one set of answers within a day or two. I also received two final MS's over the weekend, so I can work on them when my brain isn't full of brain-melting editorial tasks (e.g., who has the expertise to read this manuscript? which is fine until you consider that my reviewer database includes not only the main editorial board and the new-scholar board but a bunch of very nice volunteer ad-hoc reviewers, all totaling 160). Today, I made review assignments for 6 manuscripts. One came in either yesterday or this morning that I responded to, and there are three left I either need to make assignments for or decide not to circulate for review.

Other tasks for EPAA this week:

  • Prepare one more MS to 'galley' shape and return to author.
  • Take several MS's with enough returned reviews and make decisions.
  • Take editorial-board advice from e-mails, synthesize, and implement.

Now it's time to turn to other things for the afternoon. It's not appropriate to write MS disposition e-mails right after tedium.

July 25, 2006

Quick work...

I did get in at least one letter to a MS's authors before leaving for Ocala. Have fun and don't get into too much trouble while I'm away! More tomorrow...

July 24, 2006

Inefficient but necessary journal work today

I've mostly prepped the next article to appear at EPAA—I am waiting to hear back on a few things—and am now on the one for early August. There are the usual tedious issues, and it's taking a little more time than I expected (or maybe my brain is being inefficient today and needing frequent breaks from editing duties). Yes, that's right, faithful readers—any entertainment you've had from my entries on academic freedom are largely because I can't stand the next step in the process, whether it's hunting for hyphens to be replaced by en-dashes or whatnot. Better to work on this now, still. Tomorrow I'm off to Ocala for family reasons and then have child care teen and preteen entertainment duty starting Wednesday while Elizabeth starts back in "pre-planning days" for the K-12 school year, which begins for students August 3.

I had hoped to get a few disposition letters written, but that will have to wait until Wednesday evening.

July 14, 2006

Rice University breaks the mold in university publishing

I'm delighted to hear that Rice University is restarting its refereed university press as an online-only enterprise with print-on-demand options. I hope this provides a demonstration of the enormous stability of online publishing costs—no worries about printer costs to force presses to figure out what they can publish. The primary costs will be in the reviewing process and the compositing. And those are not negligible, certainly, but they're rationally related to what should be the priority of university presses (finding the best monographs to print).

June 19, 2006

The leisurely pace of editing

I spent over half of my worktime today doing editing stuff—sending out most of the disposition e-mails to authors I owed (still have two or three left for tomorrow) and seeing if the new article is up yet (it is, but I'm too tired to vet an e-mail announcement, so that's tomorrow's first task).

Reading manuscripts, whether incoming or after reviews, is one of the more demanding parts of editing, and it's the second-most enjoyable task I have as an editor. Polishing an article's look is detail work, I'm not perfect at it, and it's not about the ideas for the most part. Helping an author improve a piece is definitely the most rewarding part of editing. But after that, the initial read and then the re-read after reviews are returned are a fascinating exercise in listening to perspectives and reading with three lenses on: Would it contribute something? Do I see what the reviewers saw? Am I seeing everything I can here? That attempt to keep multiple perspectives in the air is harder than reading dense prose.

It also means I can't quickly skim the type of article I was going over today—papers where the reviewers were generally positive but mixed. Would it contribute something? became Would it contribute enough, given everything else? Do I see what the reviewers saw? became How do I sort these issues in order of importance? Am I seeing everything I can here? is of course the hardest one, and explains why I spent more than an hour rereading and deciding how to write a revise-and-request letter for the first MS I tackled this morning. That was relatively quick, too.

May 16, 2006

Open access and citations

Citation Advantages of Open Access Articles by Gunther Eysenbach is now available at the Public Library of Science Biology journal (which is open-access). It continues a line of literature exploring the citation advantages of open-access publications.

Thanks to Donna Martinez for pointing this out on the AERA-L listserv.

May 4, 2006

Federal bill would require open-access publishing for funded research

The Washington Post has reported on a Senate bill that would require free online publishing of research results within six months of refereed publications for research from 11 federal agencies (those that have $100 million or more in funded projects). Groups with various interests in publishing are lining up on the expected sides—open-access advocates in favor of the bill, publishers and small academic societies against it.

What's needed is some support for the infrastructure of publications, including journal websites, institutional repositories, and bridge funds for academic societies. I don't think that having windowed access will hurt their subscriptions—most research libraries will maintain subscriptions as much as they can because of faculty needs—but having some reassurance for the small academic societies that currently use journal subscriptions as revenue sources would be appropriate. Of course, I wonder how many of their journals only publish federally-funded research. Not many, I suspect...

April 19, 2006

Book review of Willinsky's The Access Principle

Thanks to A.G. Rud for noting the online publication in Teachers College Record of Gene Glass and my review of John Willinsky's The Access Principle (which you can download for free).

January 3, 2006

Nichols, Glass, and Berliner

Today, Education Policy Analysis Archives publishes High-Stakes Testing and Student Achievement by Sharon L. Nichols (U. Texas San Antonio), Gene V Glass, and David C. Berliner (both of Arizona State University). It's a provocative article that argues that an extensive effort to look for relationships between high-stakes testing and state-level performance on NAEP from the late 1980s through a few years ago found evidence that high-stakes testing increased achievement only for fourth-grade math and in a limited fashion. Because I commented on an earlier draft of this document, one that appeared as a report in September (my name's in black and white as a reviewer on the last page of the appendices), readers may well be curious of the process followed on receipt of the manuscript. There needs to be some care when members of a board submit a manuscript to the journal and when an editor has seen a manuscript before.

I received the manuscript in late March and assigned it to five reviewers (sending them each a blinded copy). Because of the manuscript's length, I gave reviewers a considerable part of the summer to return remarks. After receiving them, I stripped the names off the comments and sent the compilation to a trusted member of the editorial board who knew the identity neither of the authors nor of the reviewers, and I asked the board member to make a decision on publication as a proxy editor: publish essentially as is, ask the authors to revise and resubmit, or reject. Based on the comments and his or her own sense of the manuscript, the proxy editor for this manuscript decided to ask for a revision. I conveyed the proxy editor's statement, comment, and all of the reviews (a total of 18 pages) to the authors in early September.

When the authors returned a revised version to me in the fall, I was removed by several revisions from the version I had commented on earlier in the process and decided that it would be acceptable on balance to make a decision, based on the revisions and the earlier comments of reviewers and my proxy editor, Les McLean of the University of Toronto. (There is also a question of how fair it would have been to Professor McLean to ask him to continue to serve as a proxy editor for such a long manuscript.) The result is a long article but one that is considerably tighter and stronger than either the report that I saw in draft form or any of the versions in between. I take full responsibility for the decisions first to remove myself from the initial review and then to make a final publication decision.

This article continues a debate over the effects of high-stakes testing, in the forum of a professional journal where the merits of the research can properly be debated. Nichols, Glass, and Berliner develop two new measures of pressure—a cross-sectional measure relying on accumulated ratings of many reviewers, what they call the APR, and a proxy longitudinal measure using an expert whose cross-sectional ratings correlate positively with their APR. This article will not end the debate, and I don't think that the authors expect it to. But it continues the development of rating high-stakes pressure in an new direction, and it contributes significantly to this literature. I look forward to the continued debate.

Update (3:16 pm EST): Thanks to A.G. Rud, a member of the editorial board who caught a misspelling.

December 25, 2005

Done and not done

Many of my friends, neighbors, and colleagues have been preparing (perhaps frantically) for either of the holidays today (and Merry Christmas and Happy Hannukah to those for whom it's relevant). And, from both popular culture and personal testimony, I gather that there is a certain point at which one says (or thinks), "It's not done, but it's as done as it's going to be for now."

The publication a few days ago of John Robert Warren's State-Level High School Completion Rates gave me a similar feeling as an editor, researcher, and observer of research. Last year, EPAA published Jing Miao and Walt Haney's High School Graduation Rates, which compared several methods for estimating high school graduation. Since then, Jay Greene has adjusted his method to account for population change with census data (though he should've acknowledged Rob Warren's work on this point, which has been available from Warren's web site at the University of Minnesota for a few years). And Warren polished his method, I think presented in a refereed journal for the first time in EPAA.

More below the jump.


(For what it's worth, getting Warren's article out was the easiest process I've had yet with an article with tables or figures: I make a template document available to authors when I accept a manuscript, and he took full advantage of that opportunity. So all I needed to do was ask him for slightly different versions of two figures, work a bit with the formatting of tables, and do a few search-and-replace commands for typographical reasons, and it was ready to go. Well, until the errata, at which point I would create an updated version. So the article is done and not done.)

The substantive research, however, will go on, and here is where I'm sure any author feels that "it's as done as it's going to be, for now." The renewed interest in measuring graduation comes from No Child Left Behind, which includes a graduation rate as a key measure, but without really defining it well. So in step academic entrepreneurs, with their suggestions (and with the additional motivation for some of judging reforms by graduation rates—Warren has a number of pieces that use his measure for other purposes, so he is done and not done).

Part of the problem with measuring graduation has been school officials' and statisticians' continued publication of data based on administrative dropout counts (an awful idea and something inherited from the first headline-level concerns over dropping out as such in the 1960s). The recent research has focused properly on measuring graduation instead, and I think Warren has a pretty good approach on measuring cohort graduation at the state level. By definition, it certainly is the latest approach.

But work will continue. I have my own ideas, focused on statistics reported by age rather than grade. You can see a partial draft of that approach, with the introduction of the central concept and one illustration. The real sticking point for all of us here is estimating migration at anything below the state level. Warren's approach is good at the state level, but things get gnarly very quickly at local levels, which is where NCLB's graduation rate becomes very important, and where we'd like to have a reasonable method. In individual school districts and schools, net migration rates can be high enough to make an unadjusted cohort or period measure highly inaccurate.

And, at the level of a journal, I'm also done and not done. Warren's piece is the last article for the year in Education Policy Analysis Archives, and this is roughly the end of the first year I've been editor. It's been an intriguing transition (full of things to learn about post-acceptance processes!), and I'm delighted to have ended with a piece in my own area of interest and that continues a small series that EPAA has published on it over the years. So the article is done and the field is not done.

For now, I'm headed out of town for a week, with little if any e-mail access, having just sent the first editor's draft of the first article for next year to its authors. It's provocative and continues the journal's history of using an electronic journal for things that a print journal could never pull off—in this case, publishing a 58-page article, turning it around from acceptance to publication on short order (compared to the post-acceptance process at many hardcopy journals), and publishing a set of appendices that's longer than the article and longer than many entire issues of hardcopy journals. But having polished the 58 pages of the article and done a once-through on the appendices, it's time for me to send my version to the authors with minor queries and head out of town, done and not done.

December 20, 2005

Cranes...

So what do faculty do after turning in grades? Some escape town entirely, but while I'm heading to a family gathering next week, the rest of this week is getting to some things that are past due—which constitutes roughly 98% of my to-do list.

Today, that's mostly consisted of working on a MS that I'm hoping to get out to the world January 2nd. It's a substantial piece, about 200 pages, and while most of that bulk comprises appendices, I need to polish the main article. Right now, I'm up to Table 12 on page 25.

More on the jump.


Some months ago, I had an interesting dream in which a crowd of sandhill cranes was preventing me from moving around. The piece de resistance was the sandhill-crane couple in a car, who refused to move the car. Being entirely ignorant of most forms of dream analysis, I boldly interpreted the cranes in the dream as being my various obligations in life, including at work. For those who have never been lucky enough to see them, sandhill cranes are large birds whose early-morning honk is unmistakable (once identified). Their overhead flight in my neighborhood is usually about 15-30 feet over the ground, so you get a close look at them. I've never been threatened by the critters (different story with geese, rather nasty critters), so when I've felt overwhelmed during the semester, I've explained to myself and others that I need to take care of some cranes. After all, most of my obligations at this point are entirely voluntary, rather unique and attractive when examined close-up, and have absolutely no side-effects other than wanting to spend more time with them.

EPAA is really a small flock of cranes. The one demanding my attention today is the post-acceptance pipeline. Then there's the pre-review processing, which I hope to get to tomorrow. And the invitations-to-new-board-member-invitees. And planning-the-future bit, with one of the associate editors. And then back to this article in preparation. And so it goes, but without the Vonnegut connotation.

Open-access publishing

Today, Inside Higher Ed features a new book by one of EPAA's editorial board members, John Willinsky. The Access Principle: The Case for Open Access to Research and Scholarship will probably be one of the things I need to take with me on a trip next week. It's sure to spark considerable debate about academic publishing as an enterprise.

And given John's hard work on the editorial board and his fabulous Public Knowledge Project, he has quite a bit of practical experience with open-access publishing. Pay heed, fellow academics!

December 5, 2005

Submissions in-box

I love having a separate submissions email address for EPAA But I wish it weren't filled up more often with spam than articles. 25 deletions and counting this morning...

October 28, 2005

Praise for laggardly editors

A very nice note today from an author of an article in Education Policy Analysis Archives:

It's amazing how much faster EPAA gets articles into print compared to paper journals... I have another paper that has been "in press" for well over a year now—and I haven't seen the galleys yet. So—thanks to you, Gene Glass and the EPAA team for creating a real alternative that every journal should emulate.

Aw, shucks. To be honest, I'm not feeling entirely on top of things, but that's because this is definitely a roller-coaster I've jumped on, and it's not going to stop. There are plenty of things I'd like to do better, faster, or more of (and maybe even start on!) with the journal. Thus far, I think I'm doing fairly well to keep up an article-every-two-weeks (or slightly better) average since the beginning of the semester. And I think that'll match the pipeline reasonably well, with maybe a slight break in the summer. Sometimes (as this week, when I'm still behind on grading in classes), I want another dozen or so hours in the week, but it's much better than in the spring.

So, is there such a thing as TMI from a journal editor?

October 18, 2005

Who understands peer reviewing?

In the last few months, I've sent out a sprinkling of manuscripts for Education Policy Analysis Archives to folks outside the normal spheres of academe—not only those in think tanks who generally understand reviewing processes because they often submit to refereed journals, but also those in the surrounding "wonkosphere" (for want of a better word), whom I think would understand refereeing. In two cases, from individuals I generally respect, I've received e-mails that indicate they don't get the concept. "I'm not sure why you want me to review this. Don't publish it!" ran the spirit of one e-mail. "You wouldn't be doing your journal a favor, because of the following errors:..." So I thanked my correspondent for doing exactly what I asked by providing me a reality-check. Much to her credit, she replied to indicate she got it. I think.

Here are the rules of refereed journals:

  1. Not everything submitted is published.
  2. Nothing published is perfect.
  3. The editor(s) dispose(s) of some portion of manuscripts before anyone else sees them, for a variety of reasons.
  4. Referees are sometimes cranky and sometimes misread pieces of a manuscript.
  5. Editors take referee reports very seriously and still ignore a certain portion of them.
  6. There is no law requiring a certain number of reports, or a certain quality of reporting, before an editor makes a decision.
  7. Journals differ in what precisely is blind to either referees or authors.
  8. Thick skins are required when reading reviews. Einstein apparently got huffy when faced with peer review for the first time in the 1930s. (Thanks to Ralph Luker for the pointer.) (See also the item above about variations in refereeing practices.)
  9. There is no natural law about refereeing, but it sharpens published works, and Plato would approve of it. (See Book 7 of The Republic—no, not the cave allegory, but the part later where the Socrates character is discussing the proper training of philosopher-kings.)
  10. There are suspicions in academe that journals that charge page costs also accept a higher proportion of manuscripts.
  11. There is a suspicion in this editor that fields with higher needs to demonstrate status sometimes have obsessive-compulsive disorders over refereeing processes. The main route for getting new ideas into circulation in physics is not refereed. Then again, physics is a Science Biggie that doesn't have to prove itself to anybody.
  12. I edit a journal that is refereed, and I don't anticipate ending the refereeing for a simple reason: I don't know everything or even nearly everything I need to know to make publication decisions.

The test on this material is next Friday.

July 17, 2005

Table formatting

I've created a table-formatting tutorial because I don't think another one exists to show how one formats academic paper tables in a way that is clean and easily readable, especially using symbols for statistical significance. Let me know what you think of it!

June 12, 2005

Data labeling trick in Excel

If you've wanted to label individual labels in Excel charts with extra data, there have typically been the option to create bubble charts, where the diameter of the bubble corresponds with a third variable. But what if the extra data is text (e.g., states in a national analysis). Check out Microsoft's macro to label data points in a scatter plot with a third column of data. In general, I am wary of Excel charts because you have to reformat much of any plot to be readable, and because of the tricks one has to pull with VBA to get the type of labeling that standard statistical packages can easily produce. But, if one knows Excel, there are ways to improve the output. (I like much of Edward Tufte's advice on presenting information visually; see his web site's discussion thread on graphing software for professional alternatives to Excel.) Incidentally, please let me know if this information is helpful for authors, before I commit editorial advice of this nature again. (The next advice entry planned: how to format tables in Word.)

Acceptance statistics

In the six and a half months since I've been editor of Education Policy Analysis Archives, I've received 90 manuscripts (excluding a few withdrawals from authors who decided to resubmit a MS in Spanish to the journal). Here is the disposition:

  • 7 accepted
  • 9 revise-and-resubmit requests
  • 14 rejected after reviews
  • 37 manuscripts rejected before reviews (generally manuscripts that are aimed at the general public rather than an academic journal, manuscripts that fall outside the scope of the journal, and manuscripts that are really early drafts)
  • 18 manuscripts in review
  • 4 manuscripts received but not in review yet
  • 1 manuscript that I returned to the author for resubmission after one technical detail is changed

Technically, if one excludes revision requests, that's a 12-percent acceptance rate. But that is misleading. Because Education Policy Analysis Archives is online, it receives a greater proportion of manuscripts that get a quick no. If I excluded the submissions that are clearly intended for a general audience, the acceptance rate would be higher. In addition, I wonder if people submit what really are drafts to me because it's easy to e-mail a submission. I've been a little tougher in the last few months with these because I've been processing a long manuscript queue, but it's a judgment call. How much does someone need a review for a paper because they're a new scholar and need some outside perspective? But is it a wise use of reviewer resources to send someone a manuscript that is not well prepared?

June 4, 2005

Accepting manuscripts

Now that I've been the editor at Education Policy Analysis Archives for six months, I'm starting to settle into the job. Some things are quick but don't give too much joy. Easily and without guilt, I can reject manuscripts that are clearly inappropriate (stuff that might belong in a popular magazine or course papers that were sent to me because the professor requires that one paper be submitted to a refereed journal). In a few cases, I've been able to accept a submission right off the bat. Most of my letters are rejections based on reviewers' comments and requests for authors to revise and resubmit manuscripts.

And I have had two things that have given me great joy thus far. One is the creation of the graduate-student editorial board, twenty new scholars from around North American who bring an energy and fresh preparation to the journal. The second include the acceptances of revised manuscripts. It is especially in revisions where I feel I am making some direct mark on a journal, not just in making decisions but in shaping specific articles.

I am still climbing the next learning curve, the post-acceptance process...

April 28, 2005

Slowly, slowly...

Over the weekend, I distributed the last of January's Education Policy Analysis Archives manuscripts to reviewers. That still leaves a backlog from February on, but I'm feeling much better, and there's a reasonable chance that I can clear the backlog by the end of June, leaving me free to respond to manuscripts as they come in, instead of at the end of a queue. Much of this comes from having finished the reviewing of grad-student editorial-board applications and having made the final selections.

Electronic journals should be fast, but that's not how I feel right now. I'll get there, though, ...

March 5, 2005

Working for boba...

I'm currently in the north Tampa Boba Internet Cafe, having fled my office after a few hours of isolation (and good work on my chapter for the Imagined Communities book). I've seen it several times while driving past and decided to try it. It serves "bubble drinks," or sweetened teas or fruit drinks with tapioca pearls. Very sweet. It also has one of the pop radio stations on (tending towards rock but not too interesting), so I'm glad my work consisted entirely of setting up a reviewing-queue table for Education Policy Analysis Archives, copying titles, authors, and contact information into a table. I suppose I could have set up the next set of reviewers (my original intention, since I still have a backlog I inherited from my first day as editor, plus more manuscripts that have come in since), but I decided a good mindless task was appropriate, so I came up to the present on the titles and whatnot. I'll print it out and start assigning tentative reviewers over the next few days and start the next wave of reviews. Earlier today, I finished reading through the graduate-student editorial board applications (three dozen applicants, each with recommendation letters!), and the next step is consulting with the board. And there are two or three disposition letters I need to write—Monday, I think. I can't neglect my classes tomorrow, when I hope to get a bunch of teaching stuff done.

Anyone have a spare week or so?

February 18, 2005

Editors meeting

One of the promises I made in the proposal to become editor of Education Policy Analysis Archives was to head to the Phoenix area for a meeting to coordinate the transition early in 2005. Well, it's early in 2005, I'm on my way to the unveiling of my father's headstone, so I'm stopping off here to meet with Gene Glass (former editor), Chris Murrell (who does the technical stuff), and Gustavo Fischman (co-editor of the Spanish/Portuguese section of EPAA). I'm at the endstage of a cold—stuffy nose, forgot the decongestant at home—so I'm awake at about 4 am local time (6 am Florida time) and will try to get a bit more sleep shortly. But there are arcane matters of style to discuss, such as leading (pronounced "ledding"), indentation, bullet-point uses, etc. Or at least to just figure out. And I'm sure Gene has things he wants to discuss. No, not at 4 am! Maybe in 6 hours.