Category Archives: Peer Review

Caught Between the Old and the New

Over the past academic year I’ve worked on a research project with a colleague to study the ways that students do their scholarly work, similar to the project at the University of Rochester a few years ago. We finished with data collection for this year and are spending the summer analyzing our results. We’ve gotten an additional grant and plan to collect data at a few more sites next year; ultimately we’ll produce a comprehensive analysis of all of our data. But in the short term, we’d like to share our preliminary results and analysis from this year’s research.

Here’s my dilemma: the fastest and most efficient way to disseminate our results is to share them on the website we’ve set up for the project. When I was an archaeologist we wrote up an interim report after each field season and a final report when the project was complete, and I’m thinking along these lines. However, I’m also a junior faculty member on the road to tenure, and the currency of the realm is, of course, the peer-reviewed journal article.

A peer-reviewed article will take considerably more time to be published, up to a year or even longer, especially if our submission isn’t accepted on the first try (as seems true for most article manuscripts). I’m a strong advocate of open access publishing, and it just seems wrong to keep our data to ourselves for all that time. But I do value the peer review process, and while I hope that posting a report on our website would generate comments, there’s no guarantee.

Ideally I’d like to write both a preliminary report, to be posted online by the end of the summer, and a scholarly article, submitted around the same time and (hopefully) published sometime next year. I’m not sure that we have time for both, though. While the summer months are slower in the library, we’re still open, and there are classes and reference desk shifts to staff and programs to plan for next year. So we are probably going to have to focus our energies on just one publication.

As I’ve been thinking on this recently there’s been lots of other news in the world of academic publishing. The University of California proposed a possible faculty boycott of the Nature Publishing Group. And an unusual scholarly publishing project came out of the Center for History and New Media at George Mason University: Hacking the Academy, a book that gathered all of its submissions in just one week. I can’t help but think that we’re in an odd scholarly communication moment right now, stuck between old and new worlds of knowledge dissemination, and I’m not always sure how to chart my course.

The Future of Peer Review?

It’s still a few weeks until Open Access Week, but starting now you can help reimagine what scholarly publishing might look like in the future. Media Studies scholar Kathleen Fitzpatrick has made her new book manuscript available online for open peer review. While Planned Obsolescence: Publishing, Technology, and the Future of the Academy will go through the traditional blind review process (it’s slated to be published in print next year by NYU Press), Fitzpatrick also plans to incorporate reader comments from the online manuscript into her revisions, asserting that “peer review will be a more productive, more helpful, more transparent, and more effective process if conducted in the open.”

The beginning of open peer review for Planned Obsolescence also marks the launch of MediaCommons Press, the latest project from MediaCommons (which Barbara first alerted us to a couple of years ago). MediaCommons uses the CommentPress theme for the popular, open source WordPress blogging platform. Manuscript text is displayed side-by-side with reader comments, facilitating paragraph-level discussion of the book.

Of course this isn’t the first experiment with open peer review of scholarly works. Fitzpatrick published an article about CommentPress in the Journal of Scholarly Publishing in 2007, and also made it available online for comments. Noah Wardrip-Fruin opened up the manuscript for his book Expressive Processing to “blog-based peer review” on the group blog Grand Text Auto; it also went through the traditional peer review process before being published by MIT Press this year.

What’s most interesting to me about the Planned Obsolescence project is that the book itself discusses the process of peer review and scholarly publishing. Browsing the chapter titles and subtitles there looks to be lots of interest to academic librarians: discussions of authority, intellectual property, preservation, and the sustainability of university presses. I haven’t had a chance to read more than the first few pages yet, but I’m looking forward to continuing (and commenting, too).

Balancing Act

I’m kind of in the pickle that Maura describes – subscribed to too many sources of information that I would read if I weren’t so busy keeping up with the stream of new information. But Current Cites is always a good ‘un for finding a cross-section of interesting new stuff and this week it pointed me to a twig I must have missed in the current. Sometimes it’s only when you see it the second time, maybe just as you’re pouring a second cup of coffee int he morning, that it catches your eye.

First Mondays (an excellent and long-established open access journal) has an article by Brian Whitworth and Rob Friedman on “Reinventing Academic Publishing Online.” In a nutshell, it examines the fact that the “top” academic journals remain vested in a traditional system in which maintaining barriers and exclusivity because their exclusivity is perceived as rigor and therefore value. The higher your rejection rate, the prouder you are. But there are two mistakes academic publishing can make: publishing stuff that isn’t any good and not publishing stuff that turns out to be good. It’s the cost of the latter – failing to publish something innovative and challenging for fear it might be wrong – that these authors feel is left out of the equation.

These error types trade off, so reducing one increases the other, e.g., a journal can reduce Type I errors to 0 percent by rejecting all submissions, but this also raises Type II errors to 100 percent as nothing useful is published. The commonsense principle is that to win a lottery (get value) you must buy a ticket (take risk). In academic publishing the rigor problem occurs when reducing Type I error increases Type II error more . . . Pursuing rigor alone produces rigor mortis in the theory leg of scientific progress.

The authors point to the fact that the publishing industry essentially determines who is hired and fired in universities, which flies in the face of the mission we are supposedly on and the intellectual freedom that should enable our work.

When a system becomes the mechanism for power, profit and control, idealized goals like the search for truth can easily take a back seat. Authors may not personally want their work locked away in expensive journals that only endowed western universities can afford, but business exclusivity requires it. Authors may personally see others as colleagues in a cooperative research journey, but the system frames them as competition for jobs and grants. As academia becomes a business, new ideas become threats to power rather than opportunities for knowledge growth. Journals become the gatekeepers of academic power rather than cultivators of knowledge, and theories battle weapons in promotion arenas, rather than plows in knowledge fields.

The authors suggest that under the color of “rigor” this model sustains a system in which cross-disciplinary and innovative research is unwelcome. “As more rigorous and exclusive ‘specialties’ emerge, the expected trend is an academic publishing system that produces more and more about less and less.” (And hey, it’ll make the Big Bundle even bigger and more expensive, therefore more profitable.) They think instead technology could offer ways to facilitate information exchange rather than creation of further citadels of isolated specialization. Paying more attention to the mistake of failing to publish something that turns out to be worthwhile will require the creation of a democratic open knowledge exchange which can better balance the equation.

The funny thing is that this tension has existed for a long time. Well before the Internet enabled the opportunity for fundamental change in the way we share research, both Michael Polanyi and Thomas Kuhn described the delicate tension between maintaining an agreed-upon understanding by fending off crackpot theories and the need to allow something new to challenge the dominant paradigm. Both self interest and a more idealized notion of rigor conspire against innovation. What I find interesting about this First Monday article is the idea that our current dominant publishing model has let self-interest reign supreme, and that a new open model could let the more idealized urge to preserve that which is solid and true duke it out with ideas that challenge it. It could balance the risk/reward tradeoff involved in choosing what to publish and which questions to pursue.

By the way, what is your library planning to do for Open Access Week?

(Photo courtesy of rptnorris.)

Faculty Blog Round-Up: Writing Books

At the peak of summer, many faculty are in deep research mode, especially with longer projects, like books, that require the kind of travel or in-depth work they can’t schedule during the semester.  Here’s an overview of the book-writing process from the inside

Dr. Crazy, an anonymous literature professor, is beginning to ponder her topic.

Anthropologist Auto Ethnographer is in the throes of research - research that goes to show why sometimes we just need the original print texts.

Flavia, an anonymous professor of renaissance literature, is substantially revising her dissertation - and has come to some interesting realizations about her book-in-progress.  Check out the comments here, too.

Notorious Ph.D., a historian, is revising and ambivalent about her readers’ feedback.

Finally, John Holbo, a philosopher at National University in Singapore, has just published a book on Plato (with translation by Belle Waring).  This post is interesting for two reasons: it’s an experiment in simultaneous free e-publishing with a print book for sale, as well as reminding us how the scholarly conversation doesn’t end with the book’s publication.

Faculty Blog Round-Up: The Publishing Cycle

Over at Edge of the American West, UC Irvine English professor Scott Eric Kaufman has a bit of a rant about both the delay and format of the January issue of the journal of the Modern Language Association.

Cheer up, SEK; it could be worse.  The anonymous Lumpenprofessoriat tells a tale of woe, with an eventual happy ending, about a much longer submission-to-print process.

On the other hand, Eszter Hargittai, currently a Fellow at Harvard’s Berkman Center, writes at Crooked Timber about “Peer Review at Record Speed” – refuting the Facebook-grades correlation in a peer reviewed, open access publication in just a couple of weeks.

The enormous variation in these stories complicates everything we do, from collection development to instruction to supporting scholarly communications.  The need for speed, especially among those on the tenure-track, might be an untapped reservoir of support for open access online publishing. 

PS – Just in case you were feeling under-appreciated, see why mathematician Rudbeckia Hirta will never leave the academy. 

Enjoy the holiday weekend!