A “Befuddling and Often Capricious Crapshoot”

This is how Rick Montgomery characterizes the conduct of peer review in his front-page article in the Kansas City Star, Fraud Proves that Science Journals Can Be Fooled (1/14/06) (temporarily freely available online).

While Montgomery’s analysis of the peer review process is limited (e.g., focusing solely on the conduct of peer review for science journals), it is a good example of how issues in information literacy instruction and scholarly communication instruction sometimes cross over into the mainstream. The key points included in his spotlight box reflect some basics of what academic librarians have been teaching for years: 1) note the size of the study (i.e., apply an understanding of research methodology); 2) consider who paid for the research (i.e., look for potential bias); and 3) beware of claims made at scientific conferences (i.e., understand the nature of the scholarly communication and publication cycle to better appreciate the status of a claim in terms of peer review). The description of the time that goes into reviewing a mss. will also be enlightening to those not actively involved in the process.

Count on this article (and others that reflect the current scandal over the publication of results of fraudulent studies) to be a useful jumping off point for many instruction sessions to come.

Your Cheatin’ Heart

The New York Times Week in Review is awash in metaphysical questions this morning – what is truth? how do we know? The Million Little Pieces controversy is one of those moments in the news cycle when society seems to collectively pause to assess whether it’s been had or not. Randy Kennedy examines the public’s willingness to be entertained by “truckloads of falsehood in memoirs” and concludes that readers want redemption, not truth – and they want it packaged in confessional, reality-television mode. (Presumably the publisher anticipated that when suggesting to the author the novel he submitted be marketed as a memoir.) Memoirist Mary Kerr disagrees. “Distinguishing between fiction and non- isn’t nearly the taxing endeavor some would have us believe. Sexing a chicken is way harder.” She concludes redemption is cheap, if it’s too easy; writing a memoir is an attempt “to unearth life’s truths.”

The other fraud that’s getting the microscopic treatment in today’s paper is the breakdown of peer review in the case of Hwang Woo Suk, whose fraudulent research reports on stem cell research advances resulted in his paper being withdrawn by Science. Two articles in today’s Times examine the train wreck. Nicholas Wade points out that there are two kinds of science: texbook science (material that has been validated over time) and frontier science – “wild and untamed, and often either wrong or irrelevant to future research.” He urges fellow journalists “to recognize that journals like Science and Nature do not, and cannot, publish scientific truths. They publish roughly screened scientific claims, which may or may not turn out to be true.” In the magazine, David Dobbs says “The scientific-publishing system does little to prevent scientific fraud. Is there a better way?” He argues for “open-source” reviewing, where a paper is published with the comments of assigned reviewers and anyone else who cares to join the fray.

Open, collaborative review may seem a scary departure. But scientists may find it salutary. It stands to maintain rigor, turn review processes into productive forums and make publication less a proprietary claim to knowledge than the spark of a fruitful exchange . . . Hwang’s fabrications, as it happens, were first uncovered in Web exchanges among scientists who found his data suspicious. Might that have happened faster if such examination were built into the scientific process?

In this cynical age of spin, it’s refreshing to see the public grappling with the nature of truth and by what creative and scientific processes we arrive at it. There’s plenty of material here for an interesting lesson in information literacy.

MLA Engages Tenure, Scholarly Communication Issues

For those who haven’t followed the various reports from the recently-completed annual meeting of the MLA, there were interesting discussions of re-thinking tenure requirements for faculty in the Humanities and of the role of digital scholarship as a route to tenure. An overview can be found in today’s Inside Higher Education.

As with Andersen’s Digital Scholarship in the Tenure, Promotion, and Review Process (2004), the fact that this discussion is coming from within a discipline suggests an opportunity for the academic librarian to act as an advocate and a resource for innovation in scholarly communication.

John Willinsky on “The Access Principle”

John Willinsky of the Public Knowledge Project provides an interview on his new book, The Access Principle: The Case for Open Access to Research and Scholarship.

What is the “access principle”? In a nutshell:

“The access principle holds that with a form of knowledge that is constituted as a public good, which is the case with research and scholarship, the knowledge should be circulated as widely and publicly as possible, especially as that wider circulation increases the value and quality of that knowledge.”

Another Kind of Impact Factor

An interesting article by Eileen Gifford Fenton and Roger Schonfeld examines “The Shift Away from Print” journals in libraries and the unintended consequences this may have for small scholarly presses. We’ve been making choices about format based on a variety of variables, not least of them cost, and the authors have concluded that an electronic-only journal publishing environment would be most cost-effective. But the transition itself is costly for undercapitalized small publishers and the authors argue we need to be thoughtful about how our decisions might doom a worthy publication.

What isn’t considered, except glancingly, in this article is the impact of full-text aggregators such as Ebsco and Infotrac on users and libraries, the growth of the open access movement, or the fact that articles that can be found online are altering the relative status of publications. Easy access is having an impact on the “impact factor.”

Whether or not libraries continue to pay for print subscriptions, scholarly journal publishers may have to find ways to get online – if they want to be read.