How relevant is the peer review process in the digital age? In our fast-paced world of instant Twitter, innumerable and often-illuminating blogs, comprehensive wikis, and insightful electronic magazines, does peer review still have a place? Does it help imbue scholarly e-books and e-journals with resonance, heighten quality, and encourage objectivity?
Robert Townsend’s article about the Future of Peer Review on the American Historical Association’s web site is well worth reading for its thoughtful analysis of both the challenges of traditional peer review in the digital context, as well as some goals for perfecting the system.
It’s a long, fairly grueling process to get a peer-reviewed article published in a scholarly journal, even in a predominantly e-journal. I’ve been thinking about this as I completed the process for my article, The Progression of Digital Publishing: Innovation and the E-volution of E-books. The article touches on peer review in the context of an ongoing, digital resource and regarding digital textbooks, but it’s not the subject of the article, which focuses on innovative electronic texts. At the end of the publication cycle, however, I started to wonder about the balance between the rigor of the process and its pace. I wrote the article in November 2009, expanding on presentations I’d given in October. After review, revisions, and approval, I submitted it to the International Journal of the Book in mid-January, whence it journeyed merrily through double-blind peer review (contributors also volunteer to be a reviewer for the journal), a bit of revision and updating in April (Apple’s iPad was now a reality instead of a rumor), typesetting, and finally, in June, publication. Toward the end, I must confess, I was a bit winded, wondering if proceedings would ever come to an end (having worked with hundreds of authors over the years, I know I’m not alone in this feeling). Seven months, more or less, which is fairly rapid for a journal, so I’m not complaining.
This article isn’t about a new scientific breakthrough or method, obviously, or a gathering and manipulation of data, it’s nothing particularly controversial. I am not a professor, so the self-interest of tenure, which may steer some scholars onto the peer-review highway, is not at all applicable in my case. Both this article and my previous article on the subject were conceived and executed primarily for intellectual exercise. Peer review, one expects, contributes to accuracy, rigor, and, I suppose, legitimacy, and these motivated this scholarly pursuit.
Michael S. Gazzaniga, in Human: The Science Behind What Makes Us Unique, says “to separate the verifiable from the nonverifiable is a conscious, tedious process that most people are unwilling or unable to do. It takes energy and perseverance and training. It can be counterintuitive. It is called analytical thinking. It is not common and is difficult to do. It can even be expensive. It is what science is all about. It is uniquely human.” (p. 273-274).
Peer review is not flawless, nor is research. For an example, Jim Lindgren’s Yale Law Review article, Fall from Grace: Arming America and the Bellesiles Scandal, gives an often quite fascinating account of the twists and turns of the academic scandal involving Michael A. Bellesiles’ book on the history of gun control in America. Bellesiles’ book was awarded the prestigious Bancroft Prize, subsequently rescinded, for the first time in the award’s history, after flaws, and perhaps even fraud, in the author’s methodology came to light. Lindgren mentions that the entire scandal may have been avoided through better editing (and presumably better peer review) at the Journal of American History, which published Bellesiles’ original article on the subject. The book was published by Knopf, which while assuredly not an academic publisher, one would expect to nevertheless have an interest in publishing accurate books.
A paper by the President of the RAND Corporation, James A. Thomson, about the increasing polarization in our society, goes beyond peer review to the concept of bulletproofing: “By far the most important is the quality assurance (QA) process, especially the concept of ‘bulletproofing.’ To some of us who were trained to believe that the most important part of the QA process is the scientific peer review, this can sometimes be an alien concept. Of course, the scientific peer review is the sine qua non; the science must speak. But if controversy lurks, bulletproofing is essential. This involves thinking in advance about the political lines of attack against the results and then identifying individuals who might come from those political quarters. Such individuals should be brought into the review process.”
Dan Cohen, in a blog post about the social contract of scholarly publishing, posits that an exchange of analysis for attention, as it were, is central to academic value and reward. He underscores how the supply side of this social contract—writing, peer review, editing, publishing—and the demand side—the space for attention and consumption—must be aligned, and wonders if these may be slipping out of gear in the digital age.
What about a Threadless-style peer-review system—treating academic articles like crowd-sourced designed, limited edition t-shirts? (Threadless, in my opinion, has produced one of the web’s most successful and innovative mashups of creativity, social media, and e-commerce.) Is an e-book model a la Smashwords feasible for academic monographs and articles?
There are already numerous examples. Rice University’s Connexions, for example, is a digital textbook platform that accepts everything. Their system of peer review is the philosophy that the cream rises to the top, that better modules will be rated higher and therefore get more use, a self-reinforcing system. MERLOT (Multimedia Educational Resource for Learning and Online Teaching), is a community designed to share peer-reviewed online teaching and learning materials.
Brian Whitworth and Rob Friedman lay out in a First Monday article the tenents of an online Knowledge Exchange System that could reinvent academic publishing with a system that accepts all, reviews all, and publishes all. This system would allow for both anonymous expert ratings as well as general reader ratings in order to increase both dissemination and discrimination, both rigor and relevance. User ratings could be broken down by criteria like relevance, rigor, writing, comprehensiveness, logical flow and originality.
These new systems of scholarly publication, peer review, and dissemination won’t come easily, necessarily—the central question remains as always “who pays for this?” not to mention “who has the time for this?” Yet a hallmark of this digital age is experimentation, exploration, and upheaval.
We live in interesting times.
