Comments on: On the Future of Peer Review in Electronic Scholarly Publishing http://www.plannedobsolescence.net/on-the-future-of-peer-review-in-electronic-scholarly-publishing/ falling indelibly into the past Fri, 16 May 2008 23:34:01 +0000 http://wordpress.org/?v=2.5.1 By: Planned Obsolescence » Blog Archive » The Failure of Open Peer Review? http://www.plannedobsolescence.net/on-the-future-of-peer-review-in-electronic-scholarly-publishing/#comment-2407 Planned Obsolescence » Blog Archive » The Failure of Open Peer Review? Mon, 04 Jun 2007 20:14:28 +0000 http://www.new.plannedobsolescence.net/?p=170#comment-2407 [...] six months ago, I published a lengthy post, both on Planned Obsolescence and on if:book, about the future of peer review in electronic [...] [...] six months ago, I published a lengthy post, both on Planned Obsolescence and on if:book, about the future of peer review in electronic [...]

]]> By: KF http://www.plannedobsolescence.net/on-the-future-of-peer-review-in-electronic-scholarly-publishing/#comment-347 KF Mon, 03 Jul 2006 14:43:36 +0000 http://www.new.plannedobsolescence.net/?p=170#comment-347 <p>These are all good questions, and are the kinds of things we’re trying to hash out in imagining the network that we hope to build.  We can start, for instance, with an editorial board, each member of which has committed to a certain degree of active participation in the network.  But if we want to avoid replicating the structures of current peer-review systems, and instead build what might more properly be thought of as a “peer-to-peer review system,” we’ll need to find ways to recruit a far greater number of peers, by making their participation so attractive and valuable that they *want* to spend time in these discussions. </p> <p> And as you say, the metrics we currently have available for determining who counts as a “top expert” in any given field are pretty flawed, as are the slashdot-type ratings that might be imagined to replace them.  But we’re hoping that, as the discourse in this network grows and develops, that the community of scholars itself will find new ways of calibrating “impact.” </p> These are all good questions, and are the kinds of things we’re trying to hash out in imagining the network that we hope to build.  We can start, for instance, with an editorial board, each member of which has committed to a certain degree of active participation in the network.  But if we want to avoid replicating the structures of current peer-review systems, and instead build what might more properly be thought of as a “peer-to-peer review system,” we’ll need to find ways to recruit a far greater number of peers, by making their participation so attractive and valuable that they *want* to spend time in these discussions.

And as you say, the metrics we currently have available for determining who counts as a “top expert” in any given field are pretty flawed, as are the slashdot-type ratings that might be imagined to replace them.  But we’re hoping that, as the discourse in this network grows and develops, that the community of scholars itself will find new ways of calibrating “impact.”

]]> By: Carol Anne Meyer http://www.plannedobsolescence.net/on-the-future-of-peer-review-in-electronic-scholarly-publishing/#comment-346 Carol Anne Meyer Fri, 30 Jun 2006 18:18:20 +0000 http://www.new.plannedobsolescence.net/?p=170#comment-346 <p>This is a very thoughtful analysis of the role of peer review, and one of the first I’ve seen emphsizing the differences in peer review in the humanities and the sciences. </p> <p> I wonder, will “ten of the top experts in my field” have time to participate in open peer review.  One of the serious problems with the existing peer review system is the difficulty in getting reviewers to submit reviews in a timely fashion. </p> <p> Also, how will we determine the top ten experts?  We use publication record and impact factor now to make many of those determinations.  These metrics are not without their problems.  What will replace them?  Number of downloads, number of comments, slashdot-type ratings?  Hmmm. </p> This is a very thoughtful analysis of the role of peer review, and one of the first I’ve seen emphsizing the differences in peer review in the humanities and the sciences.

I wonder, will “ten of the top experts in my field” have time to participate in open peer review.  One of the serious problems with the existing peer review system is the difficulty in getting reviewers to submit reviews in a timely fashion.

Also, how will we determine the top ten experts?  We use publication record and impact factor now to make many of those determinations.  These metrics are not without their problems.  What will replace them?  Number of downloads, number of comments, slashdot-type ratings?  Hmmm.

]]>