Peer-Review

Thanks much to Jason for his recent entry rounding up some related thoughts at blogs around these parts and raising some interesting questions about their conjunctions. Some of my recent fretting figures into this round-up, in an ironic fashion: I worried about my sense that I was shouting down a well, and lo, but I got a response.

Nonetheless, I remain filled with anxieties (and perhaps this has something to do with the fact that my department met today to debate the Big Question looming in my academic life), a number of which revolve around the relationship between time-devoted-to-blog and need-to-produce-substantive-proof-of-scholarly-activity. I argued, in my personal statement, that the work I do here on* Planned Obsolescence needs to be taken seriously in terms of my research, as the site provides both a site of research, in which I can observe the development of new writing forms first-hand, and a sort of open forum, in which I can test out and receive feedback on the various ideas I’m in the process of forming. In that regard, the blog seems both virtual laboratory and ongoing conference, and needs to be taken seriously by one’s peers.

Ah, but the question of the peers raises the question of peer review. As Jason asks:

Can comments and trackbacks — in some fashion — lead to a sense of peer review (or do they already)? If not that, what kind of peer review could we imagine if it were facilitated by technology? Clearly, there are issues with ignoring the “blind” review process (and if blogs were the model, blind review would be nigh-impossible); E-Bay style procedures are unlikely to benefit academia, since reputation-based evaluation is hardly an effective measure in an environment that, like so many others, can burn you with a wayward comment. The superstars are unlikely to be criticized, and even the newbies will likely find themselves either coddled (because to do otherwise might ruin their career) or cut [again, the Chronicle has some interesting articles on these issues].

As Jason points out, the eBay system of evaluation has proven less than useful — an almost viral grade-inflation has resulted — and as the conversation that resulted from George’s post entitled “Conversation as Game” suggested, Slashdot-style “karma” systems of evaluation are both flawed and massively subvertible. One might argue that a trackback functions like a citation might, except that not all blogging systems trackback (and not all bloggers with trackbackable systems do, either). So what other kinds of peer-review/evaluation systems might we imagine?

*Rats. Forgot the footnote in the first build: is the proper preposition here on or in? One publishes in a journal, but appears on television. Yes, we find stuff on the web. But the phrase “on Planned Obsolescence” just rings too many “recently, on E.R.” bells for me.

5 thoughts on “Peer-Review

  1. I believe it is “on” a website. “In” a journal (or “in” an anthology), I believe, is a result of the material nature of the book object – the pages of your article are within the folded enclosure of the journal’s covers.

    Of course, I probably just made that up.

  2. To complicate: what if the journal (or newspaper) appears online? I read Paul Krugman’s editorial *in* today’s New York Times (but I read it on (?) their website). Hmmm…also, when referring to archives, one would more likely say “in” than “on.” George’s post on “Conversation as Game” was *in* his archives.

    Honestly, I have no answer here….

  3. I’d just say “the work I do here” and leave it at that. If I had to pick, though, I would say that “on” is correct. It just sounds better, ER aside. If you threw out every preposition based on its inclusion in a TV teaser, you would pretty much be limited to “whereby” and “thusly.” Well, maybe “the work I do herein” would work.

    PS – good luck on the Big Question; I’m sure it will work out.

  4. My take:

    You publish “through” and write “for”.

    You publish to reach an audience (and comments, track backs and hits) are crude measurements of the audience flow.

    You write to explore and to communicate. And possibly to establish a treasury very much like a commonplace book.

    I wonder if that committee can make the distinction? Career futures depend so much on a marketing model and visitor traffic. Instrinsic worth seems not even to register. Interesting how the why-question shades into a “for whom”…

    Couldn’t the blog be seen as an extra mural seminar that brings back rewards to the home institution? That could satisfy both the reputation-building preoccupations of the administration and the intellectual growth proclivities of the college of scholars.

  5. Thanks so much for the comments, all of you — and for waiting (I hope) for me to respond. I agree with those of you who suggest that “on” is the correct choice of preposition, but I’m leaning how toward Francois’s “for” and “through,” which I think indicate a kind of purposiveness and a mediation rather than a surface or a program, all of which I like. So I think I’ll start trying to employ those.

    As to your thoughts about peer-review, Francois — I like the idea of the site as an extramural seminar, and I very much share (what I believe to be) your sense that the traffic model of evaluation feeds the academy’s worst careerist modes of self-assessment. As much as we academics often critique mainstream publishing authors and venues for catering to numbers rather than ideas, in fact we maintain a certain variety of “bestseller” worship ourselves. (Of course the notion of the “bestseller” has different proportions within the academy, but there is nonetheless the star-following sense that if everyone’s reading something, it must be “good.”)

    So thanks for reminding me of the distinction between the writing and the publishing — and for making me think about which one is actually important to me.

Leave a Reply

Your email address will not be published. Required fields are marked *