Science blogs and public engagement with science

ResearchBlogging.orgAs you may know, I love the Journal of Science Communication. It publishes some very interesting and useful scholarly articles on a wide array of issues pertaining to the communication, education and publishing of science. I wish more science bloggers (and non-blogging scientists) read it and blogged about their articles. Unfortunately, human nature being as it is, most of the excellent papers go by un-noticed by the blogosphere, while an occasional sub-standard paper gets some play - it is so much easier to critique than to analyze or even praise.

One such paper is now making the rounds - it is mentioned on Science of the Invisible and discussed at length (not badly, mind you) on The Scholarly Kitchen. The article in question is Science blogs and public engagement with science: Practices, challenges, and opportunities, Journal of Science Communication, 9 (1), March 2010, by Inna Kouper, a graduate student in library and information science at Indiana University. The journal is Open Access and this article is now published so you can download the free PDF with a single click. Go for it, you'll need it if you want to read along with me.

First, let me get the Conflict Of Interest out of the way. I am on the Editorial Board of the Journal of Science Communication. I helped the journal find reviewers for this particular manuscript. And I have reviewed it myself. Wanting to see this journal be the best it can be, I was somewhat dismayed that the paper was published despite not being revised in any way that reflects a response to any of my criticisms I voiced in my review.

So, let me walk you through the big chunks of the paper, adding the critiques that I voiced during the review process. I will have additional commentary at the end of the post as well.

Digital information and communication technologies (ICTs) are novelty tools that can be used to facilitate broader involvement of citizens in the discussions about science. The same tools can be used to reinforce the traditional top-down model of science communication. Empirical investigations of particular technologies can help to understand how these tools are used in the dissemination of information and knowledge as well as stimulate a dialog about better models and practices of science communication.

With the Internet being over 26 years old, the World Wide Web 19 years, and blogs 12 years, I don't think it is correct to still, at this day and age, call ICTs "novel".

This study focuses on one of the ICTs that have already been adopted in science communication, on science blogging. The findings from the analysis of content and comments on eleven blogs are presented in an attempt to understand current practices of science blogging and to provide insight into the role of blogging in the promotion of more interactive forms of science communication.

Analysis of blogs has been done before, so this article needs to focus on what new it brings to the literature - the analysis of comments.

========================

So far the discussion about science blogs develops primarily in the form of journalistic and scholarly commentary rather than research-based analysis. It focuses on what blogs can and cannot do and why blogging can be a promising tool for scientists (Butler, 2005). Most often the analysis relies on a few examples of science blogging and uses these examples to contextualize general considerations and descriptions (Wilkins, 2008). To better understand challenges and opportunities science blogs can bring, it is necessary to analyze current practices of science blogging. To date no attempts have been made to do that. The present study is the first step in this direction.

Together with Wilkins 2008, this paragraph should also probably cite Goldstein 2009 which did a similar analysis (including even some of the same blogs as used in this paper). This paragraph should also accentuate the analysis of comments to differentiate it from other papers that have analyzed blog posts alone.

==========================

The data for this study consist of posts and comments from eleven blogs that write about science and technology. The blogs were sampled via the Internet search for "science blogs" and "blogs about science" and by following scientific news on the moment of data collection in Spring, 2008. Below is the list of blogs with their titles and URLs from which the posts and comments were sampled:

This needs to be clarified. Internet search for "science blogs" and "blogs about science" brings up thousands of blogs (some of which are not science blogs at all). How were these particular 11 chosen? What search method was used: Google Blogsearch, Google Web Search, Technorati, other?

================================

This is an interesting collection (see the Table). It is, first, very small, thus missing some important subsets of the science blogosphere (medblogs, nature blogs, skeptical blogs and, importantly due to cluster analysis by Christina Pikas, the female science bloggers which have a very different pattern of both posts and comments). All or most of the authors of these 11 blogs are white males, which also affects the analysis. A number of these blogs are multi-author, with each author having a different style and blogging mode (Note: the Table was modified for publication, adding the number of authors per blog, but no discussion of the importance of this appeared in the text). Please note here, up front, the potential drawbacks of your sampling methods.

Before sampling blogs were examined for posting activity. As it was determined that some blogs posted one or two messages per week and others posted several messages per day, it was decided to save 30 days of activity from less active blogs and five days of activity from very active blogs. For feasibility of qualitative analysis, the number of comments was limited to 15 comments per post. Overall, 174 posts and 1409 comments from 11 blogs were saved and analyzed.

Please justify the cut-off at 15 comments. On busy blogs like Pharyngula, the first 15 comments are likely to be quick one-liners while deeper discussions happen later, once readers had sufficient time to read and digest the content of the post, often with long, well-informed comment threads that go on for hundreds of comments per post.

================================

The findings suggest that science blogs are too heterogeneous to be understood as an emerging genre of science communication. The blogs employ a variety of writing and authoring models, and no signs of emerging or stabilizing genre conventions could be observed. Even though all blogs mentioned science or a particular scientific discipline in their descriptions, they differed in their voice representations, points of view, and content orientation. Some bloggers emphasized the first person perspective and presented themselves through religious and political affiliation (e.g., "The blog is about whatever we find interesting" at Cosmic Variance or "Evolution, development, and random biological ejaculations from a godless liberal" at Pharyngula). Others shifted the focus from their personalities to the content and featured more neutral forms of presentation (e.g., "... the latest news about microbiology" at MycrobiologyBytes or "... your source for news and commentary on science" at The Scientific Activist). Differences in sources, topics, and modes of participation among blogs are discussed below.

The small and thematically narrow sample of blogs limits the value of this paragraph. What is in an "About Us" section may have been written years ago and never revisited although a blog has evolved in a different direction in the meantime?

===========================

Personal experience and news from other media were used to discuss predominantly non-scientific, often political, matters. Thus, the blog authors commented on the issues of sports doping (Pure Pedantry), children's medical care and religion (BioEthics), creationism versus evolution disputes (Panda's Thumb), US presidential elections (The Scientific Activist), and the life of the former Serbian president Radovan Karadzic (Cosmic Variance). Other examples of using experienced events as sources of blog posts included reporting about conferences and public lectures, commemorating events from the past, or noticing the appearance of new material on the web.

Radovan Karadzic was never any kind of official in Serbia. Before the unification of Bosnia, during the war, he was the president of the self-proclaimed enclave of Bosnian Serbs called Republika Srpska which was never, officially or unofficially, a part of Serbia proper.

=================================

As can be seen from the figure above, science blogs cover a variety of issues and topics beyond science. Among the topics related to science the most frequently covered topics were evolution, health, and space. The prominence of the topics of evolution and creationism can be explained by the dominance of two highly prolific blogs Pharyngula and Panda's Thumb, which consider the promotion of evolutionary theory as their main focus. Among other scientific topics bloggers discussed genetics, physics, and biotechnology. More often, though, science bloggers discussed what has been posted on other blogs and websites and reflected on the practices in academia, on their and others' blogging, and on the issues of their personal life.

The range of topics seen suffers from the small sample of blogs. A different sample (e.g,. if all the blogs were sampled from Nature Network) would result in a completely different word cloud.

======================

Each larger group of participation modes was equally noticeable in the sample, therefore it is difficult to claim that one form of communication or the other is more common for science blogs. Being a more fluid and personal genre of communication, blogs allow for greater variability of expression, and it seems that the authors of science blogs eagerly utilize this fluidity and variability. It was observed though, that certain blogs favored one mode of participation more than others.

Do you have numbers, percentages? Can you provide a complete dataset of raw data so others can reanalyze?

===========================

The writer of this post freely interpreted the findings of the study and substituted alcohol-containing nectar mentioned in the original research with beer. This way the news becomes more entertaining, yet it may prevent the readers from getting accurate information and forming their own opinion, thereby making it difficult to rely on this form of reporting as a source of accurate information.

Potential explanation: Wired Science blog is an official blog of a magazine and most Wired bloggers are trained journalists - this may explain a number of differences seen between Wired Science and other blogs.

===============================

"Antimatter is just like ordinary matter in every way, except that every quantity you can think of (apart from mass and spin), is reversed. As an example, the electron is a particle with a specific mass and carrying a specific amount of negative electric charge. The antiparticle of the electron is a positron, which has the identical mass to an electron, but precisely the opposite charge. The thing about particles and their antiparticles is that, if one puts them together, the net value of any quantity (called a quantum number by physicists) carried by the pair of them is zero. Therefore, a particle and an antiparticle together are merely mass which, thanks to Einstein's E=mc2, can be converted entirely into energy. As a result of this, when matter and antimatter come together, they annihilate, producing energy in the form of light (photons)."

As you note later, most readers are scientists. Physicist tend to read physics blogs. Thus, the author has correctly identified his audience and is writing at the level expected from his audience. Other posts on the same blog may be more directed towards lay audience. Also, John Wilkins has collected a large number of 'Basics Posts' written specifically for lay audience by a large number of science bloggers over a period of almost two years.

===============================

Emotional and often insulting evaluations are very common for this and some other blogs that seem to be eager to demonstrate not only their rightness, but also to distinguish their group of reasonable and worthy individuals from others, who are wrong, unintelligent, and overall worthless. The frequency of such evaluations and mockery undermines the goals of rational debate and criticism. Such activities can foster solidarity among the like-minded individuals, yet at the same time, they may spur hostility in those who are undecided or hold a different opinion.

This statement (last 2 sentences) is often repeated but has never been studied and does not have, thus, empirical support. While alienation of the 'opposing side' is likely, it does not make a difference as the 'opposing side' is regarded as 'unmoveable' and is not the target audience. The undecided, on the other hand are a big unknown and there are some indications that they are likely NOT to want to join the side that is mocked.

Less complicated common forms of author participation in science blogs included announcements and summaries of documents. Announcements publicize events and sources of information (e.g., "The Kaiser network is hosting a live webcast to discuss the influence of the blogosphere on health policy" or "Tonight, on the History Channel... It's the much anticipate first episode of a new series, Evolve - Eyes"). Summaries provide elaborate descriptions of research papers and essays and often use very specific terminology such as dorsolateral prefrontal cortex, which would require the reader to have some background in a particular field. While such summaries somewhat popularize the content of research papers, the amount of minimal background necessary for a lay person to understand and follow the research news varied among different science blogs.

Most bloggers write for their own amusement and not with a specific goal of popularization of science, and, after a while, tend to adapt to what their audience actually is. Thus, a knowledgeable audience will result in further posts being written at their level of interest and understanding.

Readers of science blogs also had some relationship with science, i.e., they were not exactly non-scientists or lay persons. One author posted a message titled "Who are you?" and asked his readers for information about themselves and their background. The answers to this post as well as the overall analysis of readers' comments demonstrate that the readers are almost always associated with science one way or another. They are graduate students, postdoctoral associates, faculty members, and researchers from a variety of scientific and research fields including biology, physics, neuroscience, and medicine. Wired Science was probably the only blog in the sample where non-scientists formed a considerable portion of the audience. Nevertheless, even in this blog commenters often took the position of authority and talked as experts who are quite knowledgeable about the subject.

Remember again that Wired bloggers are journalists.

============================

After this comment a thread of comments developed defending or criticizing Barack Obama and his approach to science, religion, politics, and so on. These comments were completely unrelated to the topic of Louisiana creationism law provided by the blog post.

It is important to note the history of these blogs. Wired Science is a blog owned by a media company. Media in general, due to a bad case of misreading of an old legal case, tend not to moderate their comments. Unmoderated comment threads tend to get unruly and attract trolls and hit-and-run comments. Panda's Thumb evolved out of an old Usenet group, where the conduct is traditionally different than on modern blogs. This is also a group blog with minimal moderation. Pure Pedantry was a relatively small blog, but a Britney Spears post got on digg.com and most of the comments they got after that are one-time hit-and-run visitors from Google searches, not the regular commenting community of the blog. On the other hand, Pharyngula is a carefully moderated blog - community votes for the Commenter of the Month (the "Molly") to reward intelligent contribution and PZ Myers has over time banned several disruptive commenters (whose names are listed on his blog, as example). He will sometimes personally interfere - by deleting and by commenting himself - if someone is disruptive. As a result, Pharyngula is a community of commenters. They tend to talk to each other much more than to Myers. To some extent, but not as much as on Pharyngula, commenters on Panda's Thumb, Cosmic Variance and perhaps Wired Science, may be seen more as a community that talks among themselves than commenters addressing the owner of the blog.

======================

Insults, such as "Don't be an idiot.. rtfa" or "Could you possibly sound any more stupid with this comment?" were more common for some blogs than the others. Thus, Wired Science and Panda's Thumb were filled with insulting commentary. Offensive remarks regarding somebody's personality or intellectual abilities most often targeted other commenters and the characters of posts, but sometimes they were directed at blog authors as well, such as the following comment in DrugMonkey blog: "You are correct, I never read a post in which you claim not to be pompous and arrogant".

See my commentary above about the importance of the history of individual blogs and the importance of moderation policies. Also worth noting in this example is that DrugMonkey blog is written by two authors, one of which (the one I presume was addressed in the comment you quote) is PhysioProf who very effectively uses profanity to get readers out of their comfort zones, with predictable responses.

In addition to personal attitudes and obvious digressions, where commenters would take an element from a blog post and develop it into an independent topic of conversation, a large portion of comments offered humorous and sarcastic remarks. Thus, the Wired Science post about nuclear weapons as a way to destroy asteroids got the following comments among others: "Got Bruce Willis?", "You don't want to destroy or deflect comets or asteroids, you want to capture and harvest them...", and "Like the SF writers of yore knew: Resistance is Futile".

Again, keep in mind that Wired Science is a corporate/media blog, written by journalists, with almost no comment moderation. Thus the Wild West feel of their comment threads is to be expected - it is more like YouTube than a blog in regard to expected commenting behavior. This usually does not happen on personal blogs.

=====================

Science blogs examined in this study are very heterogeneous. They provide information and explain complicated matters, but their evaluations are often trivial and they rarely provide extensive critique or articulate positions on controversial issues. Kenix (2009) analyzed political news blogs as alternative news sources and found that the blogs offered binary, reductive analysis and dependent reporting. She also found that readers often provided caustic commentary and argued that comments can be considered a separate communicative sphere more akin to a neighborhood bar than to the Habermasian public sphere. It appears that science blogging can also be characterized as relying on reductive analysis and dependent reporting and drawing caustic and petty commentary.

Small sample, omission of blogs that almost entirely write posts for ResearchBlogging.org aggregation (eg, Not Exactly Rocket Science, Tetrapod Zoology, Neurotopia, Neurophilosophy), omission of highly technical blogs which are a center of that discipline's online community (e.g., Sauropod Vertebra Picture Of The Week, or Deep Sea News) and omission of some of the blogs with the most developed feelings of community - the female scientist blogs and Nature Network blogs, makes these points moot. This is akin to analysis of political blogs and omitting Firedoglake, Talking Points Memo, Huffington Post and Hullabaloo - the blogs that do heavy lifting, independent reporting, expert analysis, etc. Many such blogs exist in the science blogosphere but they were not included in this paper.

In their current multiplicity of forms and contents science blogs present a challenge rather than an opportunity for public engagement with science. Lack of genre conventions, which for the audience translates into broken expectations and uncertainty, impedes the development of stable readership and participation from the larger public. The "neighborhood bar" or "water cooler" commentary creates a sense of community with shared context and culture, but at the same time it creates a barrier that prevents strangers and outsiders from joining the conversation. As a community of scientists or individuals close to science, the existing readers may enjoy the entertaining nature of science blogs and not need science blogs to serve as a place for discussion and rational debate. Relying on such community of readers, bloggers may reduce their interpretive activities and resort to copying, re-distributing, and re-packaging of the existing information, which is still quite rewarding given the background of the majority of current readers and yet requires much less time and effort.

Blogs are technological tools, platforms. They can be used by corporations and organization for PR and news delivery, but that kind of blog does not attract much audience. Most blogs are personal blogs. It is the personality of the owner, combined with her/his expertise, that draws in the audience. A personal blog is a personal space for personal expression. Bloggers are likely to strongly resist any attempts by any group to influence the way they spend their free time conversing with friends online. In other words, they are not meant to be vehicles for science engagement with the public by design, but they serve that function very well precisely because of the personality of the blogger, (often self-deprecating) humor, often juicy language, and strong opinions. Scientists are supposed to be cool-headed, anti-social recluses - blogs show they are anything but, break the stereotypes and show the humanity of scientists. With this, comes the trust. And science engagement is all about trust - not the memorization of knowledge of scientific trivia.

=============================

This study has a number of limitations. The study is based on a limited sample, and the applicability of its findings and conclusions needs to be tested further. The findings can serve as an initial step in the investigations of the relationship between science blogging and public engagement with science and in the development of the taxonomy of modes of participation. Due to the small number posts and comments, certain important modes of participation could have been overlooked. A more elaborate taxonomy of participation modes could serve as a basis for further genre analysis of science blogging. The role of humor in science communication and collective interpretation of knowledge also needs to be examined. Finally, the study would benefit from extending the analysis to lurkers, i.e., those readers who follow the content but do not post comments.

These limitations should be stated at the beginning of the article as well as here.

===============================

So, this article was supposed to be the analysis of comments on science blogs, but did not actually study comments - it studied a tiny and unrepresentative sample of blogs, one of which is dead (Pure Pedantry) and thus slowly accuulating unmoderated spam comments.

I think it is important to read this article, as well as my commentary, in the light of recent discussions on The Intersection and Bioephemera.

Five years ago, I read every science blog in English language. I could, as there were only dozens of us. The science blogosphere was small and tight at the time. But remember where these blogs came from - they evolved out of political, atheist and skeptical blogs. There was 'Intersection' where Chris Mooney was collecting material for "Republican War on Science", there was 'Deltoid' fiercely fighting against Global Warning denialism, there was 'Pharyngula' providing a voice for atheists who until then thought they were alone (and who were then, after a series of anti-religious rants, delivered to some of the best written science posts ever, over and over again), there was my blog 'Science and Politics' where politics posts outnumbered the science posts at least 9:1. Not much more. Most science blogs were primarily focused on something else - politics, religion, skepticism, etc. - than on science. In many ways, early science blogs were really political blogs with a scientific twist.

Today, there are thousands of science blogs. Most of them are really science blogs - covering science in every, or almost every post. The ratio of science:other topics is much, much higher today than it was then.

I think everyone who focuses primarily on the old blogs, the same Google Reader list one had in 2005 without having it revised in the intervening five years, has no grasp of the current science blogosphere. Check out all the blogs registered at ResearchBlogging.org for starters. See the blogs on German, French Canadian, Brazilian and New Zealand science blogging networks, on Nature Network blogs, Nature Blog Network and Scientificblogging.com. Heck, if you ignore five or six blogs here on scienceblogs.com that are mainly focused on non-scientific topics and look at the remaining 70+ blogs - that's ScienceTM! Five years is eons on the Web. Any analysis of blogs and/or comments that is still in the 2005 mindset is missing everything.

Also remember that what was once a homogenuous, tightly knit group has split. There are now separate medical blogosphere, atheist blogopshere, skeptical blogosphere, birding blogosphere, green blogosphere, nature blogopshere, etc. All of those were once part of a single group. Each now has its own group, its carnivals, its unofficial leaders, its histories and customs and tone. Judging the science blogosphere by a few examples of ancient blogs that have changed a lot over the years, using old personal impressions about them to state what they are now, is misguided. There is plenty of blogs now for everyone's taste. Nobody is forcing you to read a blog that offends you. Move on, find blogs you like - there are so many good science blogs around today, there is not enough hours in a day to read them all even if you limit yourself only to those that do not blemish your thin skin.

Update: Cosmic Variance and Panda's Thumb, two blogs analyzed in this paper, chime in on it as well. And then DrugMonkey and David from The Atavism also comment.

Inna Kouper (2010). Science blogs and public engagement with science: Practices, challenges, and opportunities Journal of Science Communication, 9 (1) Link

Update: Of course, this discussion is nothing new, either in science blogosphere, or in blogosphere as a whole. We have had, over the years, many (some heated) discussions on related topics: what makes a blog a 'science blog', what good it is, what is a purpose or goal of a science blog, why should one blog, is it good or bad for one's career, as well as topics that in some ways touch on this, e.g., framing science (which we do on blogs and as it relates to science outreach), pseudonymity vs. anonymity, the whole quesiton of 'tone' on blogs, not to forget the most recent "carpet" discussion about comment moderation. So this is nothing new to science blogs and much of what some bloggers said over the years is much more detailed, thoughtful and even scholarly than what is in this paper. Not to mention that some very similar discussions also occured in other blogospheres: academic, techie, political, feminist, etc. Here, I just want to give you a sampling of some ancient posts that most directly address the questions of this paper, all posts being between two and four years old:

Science Blogging - what it can be
Blogging in the Academy: Batts et al, 2008
How Many (Science) Blogs Are There?
How much science does a science blog blog ....?
Role for Science Blogging
A postcard from academe: my tenure dossier.
Advancing Science through Conversations: Bridging the Gap between Blogs and the Academy
Bridging the blogging gap
Where will science blogging go from here?
The Value of Science Blogs
Advancing Science Thru Blogging
Feedback on 'Advancing Science Through Conversations'

Update: additional commentary by Janet, Dr.Isis and Bluegrass Blue Crab provoked some good comment threads as well.

Tags
Categories

More like this

Well. There's another paper out discussing science blogs, which is a good thing, I suppose. I just find the conclusion a bit disappointing. Bora has an exhaustive dissection, and both The Panda's Thumb and Cosmic Variance have briefer (they'd have to be! Bora got loquacious) discussions of the…
From quite early on in my blogging endeavor, I was interested in exploring science blogging, what it is, what it can do, and what it can become. So, check out some of my earliest thoughts on this here and here. Then, over about a month (from April 17, 2006 to May 17, 2006) I wrote a gazillion…
A blog is software. Importantly: a blog is free software. Everyone can use it in any way they want. If there are 100 million blogs out there, there are 100 million blogging styles and 100 million ideas what blogging "is". And anyone who dares tell others how to do it incurs the wrath of the other…
One difference between reading Open Laboratory anthologies and reading the original posts included in them is that the printed versions are slightly edited and polished. Another difference is that the Prefaces and Introductions can be found only in the books. They have never been placed online. But…

I'd be happy to just see ScienceBlogs have more actual hard science content. There's a number of regular bloggers now who just seem to use this site as their personal toilet for venting themselves. But I guess it's all about getting the page hits, just like TMZ or Twilight fan sites or World Nut Daily.

You are looking at wrong blogs, I guess. There are 80+ blogs here, MOST of which cover science, all science, only science and nothing but the science. Why are you focusing on blogs you don't like? Masochism?

Lovely dissection Bora. I especially love the bit in the middle when you take a break from correcting the dubious research methods and conclusions and correct their faulty knowledge of Serbian political history too.

That is...not a good paper. In fact, it's quite bad. It's a sorry excuse for content analysis with no clear direction. She could get a certificate in advanced tone trolling from M&K U., though.

So, this article was supposed to be the analysis of comments on science blogs, but did not actually study comments

It's even worse. As you note, the "conclusions" are mainly about the supposed effects of the bloggers' (commenters') style and emphasis on some alleged audience. These are - again, as you note - not demonstrated, but merely assumed.

And who is she to say what's properly science content?

No, not good at all.

I agree with your comments (I'll bet that's a relief). I think the sample was so small and undistributed, and the comments so relatively cavalier, that it is really pretty useless. Except that it (and your comments) did cause me to rethink my own blog, which I claim is targeted at a non-expert but literate audience. To limit the amount of reading I had to do, I chose three controversial subjects: evolution, climate change, and nuclear energy/waste management. A fourth category, and the part I especially hoped to concentrate my own writing to was a critical analysis (critique) of scientific writings intended for my targeted audience. Unfortunately, I have spent nearly all of my time finding and reading and posting articles. I have had trouble getting much of any commentary flowing so far, and am considering dropping the nuclear area because of a lack of interest in my (admittedly limited) readership. So the article with your commentary had the salutary effect of causing me to rethink my own blog.

Yes, it is appropriate.

Nobody can publish my comments without my consent, but I can do it myself.

I was not the editor of the paper in the sense people usually think of. I did not communicate with other reviewers, I did not see other reviewer's notes, I do not know if they suggested acceptance, acceptance with minor/major revisions, or rejection, and I was not a part of the decision to accept or not. I suggested major revisions, then a second round of review. That did not happen, so I am free to post my comments - both those I made at the time and additional comments.

I also have a little bit of gray area here. This being the first manuscript I did for JCOM I did not fully understand my role. I was asked to suggest some names for reviewers (I have no idea who reviewed in the end). I did not understand that this is where my job ends, so I sent in my review as well. It is, thus, an unofficial review. They (editors) read it, but they did not need to use my comments if they did not want to do that. This was more like sharing my thoughts with them. But the comments being unofficial makes them even more 'mine' and free to post online.

Uh, Smoky, I think there was a typo in your comment which I believe should have read "PZ Myers is a big, meanie, poo-head and I hate him . . . times infinity." Spell check just doesn't catch those things.

BTW, great post Bora.

By Pareidolius (not verified) on 09 Mar 2010 #permalink

"...sampled via the Internet search..." is that verbatim? Wow, that should have been a rejection right there.

By Stephen Wells (not verified) on 09 Mar 2010 #permalink

When I wrote a paper (just an interview, really!) about science blogs in 2006, the reviewer comments said that the topics "not new". It still got accepted (pdf, different journal), but FOUR years later I really expect much more depth from a piece about science blogs.

I don't agree with this comment of yours, though: "The science blogosphere was small and tight at the time. But remember where these blogs came from - they evolved out of political, atheist and skeptical blogs."
I was also blogging five years ago (my first science blogs just turned five last month) and my blog didn't have that background at all. Neither do several of the other blogs I read that came up within that year, so it was definitely not all based in politics/atheism/skepticism in the early days.

Pharyngula is a carefully moderated blog

Whoa; definitely not true except for the elimination of commercial spam. In fact it's downright silly; I have posted many things on Pharyngula that few other bloggers will put up with. Despite the bleating of trolls, PZ Myers is very tolerant about allowing postings -- which does not mean that he's tolerant in the sense of withholding criticism.

community votes for the Commenter of the Month (the "Molly") to reward intelligent contribution

Irrelevant to the issue of moderation.

and PZ Myers has over time banned several disruptive commenters (whose names are listed on his blog, as example)

A very small number, hardly counting as "carefully moderated". Also, they aren't listed "as example" but rather for information.

He will sometimes personally interfere - by deleting

Very rare; he also sometimes disemvowels comments, again very rare.

and by commenting himself

as do you, as do most bloggers, but this has nothing to do with moderation.

- if someone is disruptive. As a result, Pharyngula is a community of commenters. They tend to talk to each other much more than to Myers.

That is in no way a result of "careful moderation", which in fact does not occur. Notably, when trolls show up, the community of commenters talks to them, rather than to PZ or about PZ's post.

By truth machine, OM (not verified) on 09 Mar 2010 #permalink

@truth machine, OM: Moderation does not mean a technical fix or deletion. Moderation means that the blog owner sets and controls the tone of the comments, sets the criteria for what is "over the line" and builds a commentariat community that enforces his rules.

Eva/Janet - you are right, I was overgeneralizing from personal experience. Thanks.

Nice moving of the goal posts. To the degree that he does that, it certainly is not via "careful moderation"; you're just getting sillier.

By truth machine, OM (not verified) on 09 Mar 2010 #permalink

No, you are trying to move goalposts because you don't like PZ or something. I just defined moderation for you and PZ does it. If you don't like him, that is irrelevant. You don't get to define what moderation is by the 'pornography rule' (when I see it and don't like it, it's not moderation).

"you don't like PZ or something"

Yeah, that's why I complimented him on his tolerance, you dishonest fool.

By truth machine, OM (not verified) on 09 Mar 2010 #permalink

a) I am not tolerant like PZ.

b) I am firmly in the 'don't piss on my carpet' camp. This blog is read by my mother, brother, wife, son and daughter. If you write a comment here, first think to yourself "would I say that in his living room, in front of his family?"

c) Check out the title of this post, re-read the post itself, and the comment thread up to the point you arrived? Noticed something? This post is not about Pharyngula.

d) There is only one appropriate place to discuss Pharyngula - on Pharyngula. I have seen this happen in many places - people coming in and changing whatever topic to start discussing Pharyngula. That will NOT happen here. PZ is a friend, but my blog is a place where I dictate topics. Discussions of PZ's moderation techniques are NOT the topic of this post.

e) Several bloggers I greatly admire for their ability to keep substantive commentary going on their blogs and sticking to the topic would have already deleted yours for being off-topic.

f) I am nicer - my rule is "three strikes and out". Be very careful what you say in your next comment, as that may be your last. Go back to the topic and word your comments carefully - you do not want to give an impression of some fly-by online loonie dropping in just itching for a fight, do you?

g) If you don't like this, tough. This is my blog. My rules. Start your own blog if you want to respond. If you feel unwelcome here, you did that to yourself.

it studied a tiny and unrepresentative sample of blogs, one of which is dead (Pure Pedantry) and thus slowly accuulating unmoderated spam comments.

The study is based on a limited sample, and the applicability of its findings and conclusions needs to be tested further.

So ... why should anyone care about this study?

Circle jerk.

Um, your assertion that Pharyngula is "carefully moderated" was incorrect,* and truth machine pointed this out and politely explained why.** Your suggesting that he corrected you because he doesn't like PZ is simply bizarre. It should have been clear enough from his comment that he likes PZ's hands-off approach and that it's what makes the blog a place he enjoys commenting (for years now).

e) Several bloggers I greatly admire for their ability to keep substantive commentary going on their blogs and sticking to the topic would have already deleted yours for being off-topic.

Not PZ, notably.

*I noticed this, too, but didn't bring it up - I'm not argumentative like truth machine.

:)

**Even after your weird redefinition of moderation, it was still wrong.

I also admire Pharyngula threads and PZ's tolerance - and that IS what moderation is about. But this thread is not and will not be about Pharyngula. I am not letting this blog become Intersection or Greg Laden's blog, etc., where people who either love or hate PZ come to hijack threads and discuss Pharyngula. Back to the topic.

Or, to put it bluntly: PZ moderates carefully, I moderate bluntly - a quick Delete-finger....

Bora, nice post.

I was going to congratulate you on blowing away some of the fog that often obscures the review/editorial process, until you pointed out in the comments that you're still well and truly swathed in this fog.

I'm all for reading reviewer comments and author responses along with published work - it's educative for the whole audience, and it puts everyone's hard work under the spotlight. We get to see more about what a paper had to go through to be accepted, and how good the reviewers and editors are in a particular journal. We can evaluate the paper in more context. Thanks for posting yours.

Now you're redefining "carefully." Ay.

I don't think anyone is or was trying to make it a discussion of Pharyngula - merely to correct the false impression you were giving about moderation there. (I doubt anyone shares your understanding of "carefully moderated.") Good grief, this is silly.

Yes, the argument is silly.

There are two generally recognized definitions of the verb, "to moderate." The first is to restrict or repress. The second is to oversee and direct. Both are applicable to comment sections of blogs. It isn't "silly" or "dishonest" or "weird" or "wrong" or even particularly unusual to use the definition Bora uses here.

hi everybody,
i'm alessandro delfanti and i work for JCOM. i'm the editor of the forthcoming special issue on P2P science that contains the Kouper's paper.
let me please switch to a slightly different topic to explain our peer review policy: i want to make clear what Bora's role is, and try a quick reflection on our mechanisms

bora is a member of our scientific board (and we asked him to join us for this very occasion: the issue on P2P science and its articles on science blogging). when we receive a paper we send it to a member of the board (bora in this case) asking him/her to suggest two blind referees. so did bora, but he added some comments on the paper. we ignored them, because they were not double-blind.

the two referees answered, read the paper, made their comments (mostly similar to bora's), asked the author to change the paper. the author sent a new version. both the referees approved it for publication. in these cases, if the referees agree, we don't get back to the member of the board. we published the paper.

then bora wrote (privately) to criticise the paper. we decided that we could not ask the author to rewrite it: it had been already approved by the referees. well, except for the part on Karadzic... (which was not easy for the referees to spot: Karadzic was actually president of the Sprska Republic, the Serbian part of Bosnia). after bora's email we asked Kouper to change that part: the paper you can download now from JCOM does not say that Karadzic was a Serbiam president. we also wrote to Bora to explain him why we had not taken into account his comments.

what did we (at JCOM) learn?

1) that bora is a great editor! i'm not ironic, look at the time, efforts and intelligence he's spending on jcom! bora, this means we'll ask you to work MORE...

2) that I should have forced the review process and stopped the publication? perhaps, but i don't want to enter in the discussion about the quality of the paper: the author will hopefully defende herself.

3) that the double-blind, peer review mechanism is flawed or at least old? for sure. an author and a journal can be blamed for the quality of a research article in such a deep and argumentative way only thanks to open access. and in most cases two referees are not enough, compared to the thousands of bloggers and other persons that have a better, more rooted in practice, more active knowledge of the topic.

4) that JCOM, which is i think the only OA journal devoted to science communication, should open up its review process? for example enabling everybody to comment a paper directly on our website - PlosOne style? well, this is an old idea we decided not to implement last time we updated the website. we are probably going to discuss it again, mostly because of Bora's and other bloggers' post. but it is expensive and hard to manage...

anyway, i hope Inna Kouper will explain what she thinks about the problems the blogosphere raised.
and i hope you will enjoy the other articles of the JCOM's special issue

and it would be great to know what you think about our (so classical) review process, and how to improve it!

ciao from italy
alessandro

By alessandro delfanti (not verified) on 10 Mar 2010 #permalink

There are two generally recognized definitions of the verb, "to moderate." The first is to restrict or repress. The second is to oversee and direct. Both are applicable to comment sections of blogs. It isn't "silly" or "dishonest" or "weird" or "wrong" or even particularly unusual to use the definition Bora uses here.

These are not separate, but overlapping. The sense of restriction and active direction is generally used, and that meaning was implied in the subsequent elaboration to which truth machine responded. None of it was applicable to how PZ or the commenters at Pharyngula operate. The reason the blog works is not "careful moderation," if this term has any beyond the most idiosyncratic meaning, and some of the things Bora asserts that PZ does in support of his characterization (e.g., deleting comments) are done virtually never if at all. "Moderation means that the blog owner sets and controls the tone of the comments, sets the criteria for what is 'over the line' and builds a commentariat community that enforces his rules." He does not set or control the tone of the comments, his criteria for what is over the line are remarkably narrow, and the community does not "enforce his rules." It is wrong. What people imagine when they hear "carefully moderated" is what is being done here: controlling the direction of the comments, threatening commenters, emphasizing blog ownership,... That is very distinct from what PZ does.

Of course, you're about as unfit to judge clarity or intellectual honesty as a person could possibly be, so I expect no better from you, and won't be responding to you further.

Anyway, I enjoyed and agreed with the dissection of the article, as I said at the time I linked to it in a comment thread at Pharyngula. It's a shame more people can't simply acknowledge that they said something incorrect or presented their case in a way that is likely to be read as contrary to its intended meaning. Oh, well.

Careful does not mean Severe. It means "thoughtful" in a sense. PZ cares what is going on. He sets criteria. They are pretty loose, but the line exists and PZ makes it very clear where the line is. Thus, PZ carefully moderates his comments. I know. I've been commenting on Pharyngula since 2004 when it was a little unknown blog and I watched it develop and I watched PZ's strategies for comments develop and I talked to PZ about it.

"It's a shame more people can't simply acknowledge that they said something incorrect or presented their case in a way that is likely to be read as contrary to its intended meaning.".....so, why can't you do it?

================

OK, now that this disruption is over, let's get back on topic. I am glad that Alessandro joined us in this conversation. There has been a lot of discussion of 'double-blind' reviewing, with both pros and cons put forwards, in many places. What do you think about it? Especially in a social sciences type of journal like JCOM?

I think this is the problem:

the two referees answered, read the paper, made their comments (mostly similar to bora's), asked the author to change the paper. the author sent a new version. both the referees approved it for publication.

If their comments were mostly similar to his, and she rewrote the paper and (as is clear) didn't fix the major problems, I don't understand why they then approved it for publication. It's not necessarily a larger systemic issue, but maybe simply a matter of two people not seeing the main problems or not reading the revision closely enough or not sticking to their guns.

***

Careful does not mean Severe. It means "thoughtful" in a sense. PZ cares what is going on. He sets criteria. They are pretty loose, but the line exists and PZ makes it very clear where the line is. Thus, PZ carefully moderates his comments.

Silly. The criteria are extremely loose. All you can say is that he moderates at all. Really, the impression you're giving is incorrect, and truth machine has explained why. How and why the community works as well as it does is very interesting and important to understand, but this can't be done on the basis of a faulty, misleading presentation. I understand that you want to distinguish it from some that are completely (as I understand you to be saying) unmoderated, but talking about bannings and removed comments there, as you did in your post, or his setting and controlling the tone of the comments, as you did just above, conveys a false impression.

"It's a shame more people can't simply acknowledge that they said something incorrect or presented their case in a way that is likely to be read as contrary to its intended meaning.".....so, why can't you do it?

I've done neither of those things.

I agree with SC that this is odd:

"the two referees answered, read the paper, made their comments (mostly similar to bora's), asked the author to change the paper. the author sent a new version. both the referees approved it for publication."

It makes one wish even more that the author's responses to reviewer comments were available for public consumption. Perhaps there was something there that didn't make it into the paper, but that convinced the reviewers to approve it. Wouldn't the rest of us like to know what that was.

Going back to the paper, I think some of the problem with it is an unclear definition of what a science blog is, and a wrong impression of what a science blog contains.

When commenting on Sheril's blogpost about science blogging, I touched the same issues.

As usual, unclear definitions will confuse the matter.

And it certainly doesn't help that the sample size was definitely too small.

I'm not a scientist, just a social media PR flack. But I spend a lot of time looking at online communities, so here are my observations:

in any online community but especially in this one, it's really easy to get tripped up on nomenclature. I think your comments that the study relied on far too small a sample really says it all.

but here's the thing: if you're really interested in how we might "facilitate broader involvement of citizens in the discussions about science," we might want to consider more direct discussions between scientists and non-scientists. I think that means going beyond the people who comment on science blogs.

Let me know if you're interested in doing that.

It's rather sad that the claim is made that it is not appropriate to discuss here a claim made here. SC's #28 is correct on all points, including the excellence of the dissection of the JCOM article.

By truth machine (not verified) on 10 Mar 2010 #permalink

it would be nice to publish all the earlier versions of an accepted paper and the referees' comments - but i think this would also be too complex. you will have to publish 5-6 documents!

another possibility would be to use a more open platform. have you read this book? apart from proposing an interesting pov on scientific publishing, it is open for all to comment and improve
http://mediacommons.futureofthebook.org/mcpress/plannedobsolescence/

but again, do you think a small social sciences journal like JCOM could afford something like that? we simply cannot. we could probably enable comments in a blog style, and only to registered users

By alessandro delfanti (not verified) on 10 Mar 2010 #permalink

There are lot more cons than pros regarding open review (see what I wrote a day after this post). I would also not add commenting section.

My suggestions:
- tighten up the reviewing process
- put articles in categories, e.g., Original Research, Reviews, Opinion, etc.
- put in some kind of widget that can show, on each paper (abstract page), links to a) other articles that cite it and b) blog posts that discuss it.
- start using DOI. I tried placing this post on ResearchBlogging.org but it never showed up as I had to make the citation manually.

Alessandro, thanks for posting the details about the review process here. It's definitely appreciated.

But I'm concerned that the Editors at JCOM come out of this looking like somewhat useless paper pushers. Surely you are responsible for assessing the quality of a reviewer's report as well as the quality of a submitted manuscripts?

If you have evidence from your own experience that a paper is not of high enough quality (albeit via Bora, in this case), surely you can use that to better inform your decision over acceptance?

I understand you might want to maintain the integrity of your review process, but double blind or not, a crap review is still a crap review. Editors are in a position to make sure weak work doesn't pass through to be published.

Bora -- thanks for your suggestions, we'll discuss them. just a quinck answer:

> put articles in categories, e.g., Original Research, Reviews, Opinion, etc.

we already do this. articles are peer reviewed; comments are invited and not reviewed

> put in some kind of widget that can show, on each paper (abstract page), links to a) other articles that cite it and b) blog posts that discuss it.

can be a good idea... but a) is difficult, this is what scopus or google scholar do, while b) would be an interesting experiment, but how would you select the blog posts you want to underline?

one of the best ways could instead be to give the possibility to comment some articles,perhaps the ones we think are more interesting for people who comment and interact - bloggers, for example

By alessandro delfanti (not verified) on 13 Mar 2010 #permalink

mike - of course we are responsible for the quality of a reviewer's report as well as for the quality of a submitted manuscript. i don't think we are bad papers pushers. i explained our process because i thought that in bora's post it wasn't clear.

as for the review process, we try (and often need) to be flexible enough not to let formalities become more important that quality. but it is always hard and delicate to completely turn upside down referees' decisions

JCOM is a small journal, it was born to be a bridge between the academic world and a more practice one. our readers are not only media or sts scholars but also scientist, journalists, bloggers (!) and we are still trying to position ourselves between the rules of an academic journal and the interests of our broader public

we surely learned a lot from this uproar of criticism, and we're probably going to make some changes during the next few months

thanks for your comments
a

By alessandro delfanti (not verified) on 13 Mar 2010 #permalink

Hi Alessandro

Thanks for explaining your process.

"we ignored them, because they were not double-blind. "

If Bora's gone to all the effort to thoroughly and fairly review a paper I'm not certain it's a good idea to ignore it because of a failure to double-blind.

I've had a small journal completely ignore one of my reviews and publish a very dodgy and massively overinterpreted paper. I have withdrawn my services as a consequence.

You run the risk of alienating one of your most valuable resources.

By antipodean (not verified) on 17 Mar 2010 #permalink