You might have thought we'd have a new study for you to participate in this week. You're half right. If you've read CogDaily, Terra Sigillata, Uncertain Principles, or Chaotic Utopia in the past two weeks, you've actually been participating in today's non-scientific study.
ScienceBloggers have a private forum where we can discuss technical issues with the blogs, talk about our latest successes or failures, and even coordinate super-secret studies of our readers. Many of us have observed that a blogger can spend 5 minutes posting a YouTube link or a cartoon, and somehow that will be their most popular post of the month. Sometimes it seems futile to spend hours crafting thoughtful posts when the herds from Digg and Slashdot seem more excited by a quick photo or a user poll.
But it also seemed to me that we've never systematically studied the issue. At CogDaily, we've had our share of flash-in-the pan successes, but we've also had popular posts that we've really worked hard writing. So I convinced three of my fellow ScienceBloggers to participate in an informal study. Here are the rules:
- For the next ten days, whenever you write a post that really takes a lot of effort, make a quick one-off post.
- The thoughtful post should involve real science, and should be at least 400 words long -- not including quotes!
- The one-off post should offer as little as possible of your own brain-power. Spend no more than 15 minutes on it. But it should be at least tangentially related to the main subject of your blog.
- Self-submit both of your posts (separately) to Slashdot, Digg, Fark, or Reddit. (e.g. submit both posts to Reddit. Don't submit one to Reddit and the other to Slashdot). Do your best job talking up each piece.
- Let's not stack the deck. If you publicize a post, give equal time publicizing its partner post. And don't link to fellow Sbers any more than you would normally during the study period.
Over the study period, we wrote a total of sixteen posts. We worked hard at publicizing them, but none of them actually enjoyed a landslide of success. Still, there were two posts that received over 3,000 page views, and six with over 1,000 page views. So, did hard work pay off, or were the short, quick posts the ones receiving the glory? Here's a snapshot of the results:
As you can see, in all these areas, thoughtful posts did better than non-thoughtful posts. The difference was significant for each measure except number of comments. I expressed number of page views as a percentage, because our blogs have different levels of popularity, and thoughtful posts on average received 1.8 percent of all page views for the period, while non-thoughtful posts received just 0.6 percent. The difference in time readers spent on each page, again adjusted for blog popularity, was even larger. Finally, the average number of links from other blogs to thoughtful blog posts was significantly larger than to non-thoughtful blog posts.
A couple caveats: this is a small sample, and while the results are significant, if one of the posts in the sample had received a coveted front-page link from Digg, Fark, Reddit, or Slashdot, the resulting traffic would have dwarfed the results you see here. How close were we? Well, my post Does fear really slow down time was Dugg 46 times. A link can be elevated to the front page of Digg.com with as few as 60 Diggs. Had that happened, our casual study might have had the opposite result.
- Log in to post comments
Fun experiment, and interesting results. Now, given that the main constraint is time, what if you had invested the time "saved" by doing the quick post in promoting it, perhaps simply emailing some friends and fellow bloggers and saying "Hey, take a look at this and Digg if you like". I am pretty confident the results would have been opposite-that Digg number is to close...
Of course if you were old Usenet hands like Chad you would know that the best way to generate page hits is just to start a fight with someone. I'm sure PZ and K.Beck can tell you the same.
I took one shot at disgruntled postdocs and you should see the hits take off...
BTW, what's with all the surveyin' around here? Overlords looking for some justification? Is the plug being wiggled?
Drugmonkey: And the best way to end one is to invoke Godwin's Law :)
Alvaro:
Well, since I wrote this post, my post on The curse of knowledge just got a big boost from Stumbleupon.com, so these things can go both ways. I think a lot of these results depend on the particular blog, and where it happens to get promoted.
Interestingly, the post that was nearly Dugg wasn't even submitted to Digg by me. So you have a lot less control over these things than you think.
In some ways, I'm glad that no one was Dugg/Reddited/Slashdotted, so we can see how hard work affects normal, day-to-day traffic as opposed to striking it rich in the social networking lottery.
Drugmonkey:
We do surveys and polls all the time at CogDaily. There was no prodding from the overlords on this one, though of course they're interested in the results. Indeed, other than the occasional "ask a ScienceBlogger," we're never directed from above about our content in any way.
I disagree that the results are significant because:
1. Your sample size, three, is far too small to correct for the effects of so many relationships other than the one being considered.
2. I think having the authors taylor the posts to a specific length or "apparent thoughtfulness" invalidates the results entirely. An authentic "Not Thoughtful" post likely involves more spontaneity and less planning than a "Thoughtful Post", but you affected that ratio, and therefore the post content, by asking for the additional planning necessary to taylor the contents. Less spontaneity and no content either? Booooring!
So there, Mr Science-pants! Stick that in your Blog and Post it!
Neighborcat:
1. The sample size is four or sixteen, depending on how you measure it. But as I mentioned in the post, I agree that this is still a bit small, given what we're attempting to measure.
2. Bloggers were free to post all sorts of posts during the study period. But we wanted to measure, specifically, how longer more thoughtful posts did compared to shorter, more spontaneous posts, so we only included those in our statistics. Here are links to the posts themselves, so you can decide for yourself whether they meet the criteria:
http://scienceblogs.com/cognitivedaily/2007/09/cool_visual_attention_ex…
http://scienceblogs.com/cognitivedaily/2007/09/the_claim_politically_li…
http://scienceblogs.com/cognitivedaily/2007/09/in_1999_melissa_kamins_a…
http://scienceblogs.com/cognitivedaily/2007/09/does_fear_really_slow_do…
http://scienceblogs.com/cognitivedaily/2007/09/do_verbal_metaphors_affe…
http://scienceblogs.com/cognitivedaily/2007/09/magic_dragon_illusion_vi…
http://scienceblogs.com/cognitivedaily/2007/09/the_curse_of_knowledge_m…
http://scienceblogs.com/cognitivedaily/2007/09/sleep_disorders_the_rap…
http://scienceblogs.com/principles/2007/09/watching_wavefunctions_colla…
http://scienceblogs.com/principles/2007/09/great_ideas_in_world_cuisine…
http://scienceblogs.com/terrasig/2007/09/have_you_ever_gotten_a_good_re…
http://scienceblogs.com/terrasig/2007/09/source_of_muscadine_grape_skin…
http://scienceblogs.com/chaoticutopia/2007/09/wild_wild_west.php
http://scienceblogs.com/chaoticutopia/2007/09/speaking_of_oil.php
http://scienceblogs.com/chaoticutopia/2007/09/climate_change_in_action…
http://scienceblogs.com/chaoticutopia/2007/09/president_bushs_underwate…
I don't understand why you didn't analyze past posts.
I'd like to see a study of which blogs have the greatest ratio of "thoughtful" to "unthoughtful" posts.
I am always annoyed by posts that say something along the lines of "Need I say more?" followed by a link. It makes the blogger look really stupid.
Two things:
First, this is at least a little significant because it shows that blog reading people are capable of and interested in reading longer pieces. This contradicts the things that people like nic carr have been arguing for, like the fractured attention span of the internet public. Of course, these are people who are interested in science so i'm sure that the demographics are really skewed.
Second, sort of along the lines of what neighborcat said, i think you might have overlooked the fact that usually quick posts come as a sudden brainstorm. At least the interesting ones do. Quick ideas are easy to read and comprehend, and they have a very high payoff-to-time ratio if interesting. The kinds of things that i write when i'm trying to write are almost never as good, and aside from that they basically never have the kind of brainstorm feel that i'm sure your one-hit-wonders do.
So like i tried to say: this does (maybe) show that time and effort are worthwhile, but i don't think that it speaks to the digg effect at all.
Zachary: no, that just changes the argument to one that's about whether or not Godwin's really applies. :)
I've always found that my quickie posts tend to generate the most responses, although often the responses wander off topic.
You may have hade more data to use if you didn't self-submit to reddit, Digg, etc. Have a friend do it for you, because there are those who will automatically downvote a submission just because it's your own story.
I'm no science whiz, but I like to read your blog. Your popular post, Does fear really slow down time, always struck me as out of place here. Now I understand why.
I agree with the first respondent: had the number of posts been higher, and even one of them made it to the front page of Digg, then the results would be really different. And it's not clear how. The existence of sites like Digg screw up the analysis---not all posts are created equal.
I see what you're trying to do, though. I just think you went about it the wrong way: you collected data before getting the question straight....
If the metric for investment is time, or brain power, then writing a complex "Thoughtful" post trades off against writing lots of simple "Not Thoughtful" posts. Given that the total amount of promotion time is approximately equal per page, however, the total cost contribution of promotion for a blog of "Not Thoughtful" posts will be higher. (And more "brain power" will be devoted to promoting the blog than producing content.) As a result, in the short term, there's going to be a "sweet spot" (equilibrium) where the goal should be to use "Not Thoughtful" posts to cross-promote the better "Thoughtful" material.
Over the long term, however, I think the blog with the greatest "total brain power invested in content" has a better chance of becoming an institution. The reason, I suspect, is that this level of quality then gets associated with the blog as part of its "brand," which then in turn draws a recurring audience. As a result, those blogs that focus on content over promotion will mostly come out on top. The problem is, sustaining motivation in the "long term" is hard to do. Investing a lot, with nothing more than the promise of an uncertain return some unknown time in the future, is really risky.
One way to get around this is to join a collective with an existing brand: Science Blogs. But, if we want to "know," there's an equation from finance -- the Black-Scholes derivatives formula -- that could be used to quantify the value of "Thoughtful" posts as "options" on this future brand power and thereby help bloggers set the right level of investment. But that's a level of analysis that goes beyond what any normal person would care to do for a simple blog....
Jeremy, I think you are on to something with the brand angle.
I would emphasize uniqueness rather than quality as the necessary component for branding.
The reason thoughtful posts are important for branding is because the more of Dave Munger's thoughts that go into his blog the more unique his blog becomes. If Dave Munger maintains a thoughtful blog it is unlikely that his blog's content can be obtained elsewhere. After all nobody else has Dave Munger's thoughts.
On the other hand, if Dave Munger were to eschew thinking and instead post links to YouTube and other blogs then his blog ("brand") would lose its uniqueness.
I think the difference probably lies in that quality 'thinking' content generates more return readers and a small number of high quality views.
Non-thinking posts are very hit or miss. Most of them won't get a lot of views, but you might get one that make the front page of digg, and then you get thousands of poor quality views.
Is there a correlation between the main topic of the blog and the results shown in the charts?
wow! this post generated a lot of thoughtful comments from you science buff guys. interesting topic. I found it after you linked to my snail dribble post.
I do casual fridays over at the gimcrack too. only I call mine corset friday :-)