Why I no longer trust Yelp

One of the much-hyped benefits of social networking is that it provides a way to get personalized recommendations about businesses from a wider network. If I want to tell the world that the coffee place in my neighborhood has the best cappuccino this side of Seattle, I can do that (and it does)! That's what Yelp is for - right?

Well, not quite. I realized a while ago that my reviews don't show up on Yelp anymore. They've been "filtered" from the site, like thousands of other users' reviews. Yelp won't explain why (they say if they did explain the algorithm, it would allow merchants to game the system). And since Yelp doesn't notify you that your reviews have been filtered, the only way you can know for certain if all those reviews you spent a few hours writing are publicly visible is by logging out of Yelp, then visiting the page for each business you reviewed. Before you spend any time writing another Yelp review, I suggest you try doing this first, to anticipate if anyone else will even see your review. If your reviews are being filtered out, there's not much of a point writing them, is there?

This may be old news to many of you. But I'm writing this post because, while I thought Yelp's filtration system was general knowledge, everyone I've mentioned it to recently had no idea it was happening. This is probably because Yelp doesn't disclose that you're being filtered - you have to log out, then proactively go back and look for your own reviews, and you wouldn't have any reason to do that unless you already know something is awry! Personally, I think the lack of awareness of the filtration is more disturbing than the filtration itself.

Look: I appreciate that spam reviews pose a real problem for Yelp's credibility, that they need some sort of mechanism in place to stop them, and that said mechanism won't be perfect. But the best way to tell a spam review from a real review is to read it - just as on Amazon, Anthropologie, or any other site that hosts user reviews, some reviews will obviously be not credible. So I suggest that when you use Yelp, you scroll down to the bottom of a business's page and take a moment to click the option (and enter the Captcha) that enables you to read the filtered reviews. Based on my experience, most filtered reviews look legit, and are actually often quite helpful, especially if the establishment has very few reviews. (The fact that filtered reviews often look as legit as the unfiltered reviews is one reason Yelp has sometimes been accused of running a racket.)

Why are legit reviews being filtered out? In the case of my own reviews, I know they're useful - I got 'helpful' & 'cool' votes from other users before my reviews were exiled into invisibility. So it's not that they aren't good reviews. Yelp won't disclose their filtration algorithm, but by perusing discussions about the process on Yelp's forums and testing various changes to my own Yelp profile, I think it's probably one of the following issues. In Yelp's estimation, they seem to serve as indicators of a "less established user" (although in my case, I've been a member since 2008 - go figure).

1. I'm not active enough. It's true: I only post reviews if I have a well-founded opinion of a place - I don't review every place I visit, and usually will only review it after several experiences. But my quest for representative data means I post infrequently: at this time, I have eleven reviews on Yelp: clearly not enough. However, I don't know how many is "enough," do I?

2. I tend to post very good or very bad reviews - not a lot in the middle. Guilty as charged. Does that mean I'm a paid shill (or assassin) for local businesses? Uh, no: I just don't consider it worth my time to log onto my computer and write a neutral review! "Meh, this place was uber-meh, I don't have any advice on whether you should go" is not a review I'd find useful, so why would I write it? My reviews would only fit a standard distribution if I reviewed everything I did - and I'm not going to. The motivation isn't there.

3. I didn't have a detailed profile with a user photo. It's true - one of the oft-discussed reasons for filtering reviewers is their failure to upload profile photos. Hello people: Yelp is not my new Facebook (or my new bicycle). I don't want "Yelp friends." And I definitely don't want personal messages from creepy guys telling me how much more credible my review would be if I posted a photo of myself, preferably in their pose of choice. (Yes, I have gotten such messages on Yelp before.)

The attractiveness (or race) of a reviewer has zero relevance to the credibility of their review, so why does Yelp pressure us to have mug shots? One reason given for the profile photo requirement is to prevent people from creating fake accounts and doing one-off reviews. But again, the best way to ID a spam review is to look at the review itself, and whether the author has other reviews, and if they seem credible. Uploading a photo of a hot chick in a bikini is easy - look at the thousands of spam Twitter accounts that do that - and says nothing about the review's authenticity. (As one blog puts it, "Don't be an "Orange Head" - get a profile picture. Since you're spamming, you might as well find a demographically appropriate photo on Flickr or another image sharing site and call it your own.") When I added a profile photo (not of me), I immediately noticed that several of my reviews - though not all - became visible.

So what's my future on Yelp? At first, I was peeved that Yelp yanked my reviews - even the ones with helpful votes. I wrote them not to benefit myself or at the behest of business owners, but purely to help strangers find (or avoid) good (or bad) experiences. On the other hand, I don't have to feel bad about free-riding on the network (reading reviews and never posting) if Yelp won't even let other people see my reviews. That totally lets me off the hook - and disincentivizes me from putting in any effort at all to maintain the quality of the review pool.

Unfortunately, though, I am most interested in reviews from other people like me, who don't review everything they do ("meh!"), who only post when they have a strong and well-supported opinion based on repeated experiences, and who don't use Yelp to flirt with strangers. Based on my experience, I have to expect such people will, like me, be filtered out. So whose reviews am I actually seeing on Yelp? Yelp says "The review filter sometimes affects legitimate reviews, especially those from less established reviewers," but we don't know how many users are affected by the filter, because most users won't ever notice. The casual Yelp user probably knows much less about ensuring their reviews appear on Yelp than a motivated Yelp scammer - which means the reviews you see are quite possibly dominated by scammers. That's just great.

All of this is just another illustration of how the priorities of social media providers and the priorities of their users rarely coincide. What Yelp wants me to do (hang out there as if it's my new Facebook & make friends), and what I want to do on Yelp (find and post objective, anonymous reviews), are not equivalent. Yelp apparently wants a little piece of social network action, but Yelp is not OK Cupid, and I'm not there to flirt. Yelp chooses to punish me for noncompliance based on criteria that relate to what it cares about - frequency of my visits, profile photo, etc. - rather than the parameters I care about - the number of 'helpful' votes my reviews got, or how well-written they are. Fair enough: when you play in someone else's sandbox, you play by their rules. Just don't let them trick you into thinking it's all about your opinion, or that the types of reviews that are filtered will make sense from the perspective of either customers or merchants. It's not about quality reviews, after all - it's about profit.

In the end, I prefer to read - and discount - the spam reviews at Amazon and other sites for myself, rather than trusting a profit-motivated intermediary to do my evaluations for me. I have a lot of sympathy for the challenges Yelp faces, but their filtration system isn't working, and they're not transparent about it, which leaves me without confidence in them. C'est la vie. . . but I'm sad that social media isn't doing a great job of crowdsourcing recommendations. Can we do better?

More like this

Update: welcome Consumerist readers! While I use my own experience to illustrate concerns about third-party online merchants, this post is mainly about the bigger long-term informational problems I see with reputation, reliability, and online communities. Please feel free to weigh in! A few weeks…
During my summer blogging break, I thought I'd repost of few of my "greatest hits" from my old blog, just so you all wouldn't miss me so much. This one is from July 3, 2007. It's one of the most popular posts I've done, and it was linked quite widely in the science blogosphere. The interview…
I'll confess that I am not one who spends much time reading the reviews of books posted on the websites of online booksellers. By the time I'm within a click of those reviews, I pretty much know what I want. However, a lot of people find them helpful, and the ability to post your own review of a…
The last time I looked at the reviews of More Guns, Less Crime at Amazon.com I noted how, after a negative review was posted, Lott would post a five-star review to push the negative review off the front page. On December 11, someone posted a negative review of More Guns,…