The Five Star Problem

Has anyone noticed this lately?

It seems like a lot of companies are discovering how online reviews work:
1. If you have positive reviews (especially a lot of 5 star reviews), new customers will seek to do business with you! :)
2. If you have no reviews, or negative reviews, new customers will be hesitant to do business with you. :(
3. (You do not actually have to be a good company to get a bunch of great reviews.) :)
4. Why not just pay for reviews, or write them yourself? :D

[5. Because Emily can tell the difference, and then even if your business actually IS good, she will avoid doing business with you because you are shady, and you're trying to mislead people.]

The truth is, before I do business with almost anyone, I always check reviews online. If your business has good or pretty good reviews, I'll try working with you. If your business doesn't exist, sorry, I'm just not comfortable hiring you, because I can just make my map a little bigger and find someone that is less likely to rip me off, and more likely to give me a positive experience.

I do the same thing with products. When I consider buying a particular book that looks pretty good as I flip through it: what does Amazon have to say? If it has 4.5-5 stars, I'll buy it. If it has 2 stars and the reviews talk about how incorrect the information from the book is, or whatever, then I trust that it isn't worth my $20, and it certainly isn't worth the time I would spend reading it. (If there are only a couple reviews, and they disagree about the quality, I'll go with the review that provides the best evidence for their rating.)

Of course, every business has a few bad experiences. Sometimes customers are unreasonable and they leave a business angry because they didn't get what they want. Just because a company has one bad review doesn't mean it's a bad company. I know that, and I don't expect a place to have perfect reviews. I prefer that they don't. Usually if a place has 50 five-star reviews it's a red flag.

Here are the things I look for if a company's reviews seem too good to be true:

1. Do the reviews kind of sound the same? Do they abbreviate the same words, or have similar grammatical errors? Are they about the same length? Do they seem to comment on the same features, in the same ways? Also, were they all written on the same day, or within one week, or kind of in spurts?

2. Do the reviewers use names that look like real people? Meaning, are all of them a first name and a last initial? Are all of them "dreamprincess4309823" and those sorts of names? Usually a balanced sample of real users will have a variety of types of user names "John Smithington" "Bobby W." "susy" "byucutie48" "AfRiCaQuEeN" etc. Some people capitalize their names, some do not. Some use full names, some do not. Some add numbers to the end of their names, others do not. Etc. Usually fake reviews seem to follow the same sort of naming convention: lowercase first name and then four numbers, etc.

3. What do the reviewers' profiles look like? Do they include a picture or bio? How often do they review places? If everyone only has one review and it's for the place you're checking out, this is a red flag. With real reviews, it seems there are still some people who only review the one place, but other people have a few places they have reviewed. The reviews should seem compatible. (So, for example, the person should not have reviews for places all over the country for car dealerships. Nobody buys a car in a different state every other day.) If you're really looking at a whole lot of reviews for one place, some of the people should have at least several reviews. Some people should have pictures or favorites, etc. Usually on a real person's profile, the reviews will vary. So, if a person gave your place 5 stars but they gave their last review 2 stars, that's a good sign. If they only give places 5 stars, it could be that they only go online to review things they LOVE, but it's unlikely.

4. Is the number of reviews typical or reasonable for the type of place it is, where it is located? For instance, if you're looking at a really big, popular place that has been around for quite a while, 50 reviews may be reasonable. If you're looking at a place basically nobody has heard of, and when you go in they have no customers waiting, and they have 70 glowing reviews while the competition usually has about 3 reviews, that's fishy.

5. Do the great reviews comment about negative things, too? This is a good sign. If a reviewer gives 5 stars and talks about how great the service is and how much money she saved, and then says "but the wait was a little long, next time I'll get an appointment," or something, this is a lot more likely to be real. Real people writing honest reviews will talk about good things and bad things. Likewise, real negative reviews often mention good things: "He was clean, and he showed up on time, but after that everything went downhill!" Fake reviewers usually only gush about how excellent the place was.

6. How does the company respond to negative reviews? If they apologize and talk about things they have done to remedy the situation, that is a good sign. If they totally deny the claim and fabricate something else, that is a bad sign. Also, as a potential customer, don't you think the business trash-talking a customer (even if it is legit) is really unprofessional?

7. Are reviews really polarized? If there are 1 star reviews and 5 star reviews, but not much in the middle, this is a bad sign. Often, the people who leave 1 star reviews express doubt about the legitimacy of the 5 star reviews. If the reviews are legitimate, the ratings usually vary more.


These questions are usually more than sufficient to separate the wheat from the tares, if you know what I mean.

Let's review some case studies!

Case study #1 - Auto Maxima (South Salt Lake)- used car dealership and auto body shop
68 reviews. 4.5 stars.
Maybe you remember the experience I blogged about before. These guys are so sketchy!
1. Do reviews sound the same? Yes. Most of the reviews have non-native English. 'Dollars' is repeatedly abbreviated as 'dols' which is kind of unusual. Many of the reviews mention the owners by name. Many reviewers say their car was "smelly" or talk about dents in their car, or both. Several people say they have bought 3 cars from them. Most of the reviews are written totally lowercase.
2. Do the reviewers use names that look like real people? Sort of. On Google, most of the reviewers do not have names. Several have only first names. A couple are names that sound like businesses.
3. What do reviewers' profiles look like? Bad. Most only have 1 review--for Auto Maxima.
4. Number of reviews reasonable? No. Nearby, Low Book Sales has 190 reviews, which is to be expected because they are huge and they advertise a lot. Auto Maxima is tiny; they only sell about 4 cars at a time. Other small dealerships nearby have mostly 0-4 reviews. One place has 14 reviews, but they sound like they have been around for a long time, which Auto Maxima has not.
5. Do the great reviews comment about negative things too? No. Out of all of them, only one does.
6. Company response? Completely denied all claims from 1 star reviews. Wrote "thank you" in response to several of the fake reviews.
7. Polarized reviews? Yes.
CONCLUSION: There is a lot of reason to be suspicious of these reviews. (And, after our experience, we know they would have ripped us off, so it would have been better to avoid them.)

Case Study #2 - Radiance Salon (Riverton) - a salon I was thinking of going to because it is close, and pretty on the outside (I ended up sticking with my stylist that I love, even though she is far.)
58 reviews. 2.5 stars.
1. Do reviews sound the same? Sort of. All of the 5 star reviews are recommending particular stylists, and claiming that the people give the best haircut the person has ever had. All of the other reviews complain that the guy who schedules appointments messes up a lot, the workers are really catty, things are way overpriced, and people mess up their hair.
2. Reviewer names look real? They're fine.
3. How about reviewer profiles? Bad. Only a couple people review anything other than this salon.
4. Number of reviews reasonable? A little high. Upscale salons tend to have more reviews (like, 30, sometimes) than little cheap places. However, many of these reviews were posted on multiple sites and they are double-counted.
5. Do great reviews comment about negative things too? No. Except for the most recent review, which claims that the place is under new management and that they asked her to write a review. Her review is really good, and she lists one drawback, which is that she has to drive 30 minutes to get there. The other good reviews should be like that, kind of. (Also, this suggests the salon may have actually improved.)
6. Company response? Creepy! They posted a 5-star weird review complaining about people coming into the store with a black raincloud over their heads, etc. It looked like it was supposed to be from a customer, but it was obviously from the owner or someone because of the comments they made.
7. Polarized reviews? Yes.
CONCLUSION: At a minimum there is some serious drama going down there; the complaints seem well-substantiated and legitimate. The positive reviews seem uncharacteristic. However, things may have improved with new management, maybe.

Case Study #3 - Alcor Cresta Veterinary Hospital (Midvale) - Chalcy's vet, which I found because of the reviews online.
44 reviews. 5 stars.
1. Do reviews sound the same? Nope. Some reviews are long, some are short. Some people have typos, some do not. Some people misspell the vet's name. Some people write with several paragraphs, some people write one paragraph. People describe different experiences and things.
2. Reviewer names look real? Yes. Wide variety of user names.
3. Reviewer profiles? Varied! Some have pictures. Many have 2 or 3 reviews, for reasonable things. Some have only 1 review but a photo on the profile.
4. Number of reviews reasonable? Yes, but a little high. Most nearby vets have about 20 reviews. Some have 10. A few have no reviews at all. One vet nearby has 70 reviews.
5. Do great reviews comment about negative things too? Yes. The signage is kind of bad, although it is better than it used to be. It can be hard to get parking. (I experienced both of these things, actually...the parking lot is tiny; there are only spaces for like 3-4 cars in front. The first time we accidentally drove past because we missed the sign.)
6. Company response? None at all. There are no negative reviews to respond to. None of the reviews seem to be written by staff, though.
7. Polarized reviews? No. All reviews are 5 stars.
CONCLUSION: There are more reviews than average because Dr. Good really is an amazing vet that everyone loves. Seriously, he's wonderful, and the prices are really reasonable. We went for all of Chalcy's puppy visits (1 visit to say hello, plus several more for her puppy shots), plus I've called them for help a couple other times, and they are outstanding.

Case Study #4 - Neerings Plumbing & Heating (Salt Lake)
134 reviews. 3.5 stars.
1. Reviews sound the same? Yes. Most of the reviews are one or two sentences long and say something like "Matt was great! He fixed our furnace. Thank you!" or "This was a great experience. Thank you Steve!" Very little variation, actually, except among 1 star reviews.
2. Reviewer names look real? Not really. Most are by Firstname Lastname. On Google the reviewers all use names that are first names only, like "Charlene."
3. Reviewer profiles? VERY BAD. The 5-star reviewers have all reviewed the same places! "Diggity Dog Resort" "Starving Students" "Barristas"and several of the same car dealerships. Yikes! What are the chances that a whole bunch of random people would all visit those same places, and all have 5-star experiences, all within the same period, and not visit or review other places? This is very obviously a case where Neerings hired some company to give them a whole bunch of fake 5-star reviews. So, if you were to just look at one reviewer, it would seem good because they haven't just reviewed the one place, but actually it is way worse.
4. Number of reviews reasonable? Much too high. Neerings advertises a ton, though, so this wouldn't be a huge surprise...but Action advertises a lot and they have 12 reviews. Most of the other plumbers in the area have about 10-12 reviews. (Note: Whipple has 225 reviews--which are also super polarized...225 is way too many, and it just seems to be the number necessary to balance out all of the 1 star reviews.)
5. Do great reviews comment about negative things too? No.
6. Company response? None, and there should be. Plus, all of the 1-star reviews are rated "unhelpful" a whole bunch of times...even though the reviews actually are helpful. This is pretty common with sketchy companies.
7. Polarized reviews? Yes. It's all 5-star and 1-star reviews.
CONCLUSION: The 1-star reviews all state that Neerings is way overpriced, and I suspect this is true. Neerings definitely has almost entirely fake reviews.

It's important not to judge places too quickly; to accuse someone of fabricating reviews is a pretty big deal, and it's really rude if you don't actually know they've lied. On the other hand, I think it is really awkward that places want to raise their reviews and improve their online presence by lying to us. (Come on, guys! Show some class.)

Be smart, folks. Do your homework.