Ratings "J Curve"

 — 

We’ve talked to several advisors and prospects who expect to see a “U” shape in the distribution of their review submissions by ratings. In other words, there should be an equal number of “1” ratings as there is “5” ratings, displaying the extremes of customer opinion. We weren’t sure what to expect when Bazaarvoice got off the ground a year ago.

We didn’t know what to expect then, but we do now.

Across many clients in diverse industries this “U” curve turns out to be more like a “J” curve…almost a reverse "L" (see below). The average rating across all clients is 4.3 out of 5 stars. The distribution looks like a J, where there are more 1s than 2s, but far more 4s and 5s than the lower ratings.

Why is this? Aren’t people more likely to share their word of mouth about bad experiences? Perhaps they are more likely to share negative opinions when they have personal experiences with a company (service, sales) than the product they buy?

And perhaps customers are interested in sharing their opinion about great products they buy, because there are so many mediocre products. So there’s some satisfaction in sharing the news when we find a product we love.

We’ll learn more and share more here. But in the meantime, this “J” curve is part of the answer to the conern: “What about negative reviews?”

  • motorolaVE440

    If the product quality is maintained then you can get a J curve in every business, with respect to users reviews. But once the product gets famous, the product sellers gives a cheaper quality in sake of brand.
    http://www.cdmacellulars.com/

  • motorolaVE440

    If the product quality is maintained then you can get a J curve in every business, with respect to users reviews. But once the product gets famous, the product sellers gives a cheaper quality in sake of brand.
    http://www.cdmacellulars.com/

  • Pingback: Surveying 3,000 Property Management Reviews | ManageMyProperty.com Blog

  • Pingback: Bazaarvoice Research Discovers a New Holiday Tradition, and 71 Million Reviews Served on Cyber Monday « Bazaarblog

  • Pingback: NSW KM Forum - Online Communities « learn.amniisia.com

  • http://www.trustfx.com Andrew Ward

    An obvious explanation for the J Curve.

    If people are influenced by user generated reviews and I believe that they are, than customers are more likely to buy items with good reviews. Thus it stands to reason that future customers will also likely be satisfied with their purchase. In this situation the accurate representation of a product and its relative price should normally drive a J Curve.

    Also it is most likely that products or companies whose customer satisfaction is inconsistent would typically not use a feedback system that would demonstrate their relative poor satisfaction levels.

  • http://www.trustfx.com Andrew Ward

    An obvious explanation for the J Curve.

    If people are influenced by user generated reviews and I believe that they are, than customers are more likely to buy items with good reviews. Thus it stands to reason that future customers will also likely be satisfied with their purchase. In this situation the accurate representation of a product and its relative price should normally drive a J Curve.

    Also it is most likely that products or companies whose customer satisfaction is inconsistent would typically not use a feedback system that would demonstrate their relative poor satisfaction levels.

  • Pingback: Word-of-Mouth Wisdom #7: Ed Keller, The Keller Fay Group » Shop.org Blog

  • http://allencrane.typepad.com/ Allen

    In my experience, the take-away from a U-shape curve is not that your respondents are more "balanced", but that your quality issues are rampant – inconsistent process, customer service, and product quality to name a few. The more the J-curve looks like a U, quality is root cause because it indicates high levels of variation among customer experiences. L's say you're not meeting customer expectations, while U's suggest you are completely out of control. Conversely, the more hockey-stick the J (upward to positive scores), the greater an indication that prodcut quality, customer service, and internal processes are in sync. Virtuous cycles–by definition–work.

    That said, you can't rest on your laurels. I like the way Ben suggests variation among "happy" responses to tease out additional shades of gray, and using that as a segue for getting additional feedback.

  • http://allencrane.typepad.com/ Allen

    In my experience, the take-away from a U-shape curve is not that your respondents are more "balanced", but that your quality issues are rampant – inconsistent process, customer service, and product quality to name a few. The more the J-curve looks like a U, quality is root cause because it indicates high levels of variation among customer experiences. L's say you're not meeting customer expectations, while U's suggest you are completely out of control. Conversely, the more hockey-stick the J (upward to positive scores), the greater an indication that prodcut quality, customer service, and internal processes are in sync. Virtuous cycles–by definition–work.

    That said, you can't rest on your laurels. I like the way Ben suggests variation among "happy" responses to tease out additional shades of gray, and using that as a segue for getting additional feedback.

  • http://www.explore-mobiles.com mobile-reviews

    Product reviews are great for the webmasters. They get a lot of content on the site free. Either by way of their own reviews, which are anyway copied from some other source, or by way of reviews written by readers. This helps a lot in SEO.
    When you reads posts on any site, there is so much of contradicting views that one is bound to get confused. Moreover, you do not know which review is genuine.
    I think, if you are really unhappy with a product, writing a reviews vent out your frustrations for the product. Although, you are sure that not even 1% of the site visitors will not read it. The same holds true for happy buyers of products.

  • http://www.explore-mobiles.com mobile-reviews

    Product reviews are great for the webmasters. They get a lot of content on the site free. Either by way of their own reviews, which are anyway copied from some other source, or by way of reviews written by readers. This helps a lot in SEO.
    When you reads posts on any site, there is so much of contradicting views that one is bound to get confused. Moreover, you do not know which review is genuine.
    I think, if you are really unhappy with a product, writing a reviews vent out your frustrations for the product. Although, you are sure that not even 1% of the site visitors will not read it. The same holds true for happy buyers of products.

  • Pingback: » Product reviews: are they really useful? ____nicolas leroy___

  • http://www.tomslist.net tom

    I wonder if it would be more of a U shape if the reviews were anonymous. Perhaps people are less likely to give negative ratings or sumbit any ratings if they know their identity will be revealed. E.g., how many times have you given you waitress negative feedback on those restaurant surveys that occassionally accompany the bill? I'm running a small experiment myself WRT rating people anonymously online (www.tomslist.net). There's only a couple hundred ratings so far and the distribution looks a bit more U-ish but it's still quite early. We'll see how it goes.

  • http://www.tomslist.net tom

    I wonder if it would be more of a U shape if the reviews were anonymous. Perhaps people are less likely to give negative ratings or sumbit any ratings if they know their identity will be revealed. E.g., how many times have you given you waitress negative feedback on those restaurant surveys that occassionally accompany the bill? I'm running a small experiment myself WRT rating people anonymously online (www.tomslist.net). There's only a couple hundred ratings so far and the distribution looks a bit more U-ish but it's still quite early. We'll see how it goes.

  • Pingback: Deep Jive Interests » Recommended Reads for September 8, 2006

  • http://www.cellphones.ca Branden

    Sam,

    We have approx. 18,000 user reviews of mobile phones contributed to our site and aggregated from other sources. Of those 18k 25% of reviewers gave their phone full marks, 10/10.
    More on that here if you're interested: http://www.cellphones.ca/1779

  • http://www.cellphones.ca Branden

    Sam,

    We have approx. 18,000 user reviews of mobile phones contributed to our site and aggregated from other sources. Of those 18k 25% of reviewers gave their phone full marks, 10/10.
    More on that here if you're interested: http://www.cellphones.ca/1779

  • Ben

    Hi – I originally left this comment at a blog that links here, but am reposting here as I thought you might find it useful.

    This is as much of a question anchoring problem as it is a problem of certain customers responding. I'm a survey researcher, and attended a presentation recently which covered this issue (I've hunted, but cannot find a relevant link, sorry). The gist was that more useful, distinguishing, information could be obtained by chaninging the response options from somthing like:

    Extremely good

    Extremely bad

    To something like:
    Way beyond expectations
    Better than expected for a good firm
    About what I expect from a good firm
    Not as good as expected for a good firm
    Way below expectations

    You'd need to play around with the wording, but the effect should be to gravitate answers to the midpoint, so that the extremes actually serve their purpose: to discriminate between the average, the really good and the really bad.

    Of course, if the aim is only to weed out the really bad, then this may not matter much – the bad should show up regardless.

  • Ben

    Hi – I originally left this comment at a blog that links here, but am reposting here as I thought you might find it useful.

    This is as much of a question anchoring problem as it is a problem of certain customers responding. I'm a survey researcher, and attended a presentation recently which covered this issue (I've hunted, but cannot find a relevant link, sorry). The gist was that more useful, distinguishing, information could be obtained by chaninging the response options from somthing like:

    Extremely good

    Extremely bad

    To something like:
    Way beyond expectations
    Better than expected for a good firm
    About what I expect from a good firm
    Not as good as expected for a good firm
    Way below expectations

    You'd need to play around with the wording, but the effect should be to gravitate answers to the midpoint, so that the extremes actually serve their purpose: to discriminate between the average, the really good and the really bad.

    Of course, if the aim is only to weed out the really bad, then this may not matter much – the bad should show up regardless.

  • http://sparkplug9.com/bizhack/ John Koetsier

    Thanks for sharing – very counter-intuitive. And very good to know when you're pushing to enable more openness online.

  • http://sparkplug9.com/bizhack/ John Koetsier

    Thanks for sharing – very counter-intuitive. And very good to know when you're pushing to enable more openness online.

  • http://www.bazaarvoice.com Sam Decker | Vice President of Marketing and Products

    Coincidentally, a couple weeks after we announced the J-Curve KellerFay came out with a study that found 63% of word of mouth (online or offline) is positive. The conventional wisdom that the majority of word of mouth is negative is no longer conventional nor wisdom!

  • http://www.bazaarvoice.com Sam Decker | Vice President of

    Coincidentally, a couple weeks after we announced the J-Curve KellerFay came out with a study that found 63% of word of mouth (online or offline) is positive. The conventional wisdom that the majority of word of mouth is negative is no longer conventional nor wisdom!

  • http://makemarketinghistory.blogspot.com John Dodds

    I think the negative reviewers will be less likely to rate but will provide bad word of mouth in the real world.

    This provides two possible dangers – firstly passive viewers will see the J curve effect and discount its veracity, secondly and more importantly the absence of low ratings may breed a false sense of achievement in those being rated. My philosophy has always been to look for the dissatisfied and determine if their dissatisfaction is valid – that way lies progress.

  • http://makemarketinghistory.blogspot.com John Dodds

    I think the negative reviewers will be less likely to rate but will provide bad word of mouth in the real world.

    This provides two possible dangers – firstly passive viewers will see the J curve effect and discount its veracity, secondly and more importantly the absence of low ratings may breed a false sense of achievement in those being rated. My philosophy has always been to look for the dissatisfied and determine if their dissatisfaction is valid – that way lies progress.

  • Pingback: Bazaarblog » Blog Archive » Five updates: Business Weeks articles, Red Herring 100, and Ed Keller