This stat came to mind when I read a recent Ad Age article, “They learned it by watching you.” The article discussed increases in marketing budgets focused on why consumers buy. Research ranges from watching consumers in stores to focus groups and surveys. Today’s marketers employ “a cadre of technologists and scientists to uncover consumer needs and emotional states… It’s hard to pinpoint how much is spent on shopper marketing, but estimates range from $3 billion to as high as $18 billion.”

Consumers readily and willingly contribute their opinions, knowledge, and experiences online about brands and products. In fact, some of our clients like Sephora have up to hundreds of reviews per product. So, why are brands still spending so much time and money on complex consumer research and focus groups that can easily cost six figures, span multiple cities, and reach consumers who may or may not even be target customers?

With all of the attention and investment going to these types of research, are manufacturers and retailers missing something? Focus groups and surveys place customers in an unnatural situation, overtly prompting them to explain their choices. These types of research are fundamentally flawed – most customers aren’t sure why they chose one product over another. A 2005 Business Week article, “Shoot the Focus Group,” highlighted the short-comings and response bias of traditional focus groups. This fundamental flaw was echoed by Malcolm Gladwell in Blink: “Asking someone to explain their [behavior and intent] is not only a psychological impossibility…but it biases them in favor of the known over the unknown.”

Internet Retailer reported that 66% of consumers shop online before making purchases, online or off. If consumers begin their product research online, shouldn’t companies focus their research there too? With the right combination of strategic partner and technology to help, brands can ask the customer why they buy while they are actively researching products. This approach is faster, 24×7, less expensive, more comprehensive, less overt, and less likely to cause response bias.

For example, one of our food product clients captured their customers’ experiences around healthy eating by asking customers to submit experiences in the form of stories online about what products and recipes worked for them. The campaign also asked consumers about themselves, with criteria like occupation, exercise frequency, favorite exercise, and favorite snack.

With this single prompt, consumers (without response bias or expensive testing equipment) contributed their experiences about a relevant topic, provided their interests and demographics, and highlighted the products they purchased. The brand can take this content and deeply analyze it to understand why customers bought these products, who these customers are, and what their other interests and motivations are.

Even better, manufacturers and retailers can conduct this research on their own sites. Using their sites, brands can combine consumer research with marketing outreach and user-generated content. People are more honest when they are relaxed and not under the pressure of an interview. And, with 82.5 million consumers willing to contribute, there will be no shortage of participants for consumer behavior research through UGC.

How CPG Brands Win Big with Social Media Marketing

Current and former execs from Rubbermaid and Kraft share real world stories of success through social commerce. Download the free webinar to learn how reviews are the new focus group.

Question: What do the following silly things have in common?

1.    what is the approximate size of a banjo
2.    redheads and the men that love them
3.    how to hit the sasquatch
4.    used men’s hunting socks

Answer: they are all actual phrases that searchers have typed in to discover various Bazaarvoice clients’ review content.  (For the record, I cannot fathom why someone was searching for “used men’s hunting socks”.)

The long tail of search engine data tells a fascinating story that smart marketers are listening to.  It is not just silly phrases like those above, but obscure and unique terms – perhaps used only once or twice – that give insight into what your visitors are really looking for.  Long tail search terms might not match your page titles or your carefully crafted page descriptions, but they do match some other combination of words on your page – even when you are not anticipating them.   For many sites, the unanticipated terms account for over 80% of the search terms and over 50% of the referrals, and this makes it imperative to understand how best to target those terms.

With consumer-generated content, targeting unanticipated search terms is easy.  Consumers writing reviews, for example, are using their own words to describe their experiences with a product or service.  Their reviews do not go through a formal editorial process to correct misspellings or to remove personal words like “I”, “me”, and “my”.  The reviewers themselves are not employees who must adhere to the official company marketing message in what they write.  Further, a typical page of reviews is written by multiple, unrelated authors who have different perspectives on the product being reviewed, and these different perspectives result in a greater variance in the words used in the reviews.  This variance means that pages of reviews are more likely than homogenous product pages written by one or two authors to drive search terms from unanticipated combinations of words, that is, long tail search terms.

For example consider the following excerpt of search terms from one of our clients’ long tail data.  Each of the following words was searched on just a few times during the reporting period.  Notice the variance in the words and phrases used:

•    nike free trainer 7.0 review
•    nautica jeans colorblocked polar fleece pullover
•    love tease pleated rhinestone-trim dress
•    desperate housewives perfume
•    simmons beautyrest adrian
•    dz7064
•    love tease beaded organza halter dress
•    alfani safari field jacket
•    belique cookware
•    arturo chiang umiko
•    nike free 7.0 review
•    tools of the trade cookware reviews
•    dkny draped swimdress
•    reviews on benefit firmology
•    towelbuyer
•    love tease beaded babydoll dress
•    nike free trainer 7.0 reviews
•    keds mosaic slipon

Lists of long tail terms like this are typical for publishers of reviews:  some are product names, some include keywords like ‘review’ or ‘reviews’, and still others are brand names or generic descriptions of products.  Even though each term is searched on once or twice per reporting period, the sum of all of them together frequently accounts for tens of thousands or hundreds of thousands of search referrals.

No marketer can predict all of the terms that their searchers will use during a reporting period, but it is possible to identify the types of words that are most frequently associated with success – the words that are used most frequently in search referrals that lead to conversions.  For example, consider the following illustration.  On the left side is the excerpt of long tail data mentioned above with each keyword phrase now associated with the number of “Referrals” generated and the number of “Conversions” accomplished during the reporting period.  (A referral is counted when a searcher clicks through to your page from the search engine.  A conversion is counted when the visitor subsequently takes the desired action on your site, whether that is a purchase or some other metric.)
The list on the left is aggregated into a similar list on the right by tallying the number of conversions that were associated with each individual word in the search phrases on the left.  For example, the words “nike”, “free”, “trainer”, “7.0”, and “review” each count for one conversion, based on the first row of data in the chart on the left.  Tallying these numbers up for the entire list (including the rest of the search terms that were not included in the list above), we see some interesting trends emerge.  For one thing, the most important words for generating conversions were “review” and “reviews”.  This is not unexpected, since we are dealing with pages that contain review data.  Other words that led to conversions were “top”, “best”, “ratings”, and so on.

Wise marketers use lists like this to help determine which keywords to put into the page templates of their consumer-generated content pages.  (Incidentally, this is one of the basic principles Bazaarvoice uses in building the SearchVoice Pages that come with our Ratings and Reviews product and in building the SyndicateVoice microsite.)

The long tail is a gold mine for search marketers and the variance of the content contributed by consumers helps you tap it.  Read and analyze terms in the long tail of your data to find out what your site visitors are really looking for.  Oh, and if you are the one that was looking for used men’s hunting socks, please drop me a line and let me know what you were thinking.