Online word-of-mouth communication -- such as product reviews by consumers who have purchased a product -- has become a major information source for consumers and marketers about a product’s quality. Previous research has shown a positive correlation between online product reviews and their impact on product sales. These studies assume, however, that consumers only use review scores when making purchase decisions. This is not the case, says Singapore Management University information systems professor Nan Hu. In an eight month long study, Hu and a team of three researchers investigated both quantitative and qualitative aspects of online reviews. They found that, besides review scores, consumers do pay attention to other contextual information such as a reviewer’s reputation and exposure. Hu’s research also showed that the longer an item was on the market, the smaller the impact of online reviews on sales, suggesting that companies need not provide incentives for product reviews beyond a certain point after the original product release date.

Knowledge@SMU talked to Hu about the study which analysed review data from Amazon.com using a stock portfolio analysis methodology.

Knowledge@SMU: What led you to study online reviews and how they impact product sales?

Hu: My interest in this area really began with the advent of Web 2.0 a few years ago, which really changed how people access information online. Before Web 2.0, there wasn’t such an emphasis on content. What I wanted to do was to embark on a proper study to see how the growing amount of information online is changing consumer behaviour patterns, and whether consumers really use this information in deciding how to make their purchases. We found that previous research in this area just stayed on the surface in trying to make sense of reviews. We believe, however, that consumers are much more rational and sophisticated, and look at both the qualitative and quantitative aspects of these reviews. That’s where we came in with our study -- to try to piece together the links between how changes in reviews affect changes in sales. The qualitative aspect of our research concerns how useful the content of the review is, such as does it say anything new? The quantitative aspect refers to the average rate and the distribution of reviews an item actually gets.  

Knowledge@SMU: How powerful is the online word in determining whether a product does well or not?

Hu: I would say it is pretty powerful given that the internet speeds up the diffusion of word-of-mouth. According to some industry reports I’ve read, 44% of users consulted opinion sites prior to making a purchase and 59% of respondents considered consumer-generated reviews to be more valuable than expert reviews. There are a lot third-party websites or online community forums, such as Epinions.com, which specialise in collecting consumer online reviews. I believe vendors are using online reviews to access the quality and market impression of their products. They have even pushed their R&D over to the consumer side, and are learning valuable lessons from this community on how to improve product design. I call this “reverse-knowledge-sharing”.

Knowledge@SMU: Could you summarise the key findings of your research study?

We’ve found that that consumers understand the value difference between favourable and unfavourable reviews, and that they respond differently in each case. When consumers read online reviews, they not only pay attention to review scores, but also consider other contextual information such as a reviewer’s reputation and the reviewer’s exposure. The market responds more positively to reviews written by reviewers with better reputation and higher exposure. Finally, we demonstrate that the longer an item has been released to the market, the smaller the impact of online reviews on its sales which suggests that firms need not provide incentives for reviews to be written after a certain period following product release.

Knowledge@SMU: Why are positive reviews more effective in the early stages of a product?

How the market responds to new information really depends on whether this extra information bring in something new that fundamentally changes a consumer’s expectation of a product or a company’s value. Empirical accounting research shows that security price reactions to unanticipated information conveyed to the market by actual earnings and earnings forecasts are more substantial for smaller firms, because the amount of private pre-disclosure information increases with firm size. Thus, with larger firms, any extra disclosure is not going to introduce too much new information to the market. Similarly,in the initial phase of a product’s introduction, online reviews may be one of the early, limited sources of information available for consumers to gauge a product’s quality. Hence the market is likely to rely heavily on these reviews for purchase-related decisions. However, with the passage of time and as the market gains some experience with the product, consumers can obtain product information from other sources as well, such as recommendations from friends, and comments in newspapers and magazines. At a later stage of a product, no matter who writes the reviews, consumers will not respond to them as they can add only very limited new information.

Knowledge@SMU: What methodology did you use in this study?

Hu: We used several theories and methodologies to assess the efficiency of online reviews, and the extent to which the market reacts to contextual information about reviewer quality, reviewer exposure and product coverage. One is the analogy of the stock market which we applied to an online store – in this case, Amazon.com – where we considered individual store items as equivalent to individual stocks. We treated the purchase of an item in response to an online review as being similar to how an investor would buy or sell stock in response to revisions in the analysts’ forecasts. As we had predicted (correctly), the impact on both markets – online purchases and stocks – did have the potential to change the consumers’ intrinsic valuation of a product. We also brought in the stock market, analyst forecast, and a modified version of the Fama-French risk portfolio to estimate the “abnormal” sales associated with events, both favourable and unfavourable. Fama and French use the average return from the benchmark portfolios formed on the basis of size (market equity) and book-to-market (the ratio of book equity to market equity) to adjust the actual firm return to derive the abnormal return. The size and ratio between book value and market value are the risk factors that can explain the cross-sectional variance of expected stock returns. In the Amazon market context, we believe that each item has some “normal” changes that are driven by the product sub-category and its list price. Any extra sales changes over the “normal” change are classified as “abnormal” sales. To the best of our knowledge, this is the first time researchers have employed portfolio methods and analyst forecast theory to study the efficiency of Amazon market. Incidentally, the feedback I’ve received at conferences where we’ve submitted our research paper, is that we may have over-stated the comparison between the Amazon market and the stock market. But we can afford to take that risk because we think our approach is quite innovative.

Knowledge@SMU: Were you and your team surprised by the study findings?

Hu: Not really. I think the results were what we had expected them to be. They showed that a higher review score does not always translate into higher sales, and thatI it actually depends on who writes the review and what the content of that particular review is. In the online review environment, there is an enormous information asymmetry between online reviewers and new customers. For instance, online transactions of “experience goods” such as music CDs and books involve product, process, and psychological uncertainties. The product descriptions might not provide sufficient information, and quality can only be evaluated after trying or inspecting the products. Assuming that consumers only write reviews after they use the products, the reviewers know the actual quality of the product while new consumers do not know. Consumers may be inclined, therefore, to put more weight on a review written by a reviewer with higher quality reputation or higher exposure to the community because his or her reviews are more credible and trustworthy. Reviewer exposure refers to a celebrity reviewer known for his or her name recall. It is different from reviewer reputation which is based on performance-related qualities. For each review posted online, besides review score, Amazon reveals how many customers read that review and how many think that review is “useful”. If you click on the reviewer’s name, you can read all the reviews ever written by that person and how he or she has been ranked. Thus, consumers can get a sense of the reviewer’s reputation and exposure.

Knowledge@SMU: What about online reviews that are not genuine but may be “planted” by a company?

Hu: Nothing can really prevent that from happening. However, we do believe that consumers are smart and can tell the difference between a customer review and a “planted” review. Take the example of Amazon.com which allows their customers to rate the reviews of whatever they’re selling on their website. Any future consumer who is interested to find out more about a particular product on Amazon.com would be able to see whether that particular review or reviews have been useful to other users or not, and why. Unfortunately, not many other sites offer this additional service.