How written product reviews influence consumer impressions of star ratings

When shopping online, most consumers are drawn to products with a high star rating. Not only are star ratings an easy way for past buyers to assess purchases, they also provide a visual marker that helps potential buyers quickly sort through various options. Quantitative star ratings have long been known to positively affect consumer demand for a product, but less is known about qualitative text (written) reviews. New research from Vanderbilt Business explores the relationships between quantitative star ratings and qualitative text reviews.

“Reading Between the Stars: Understanding the Effects of Online Customer Reviews on Product Demand” investigates the relationship between quantitative star ratings, qualitative text reviews, and product demand. The study is co-authored by Hallie Cho, Assistant Professor of Operations Management at Vanderbilt Business, and Sameer Hasija and Manuel Sosa of INSEAD.

While past research has focused on the effects of star ratings on product demand, this study is the first to explicitly test whether (or not) star ratings and text reviews convey the same information. The study also investigates the interplay between these two reviewing modes and how they affect product demand.

To study these variables, the authors took purchasing data and customer reviews on 416 car models sold in the U.S. auto market from 2002-2013. Each model’s characteristics were entered into a purchasing model to simulate the decision-making process. Then the authors measured market size by dividing the number of models sold each year by the number of households in the U.S.

Next, the authors sourced star ratings and text reviews for these car models from a popular third-party review site. They then completed a text analysis of all the written reviews using a supervised machine learning algorithm to quantify the sentiment expressed. Lastly, the authors compared how the star ratings and the text review sentiments tracked with the sales of different new vehicle models over time.

“One of the more interesting findings is that the sentiments expressed through text reviews are not completely represented in star ratings,” says Cho.

The study finds that text reviews do help to drive product demand and influence consumer demand in several ways. First of all, positive text reviews provide supportive evidence for high star ratings, both of which drive consumer demand. Positive text reviews also help to counteract consumers’ tendency to discount extremely high star ratings for being sponsored or fake. These positive effects tend to diminish the higher the ratings become, so they have the greatest effect on products that initially have a lower star rating. However, if the text reviews are negative, they reinforce the consumer tendency to discount the high star rating.

The research also shows that these two types of reviews capture two slightly different types of assessments, though the information is still complementary. The authors liken this to Daniel Kahneman’s model of “System 1” and “System 2” framework of human cognition, popularized in his best-selling book Thinking, Fast and Slow. The quick and easy nature of star ratings means that they tend to present more automatic, intuitive, and emotional (System 1 thinking). Meanwhile, the longer and more involved nature of text reviews activates a more deliberate and logical assessment (System 2 thinking).

“Star ratings capture a reviewer’s gut reaction—a fast process—whereas text reviews capture the reviewer’s conclusion at the end of a slow thought process,” says Cho. “Even more interesting finding is that the two do not always agree!”

When buyers are making complex and expensive purchases such as a new vehicle, extremely high star ratings prompt them to seek out corroborating evidence such as text reviews. If the text reviews are less enthusiastic than the intuitive ratings, potential buyers may take a step back and seek out purchasing alternatives.

The study, “Reading Between the Stars: Understanding the Effects of Online Customer Reviews on Product Demand,” is published in Manufacturing and Service Operations Management.