Yelp Reviewers Take a Dimmer View of Nursing Homes

Anna Rahman

The stars are not aligning when it comes to online reviews of nursing homes.

Yelp reviewers give nursing homes significantly less favorable ratings than those found on the government’s nursing home comparison website, according to a new study from the University of Southern California Leonard Davis School of Gerontology.

The Centers for Medicaid and Medicare Services’ (CMS) Nursing Home Compare website and Yelp, the popular online business recommendation site, both feature five-star rating systems to help consumers evaluate nursing homes. Results of the study, published online in the journal BMJ Quality and Safety, show that a nursing home’s Yelp rating differed from the Nursing Home Compare rating more than 75% of the time, and it was significantly lower in three of four categories reviewed.

“Consumers view the quality of nursing homes as poorer than the CMS is reporting it to be,” said Anna Rahman, a co-author of the report and an assistant research professor of gerontology at the USC Leonard Davis School. “It is important to understand why we have this discrepancy in order to help empower consumers to make informed decisions when selecting a nursing home.”

Different Inputs

Susan Enguidanos

Nursing Home Compare rolled-out its five-star rating system in 2009 using data from nursing home inspections, staffing reports and resident assessments. The overall rating combines scores from three areas: health inspections, staffing levels and quality measures, such as changes in physical, functional, psychosocial and cognitive well-being. Health inspections was the only category that rated significantly higher when compared to the Yelp star ratings.

Founded in 2004, Yelp boasts 145 million first-person word-of-mouth reviews of business and service providers. These reviews generally describe a consumer’s personal experience with the provider. Previous research in hospitals shows that Yelp reviewers tend to focus on subjective experiences of health care, such as a reviewer’s personal assessment of staff attitudes, the physical setting or the cost of care.

“Choosing a nursing home for oneself or a loved one is a complex, challenging, often emotionally charged task,” said co-author Susan Enguidanos, associate professor of gerontology at the USC Leonard Davis School. “Our study results highlight the need for further research to improve and strengthen the ratings systems used by nursing home consumers.”

Gaming Concerns

Both Nursing Home Compare and Yelp have faced charges of artificially manipulated ratings. Previous research estimated that faked reviews and ratings on Yelp range from 16 percent to 40 percent. However, the USC researchers say that since gamed Yelp ratings may encompass either or both extremes of the 5-star rating scale, this bi-directional practice, coupled with increasingly savvy consumers and Yelp’s efforts to weed out fake reviews, likely cancels out their ability to impact ratings.

Recent reviews of the Nursing Home Compare system concluded that the rating system provides a financial incentive for gaming and that the practice of nursing homes reporting purposefully inflated data continues, despite modifications designed to discourage it. These findings may shed light on why the Nursing Home Compare ratings are mostly higher than the Yelp ones, according to the study authors.

“If California nursing homes gamed some of the data they reported to CMS, that could explain why Yelp ratings in our study were significantly lower than the nursing home ratings for quality measures and staffing, which are derived from data self-reported by nursing homes,” Rahman said.

Future Work

The authors note that Yelp recently partnered with the New York-based news non-profit ProPublica to report government data about nursing home fines, serious deficiencies and payment suspensions on its review pages. Future studies will aim to analyze the content of Yelp reviews and compare the themes and topics consumers identify to those reported by CMS.

“The Ratings Game: An Analysis of Nursing Home Compare and Yelp Ratings” was published on November 13 in BMJ Quality and Safety. USC Leonard Davis School researchers compared Nursing Home Compare and Yelp ratings for 675 California nursing homes, with a minimum of five Yelp reviews, between September and November 2016.