Restaurant reviews: Taking subjectivity off the menu

Taipei's research-based dining guide sets an objective benchmark

The credibility of restaurant reviews is a sizzling hot topic for both restaurateurs and diners – and with the proliferation of fake and paid-for online critiques, it has become a matter of concern for regulators.

Last year, the Australian Consumer and Competition Commission took up the issue and released a set of guidelines for online platforms and businesses. The consumer watchdog announced it would prioritise monitoring of the veracity of online reviews. 

Fake reviews had become a huge problem, according to John Hart, chief executive of Restaurant & Catering Australia, the industry group that lobbied for the 2014 regulatory intervention.

At the time, Hart estimated one in 10 reviews had been written by aggrieved staff and others with "an axe to grind" about a particular venue. Reviews, he emphasised, can have a huge impact.

Undoubtedly restaurant reviews are proliferating. Manipulators and fraudsters aside, the internet has given rise to a new generation of foodie critics.

Those who have consumed a dud or delectable meal at an eating house can sign on to rating sites such as Dimmi, Urban Spoon, Eatability or Yelp and spout on about their experiences.

It's estimated that about three-quarters of the Australian population is using these sites. Instagram users and Facebookers also post effusive or derisory comments with images snapped on smart phones.

While food bloggers abound, there are also numerous respected and lesser known "public stomachs" whose opinions of eating establishments in newspapers, magazines and guides, print and digital, can make or break a business.

'There’s such a large degree of subjectivity in people’s tastes and preferences, we wanted a proven method of evaluation'

 BERNARD GAN

Trained mystery shoppers

The major concern is that subjectivity rules in the restaurant reviewing game, leaving hungry diners perplexed in the selection process and restaurateurs frantic to improve ratings or retain status. A slip in ranking can prove costly to the business and the chef's reputation.

Subjectivity was top of mind for a team of researchers commissioned by the Taipei City Government to produce a list of the Top 100 recommended restaurants for visitors to the Taiwanese capital, which relies heavily on its tourism allure.

The research team of Chih-Hsing Sam Liu, Ching-Shu Su, Bernard Gan and Sheng-Fang Chou soon realised there was no reliable measurement tool or standardised rating system for restaurants in Asia.

"There's such a large degree of subjectivity in people's tastes and preferences, we wanted a proven method of evaluation," says Gan, a lecturer in the school of management at UNSW Business School.

The researchers conducted an extensive literature review and explored well-known evaluation methods, including New York's Zagat Restaurant Survey, which uses diners' shared opinions to form its ratings, and the Thai Select certification system, which recognises restaurants outside of Thailand serving authentic Thai cuisine.

The popular mystery shopper technique long used by the Michelin Red Guide – the world's most authoritative and widely recognised restaurant and hotel directory – was determined to be the most reliable approach.

"Rather than use untrained everyday consumers, the mystery shopper approach adopts a rigorous process of selecting and training its evaluators before sending them out to restaurants," Gan says.

 New criteria

Famed for its coveted restaurant star ratings from one to three, Michelin uses a symbol system for primary indicators in restaurants, from quality of ingredients, cooking skill and talent through to value for money and consistent experience on each visit.

Intriguingly, Michelin has never disclosed the written criteria for the quality required for its different star ratings. And notably, it zooms in on the food rather than the décor, table settings or service quality.

In their paper, Effective restaurant rating scale development and a mystery shopper evaluation approach, Gan and colleagues highlight the importance of quality of service for diners when choosing restaurants.

For the Taipei Top 100, the research team developed new criteria, categorising the main indicators of exceptional restaurants as meals, service, dining environment and atmosphere, price and reputation/popularity.

An open enrolment process run by the Taipei City Government recruited 48 mystery shoppers from a wide range of backgrounds. All were passionate about food and regularly ate out.

Then followed three days of training covering Western and Chinese cuisines. A 20-question survey was compiled in consultation with a team of experts starting with quality factors such as the temperature, presentation, smell and taste of the food and moving through hygiene and efficiency to ambience, pricing and heritage.

In all, the reviewers dined in 500 restaurants in Taipei, primarily selected for offering table service, with priority given to award winners, gourmet restaurants and others purported to offer high quality dining experiences.

Sophisticated methodologies were employed by the research team to control for variables – discriminating factors – and provide validity. The team also narrowed the 20-questions into four ratings fields: service, ambience, meals, and value for the price.

'Recalling the recent scandal in Australia over frozen fruits from China, increasingly we do not know what is safe to consume' 

BERNARD GAN

Size matters

Despite their mystery shopper training program, the researchers found it was not easy to change a person's preference in a short period of time and the most difficult evaluation to control for was meals.

"It's much easier to have an objective evaluation of the cleanliness and hygiene of the dish and utensils than the food looking and smelling delicious, or whether the excellent taste of the food left a long-lasting impression," Gan says.

Price also affected perceptions. Expectations in all four rating categories were inevitably higher for dining experiences that ended with a big bill and the popularity of the most expensive restaurants was more likely to fall down due to service quality.

"When the level of service fell short, the mystery shoppers decided not to recommend the restaurant – so no repeat business," Gan explains.

Size also mattered for restaurants. Capacity turned out to be a crucial factor affecting ambience and, ultimately, customer perception. Restaurants that could accommodate more than 50 people performed better, according to the reviewers.

"This is an important finding in our study. Space is money – the bigger retail space you rent or buy, the more money you have to invest," Gan says.

"We found in Taipei where retail space in the central district is very expensive, you need to have more than 50 seats if you want to charge more, although how you design and maximise the use of space to create a comfortable dining environment is important as well."

Broader usage

The research team believes their evaluation tool – used as a one-off exercise in Taipei in 2009 – could be employed to investigate restaurant quality in other countries, offering a benchmarking resource to help poorly performing operators lift their game.

It also offers a scientific improvement to the reliability of restaurant reviews, cutting through at least some of the subjectivity.

And Gan believes there's room to broaden rated groups to include cafés and street food and plenty of potential to change criteria as well. For example, environmentally conscious consumers may be interested to see ratings for restaurant energy consumption.

"How would the social responsibility of restaurateurs affect repeat patronage?" he asks.

Another potential field for evaluation is food safety.

"Recalling the recent scandal in Australia over frozen fruits from China, increasingly we do not know what is safe to consume," Gan says.

"People may be prepared to pay more if restaurants can ensure the freshness and safety of their food."