Consumer Reports WebWatch Examines Hotel Booking Sites

Independent Study Reveals Pros and Cons of Wide Variety of Travel Sites and Reveals Potential For More Complexity and Bias

YONKERS, N.Y. — Consumer Reports WebWatch, a project of Consumers Union, the non-profit publisher of Consumer Reports magazine, today unveiled a study that shows the number of unregulated, integrated travel sites on the Internet has grown, and the complexity of booking rooms and subjectivity of information has similarly increased. The study on hotel-booking Web sites compares and contrasts rates and services available through a variety of popular travel sites.

As part of its mission to improve the credibility of service-oriented Web sites, Consumer Reports WebWatch undertook its first independent examination of online hotel-booking services in late February of 2003. The February 2003 project aimed to determine which sites offer consumers the best services as well as to provide consumers with guidelines for getting the most out of the popular sites.

As part of the evaluation process, several of the leading integrated travel sites were analyzed in controlled, real-time, head-to-head testing, in order to determine which sites offered the most thorough displays and lowest rates for hotel rooms. The sites tested included Expedia, Hotels.com, Lodging.com, Orbitz and Travelocity, as well as several branded sites maintained by national hotel chains.

Methodology:
The testing was benchmarked with Sabre, the leading global distribution system (GDS) used by travel agencies in the U.S. of hotel booking sites. All testing was performed by trained individuals and statistical analysis provided by Consumers Union led to the creation of five separate tests, each consisting of 30 entries, across all seven sites. The testing simulated a variety of trips in 30 cities throughout the U.S. and the Caribbean. Consumer Reports WebWatch's research detailed the best and worst sites for searching for hotel rooms by price, location star ratings, and specific brands and properties. The findings of the testing were grouped into four major categories: lowest raters, closest rates, lowest rates by test and lowest average rates.

"For better or worse, the Web has most definitely revolutionized the travel distribution system," said William McGee, a travel journalist and consultant to Consumer Reports WebWatch and the Editor of Consumer Reports Travel Letter from 2000 to 2002. "As more and more consumers turn to the Web to book hotels, it is important that they understand the limitations of hotel-booking sites."

The research project was completely funded by Consumer Reports WebWatch. Outside consulting and GDS benchmarking were provided by New York-based Harrell Associates, Inc., an independent consulting firm.

Among the most significant findings presented by this testing of hotel sites:

• Overall, Travelocity provided the highest number of lowest rates: 29% in 150 tests, outpacing its nearest competitors, Orbitz (21%) and Expedia (17%).
  
  
• The advent of "Web rates" was a potent factor. In 85% of the tests, one or more of the six Web sites provided lower rates than Sabre, the largest GDS used by travel agencies. Sabre and all other GDSs are regulated by the U.S. Department of Transportation to ensure fair and unbiased listings. Although Consumer Reports WebWatch included Sabre as a benchmark for these tests, the lowest rates provided by the GDS were consistently beaten by one or more of the Web sites, sometimes by hundreds of dollars. The poor performance posted by Sabre, however, does not mean that travel agencies have lost a competitive edge. Some travel agents employ a variety of tools to find the best fares and rates, both online and offline.
  
  
• The lowest average rate was provided by Travelocity. Hotels.com, which did not perform very well overall, provided the second-lowest average.
  
  
• Overall, the "branded" sites maintained by hotel chains and specific hotel properties performed rather poorly. But there was one significant exception: the branded sites outperformed all competitors except Travelocity and provided the lowest rates in one out of three cases.
  
  
• Orbitz was especially capable of coming quite close to providing the lowest rates. In 14 separate queries, Orbitz provided rates that were no higher than $5 per night compared to the lowest rates provided.
  
  
• Expedia, in particular, proved to be quite adept at beating competitive rates by a very small margin. In several cases, Expedia edged the nearest competitor by less than $1 per night.

Regarding increasing complexity, one problem common to all five integrated travel Web sites is that, overall, their search tools are not very intuitive in the manner of traditional branded search engines, such as Google or Lycos. These travel sites are quite intolerant when the user incorrectly inputs information; the slightest mistake in spelling or punctuation can render the search useless. In addition, when these intolerances are coupled with slight variances in the listings of specific hotel properties from site to site (e.g., "Four Points by Sheraton St. Louis Downtown" vs. "Four Points by Sheraton Downtown St. Louis"), searching can be even more difficult. It's worth stressing that consumers need to engage in extensive trouble-shooting when searching for hotel rates on these sites. Multiple queries are sometimes necessary to obtain the proper rates.

Bias and usability concerns were also unearthed by the study. For example, even though the integrated travel Web sites tout the power of sophisticated tools to assist in searching for rates and availability, it's quite clear that the sites are not offering consumers simple, easy-to-use functionality that would allow them to search for hotel rates in lowest-to-highest rankings.

In fact, none of these five integrated travel Web sites — Expedia, Hotels.com, Lodging.com, Orbitz, and Travelocity — offered easy functionality to obtain lowest rates during the first search. Obtaining lowest-to-highest listings by price required more advanced searches, when available. Further, these lowest-to-highest search tools were not always clearly labeled for consumers to find.

This was one of the most disturbing findings to evolve during this testing. The supposed value of an integrated site lies in the inherent choices it offers a consumer, rather than the limited choices offered by branded or "dealership-model" sites. Although there is little doubt that these integrated sites have the technological capability to deliver easy-to-use listings of lowest-to-highest hotel rates, they are not readily doing so.

Consumers may well feel that they are not controlling the interface, and that in fact these integrated travel Web sites are pushing preferred suppliers at them.

"The depth and intensity of the testing and investigation of these sites is unprecedented, and the takeaway for consumers is clear: The convenience and exclusive Web fares offered by these sites come with a responsibility to dig deep, and make sure you are really getting as good a deal as you think, at the property, location and time you want," said Beau Brendler, Director of Consumer Reports WebWatch.

These questions will likely remain unanswered. In an effort to help consumers seeking deals on the Internet, Consumer Reports WebWatch has developed practical tips for consumers shopping for a hotel room on the Web.

Consumer Reports WebWatch Tips for Booking Hotels Online

• Keep shopping around. Yes, Consumers Union has issued this warning before — more than once. But it remains the single most important piece of advice to help you find the best travel bargains online. For context: After extensive testing and analysis, Travelocity emerged as the winner in this session of hotel testing. And yet Travelocity's strong performance needs to be put into context. Simply put, Travelocity failed to return the lowest hotel rate more than 70% of the time. All the other sites performed even worse. If it's important to you to find the best rates and fares, you need to consult more than one site.
  
  
• Always ensure that the rate you've found is available for immediate booking. The better sites will provide explanations of their booking policies.
  
  
• Before booking a hotel room online, make sure you've determined the final rate. There may be additional charges, including federal, state, and local taxes; security and lodging fees; and booking or service fees from the hotel and/or the Web site.
  
  
• Beware of potential bias. The screens on the leading independent travel web sites are inundated with banners, pop-ups, pop-unders, and even rate listings that are bought and paid for with hotel advertising and marketing dollars. Make sure that you've examined all the choices before booking.
  
  
• Never book a hotel room online until you've read and understood the cancellation policy. Many independent travel web sites charge additional fees for changes and cancellations, in addition to fees levied by the hotel.
  
  
• If you know the specific hotel property you want to visit, conduct an online search and find the branded web site for that hotel (which may be maintained by either the chain or the property itself). The chances are good that you may find a lower rate there. In fact, when Consumer Reports WebWatch queried rates at specific properties, lower rates were returned by the branded sites in 1 out of 3 cases.
  
  
• When shopping for hotel rates online, always ensure that you are extremely accurate about the information you input. For example, many hotel chains maintain properties with similar-sounding names (e.g., "Marriott Downtown" vs. "Marriott Midtown" or "Holiday Inn Airport" vs. "Holiday Inn Airport North"). If you're comparing rates for a specific property, determine its full address and zip code before comparison shopping on other sites.
  
  
• Be very careful when comparison shopping from site to site. During the course of this testing, Consumer Reports WebWatch found numerous problems with the default functions on some travel sites. Each site may not store and "remember" the information you input as you shop, so the data you entered may be lost and the search engine may revert to incorrect dates or locations.
  
  
• Use a credit card for online purchases. Credit cards generally provide the most federal consumer protections in the United States. Under the Fair Credit Billing Act, your liability for unauthorized charges is limited to $50 — if you report the billing error to the credit card company in writing within 60 days after the bill was mailed to you. Credit card companies and e-merchants may cover this fee in certain situations. Some credit card companies also will let you use a temporary "throw-away" credit card number when making purchases online, so that payments are credited to your actual credit card but without your needing to share electronically your real account number or password. Inquire with your credit card company about this option. You may also want to consider setting aside a single credit card for online use. That way, if a security breach occurs, you will still be able to use your other credit cards.

e-Ratings of hotel-booking sites, based on the results of this report, will be available to consumers later this year. For the near term, consumers can go to www.consumerreports.org to access an article that summarizes the findings from the research report. The full results from this study can be found at www.consumerwebwatch.org.

Consumer Reports WebWatch Also Takes A Closer Look at Search & Health
In addition to the hotel-booking study, Consumer Reports WebWatch today unveiled findings from four new research reports on search engines and health sites at its first national summit on Web credibility, entitled "Building Trust on the Web."

Consumer Reports WebWatch commissioned Context-based Research Group, a Baltimore-based anthropological firm, to conduct an ethnographic study that examined consumers' attitudes toward the pay-for-placement practices employed by some search engine Web sites. The study revealed that all 17 of its participants were surprised to learn commonly used search engines are paid to list some results and sites more prominently than others and changed their search behavior as a result. Specifically, many respondents claimed that they were less likely to trust the first result listed, or even the entire first page of listings, upon learning about the pay-for-placement practice.

While some said they had no real aversion to the practice, they did stress that paid links and ads should be more clearly marked to distinguish them from "pure" search listings. These findings among others prompted some study participants to speculate that the practice would become less tolerable as consumer understanding of pay-for-placement marketing increases. The ethnographic approach enabled Context researchers to spend a significant period of time with each participant in their natural setting, observing what they actually do versus relying on what they say they do.

In January of 2003, the Health Improvement Institute (HII) and Consumer Reports WebWatch initiated a research project to examine the credibility of health Web sites by first looking at guidelines available from various entities. The study found that there are no criteria readily available to consumers or to professionals that allow them to assess the credibility and reliability of health Web sites. In the next phase of the project, the Institute intends to develop, test, and apply such a tool that will produce independent ratings of health Web sites. The tool will allow a user to judge a site's ease of use as well as its purpose, sponsors, currency and sources.

***
 
About Consumer Reports WebWatch
The Consumer Reports WebWatch mission is to investigate; inform; and improve the credibility of information on the Web. ConsumerWebWatch.org will be home to research and analysis on issues of credibility, usability and content, along with news of use to consumers navigating the Web.

Consumer Reports WebWatch is a project of Consumers Union, the non-profit publisher of Consumer Reports magazine and ConsumerReports.org. The project is supported by The Pew Charitable Trusts, which invests in ideas that fuel timely action and results; the John S. and James L. Knight Foundation, which promotes excellence in journalism worldwide and invests in the vitality of 26 U.S. communities; and the Open Society Institute, which encourages debate in areas in which one view of an issue dominates all others.