Booking Hotels Online:

An In-Depth Examination of Leading Hotel Web Sites (Full Report)

Author: William J. McGee, Consultant to Consumer Reports WebWatch,

TABLE OF CONTENTS

Executive Summary

Consumer Reports WebWatch Mission Statement

Consumer Reports WebWatch Travel Projects Slated for 2003

History: The Growth of Online Hotel Distribution

Hotel Project Testing Methodology and Parameters
Selection of Web Sites
Methodology
Testing Parameters
Specific Tests
Inclusion of Multiple Results
Currency Issues
Availability
Valid Tests
Invalid Data
System Failures

Questions of Potential Bias
The Hedna White Paper
Searching by Lowest Rates
Availability of Rates
Other Findings

On-Site Advertising
Promotional Rates
Omissions of Chains or Properties
"Wacky Properties"

The Lodging.com Ownership Issue

Testing Results and Conclusions
Lowest Hotel Rates
Closest Hotel Rates
Lowest Hotel Rates by Test
Hotel Online Service Fees
Hotel Online Customer Service Policies

The Hotels.com Price Guarantee

Consumer Reports WebWatch Tips for Booking Hotels Online

Appendix I: Ease-of-Use Factors

Appendix II: Brief Recap of Findings of Consumer Reports Travel Letter Projects

Fig. 1: Lowest Hotel Rates

Fig. 2: Lowest and Closest Hotel Chains

Fig. 3: Lowest Hotel Rates by Test

Fig. 4: Online Service Fees

Fig. 5: Online Customer Service Policies

Executive Summary

The consumer's need for reliable, accurate, and unbiased information when booking travel has never been greater. Unfortunately, the process of booking travel has never been more complicated, thanks in large part to the rapid growth of the Internet and the emergence of unregulated integrated travel Web sites. This growth has been so significant that travel has become the largest source of commerce on the Internet. Yet the potential for biased or incomplete displays has grown greater.

This research report summarizes the work undertaken by Consumer Reports WebWatch, which launched an extensive examination of hotel booking sites. Several of the leading integrated competitors were analyzed in controlled, real-time, head-to-head testing, in order to determine which Web sites offered the most thorough displays and lowest rates. The sites tested were Expedia, Hotels.com, Lodging.com, Orbitz, and Travelocity, as well as several branded Web sites maintained by national hotel chains. The testing was benchmarked with Sabre, the leading global distribution system (GDS) used by travel agencies in the U.S.

Among the most significant findings presented by this testing of hotel sites:

• Since most of the lowest hotel rates are being provided through the unregulated medium of the Internet, it's becoming harder and harder to uncover and address issues of bias on integrated travel Web sites. This report addresses Consumer Reports WebWatch's findings in this area.
  
  
• None of the five integrated travel Web sites tested — Expedia, Hotels.com, Lodging.com, Orbitz, and Travelocity — offered easy functionality to obtain lowest rates during the first search. Obtaining lowest-to-highest listings by price required more advanced searches, when available.
  
  
• Overall, Travelocity provided the highest number of lowest rates: 29% in 150 tests, outpacing its nearest competitors, Orbitz (21%) and Expedia (17%).
  
  
• The advent of "Web rates" was a potent factor. In 85% of the tests, one or more of the six Web sites provided lower rates than Sabre, the largest GDS used by travel agencies. Sabre and all other GDSs are regulated by the U.S. Department of Transportation (DOT) to ensure fair and unbiased listings. Although Consumer Reports WebWatch included Sabre as a benchmark for these tests, the lowest rates provided by the GDS were consistently beaten by one or more of the Web sites, sometimes by hundreds of dollars.
  
  
• The poor peformance posted by Sabre does not mean that travel agencies have lost a competitive edge. Some travel agents employ a variety of tools to find the best fares and rates, both online and offline.
  
  
• Overall, the branded Web sites maintained by hotel chains and specific hotel properties performed rather poorly. But there was one significant exception: When rates were queried at specific properties, the branded Web sites outperformed all competitors except Travelocity and provided the lowest rates in 1 out of 3 cases.
  
  
• Orbitz was especially capable of coming quite close to providing the lowest rates. In 16 separate queries, Orbitz provided rates that were no higher than $5 per night compared to the lowest rates provided. Hotels.com was also very good at returning close rates, by providing 14 separate rates that were no higher than $5 per night compared to the lowest rates.
  
  
• Expedia, in particular, proved to be quite adept at beating competitive rates by a very small margin. In several cases, Expedia edged the nearest competitor by less than $1 per night.

The project was completely funded by Consumer Reports WebWatch. Employees of Consumer Reports WebWatch assisted in drafting the methodology, participated in the testing, and contributed to this research report. It was edited by Beau Brendler, the director of Consumer Reports WebWatch.

The project was directed by William J. McGee, a travel journalist and consultant to Consumer Reports WebWatch and the Editor of Consumer Reports Travel Letter from 2000 to 2002. Outside consulting and GDS benchmarking were provided by New York-based Harrell Associates, Inc., an independent consulting firm.

back to top

Consumer Reports WebWatch Mission Statement

Consumer Reports WebWatch is a project of Consumers Union, the non-profit publisher of Consumer Reports magazine. The project is supported by grants from The Pew Charitable Trusts, the John S. and James L. Knight Foundation, and the Open Society Institute. Consumer Reports WebWatch in turn funded the research and production of this report, as it has done with similar reports on Web site credibility, airline ticket-booking Web sites (performed and published in conjunction with Consumer Reports Travel Letter, which ceased publication in December 2002), search engines, and health Web sites. Consumer Reports WebWatch's mission is to improve the credibility of Web sites, through research, through articulation of best practices guidelines in specific sectors of Web publishing, and by working with ConsumerReports.org to produce ratings of Web sites using those guidelines. Ratings of these sites will be forthcoming in 2003, using research conclusions from this report. Consumer Reports WebWatch's research, guidelines, and e-Ratings are available for free at http://www.consumerwebwatch.org.

back to top

Consumer Reports WebWatch Travel Projects Slated for 2003

Consumer Reports WebWatch remains committed to testing, researching, and reporting on travel Web sites, particularly since travel is the single largest source of online commerce. Throughout 2003, it will continue the work that was begun in Consumer Reports Travel Letter, by providing unbiased, detailed, accurate, and repetitive testing of travel Web sites.

Until now, Consumers Union's testing of travel Web sites was focused on purchasing airline tickets. With this project, Consumer Reports WebWatch broadened that focus to include shopping for hotels, a fast-growing segment of the online travel market. In future months, Consumer Reports WebWatch will analyze car rental Web sites. In addition, Consumer Reports WebWatch will also examine alternative travel Web sites, such as "blind-bidding" and "opaque" sites.

back to top

History: The Growth of Online Hotel Distribution

Distribution of travel products is unlike the distribution methods used in most industries. Consumers face an array of choices when buying travel, and each of these choices — including those that are convenient and those that are revolutionary — carries particular concerns, potential hazards, and a variety of costs particular to the travel industry.

Consider that a consumer who wishes to book a one-night stay in a moderately-priced hotel can do so through a vast array of channels. These methods could include:

• Calling the hotel chain's toll-free reservation line;
  
  
• Calling the hotel property directly;
  
  
• Using the paid services of a travel agency, that may or may not maintain specific booking agreements and bonus commission programs with that hotel chain or property, and may book the hotel through a global distribution system (GDS) or directly online;
  
  
• Using the services of a third-party consolidator or broker, either in-person, by telephone, or online;
  
  
• Using a corporate travel management program, that may provide pre-negotiated rates for specific hotel chains or properties;
  
  
• Using an integrated travel Web site, such as those tested by Consumer Reports WebWatch;
  
  
• Using a branded hotel chain Web site;
  
  
• Using a branded hotel property Web site;
  
  
• Using an "opaque" bidding Web site that may or may not reveal the property's identity prior to a confirmed booking;
  
  
• Booking directly in person upon arrival at the property.

Each of these channels provides distinct benefits, potential drawbacks, and costs for consumers, as well as varying degrees of cost and compensation for the hotel companies and their intermediaries. For example, in 2001 the Hotel Electronic Distribution Network Association (HEDNA) estimated that the average GDS booking fee cost $4.50 for a hotel company, while a travel agency commission cost the same hotel company between 5% and 10%, a figure which could total hundreds of dollars per booking.

Several issues are clear. One is that the Internet has become the least expensive method for travel suppliers (including hotel companies) to distribute their products. Another is that many travel companies are using all the tools at their disposal — including discounted rates and bonus loyalty-program points — to direct more consumers to the Web in an effort to reduce distribution costs. And another is that the percentage of travel booked on the Internet is rapidly growing.

David Litman, the Chairman and CEO of Hotels.com, predicted at an industry conference in November 2002 that 5% of hotel rooms will be booked online this year, in contrast to 15% of airline tickets. However, Litman estimated that the hotel figure would rise to 15% by 2005. There is widespread agreement within the travel technology sector that hotels represent the fastest-growing portion of online bookings.

In this testing, Consumer Reports WebWatch has focused on integrated travel Web sites, as well as a leading GDS and several branded Web sites maintained by hotel chains and specific hotel properties. It's helpful to recall how these separate channels developed.

The first computer reservations systems (CRSs) trace their roots to the launching of Sabre by American Airlines in 1964. As the technology improved, the CRSs grew, along with concerns about the potential bias inherent in reservations systems owned and operated by airlines. Throughout the 1970s and early 1980s the federal government investigated such issues, and in 1984 the U.S. Department of Transportation (DOT) began overseeing the implementation of rules governing CRSs, which eventually became known as global distribution systems (GDSs). Although the DOT rules specifically applied to airline bookings, the practices and methodologies of listing the availability and pricing of airline tickets in an ordered, fair, complete, and unbiased manner extended to other travel products, including hotels.

The issue of bias in travel distribution channels is not new. Many consumer organizations, including Consumers Union, have long voiced concerns over bias in GDSs; bias in travel agency "override" and bonus commission programs; and, most recently, bias in the displays of integrated travel Web sites.

Throughout the short history of the Internet, travel Web sites have set the pace for growth. Today the leading analysts of Web commerce all state that travel is the single largest source of Internet revenue. And it's important to note that this applies not just to online traffic ("looking") but also to online sales ("booking").

The three leading integrated travel Web sites — Expedia, Orbitz, and Travelocity — are all included in this testing. During the last several years, and particularly after the terrorist attacks in September 2001, several integrated travel Web sites have consolidated or shut down. Other integrated travel Web sites, including some specifically tailored for hotel bookings, have been launched.

That said, some analysts believe there are now significant barriers for new-entrant online travel Web sites. In late 2002, industry analyst Tom Underwood of Legg Mason Wood Walker Inc., said, "I don't think we're going to see any new hotel players in the next year." Another travel analyst, Scott Barry of Credit Suisse First Boston, predicted that 2003 will see "the emergence of a two-tier market, where there are haves and have-nots."

The testing conducted by Consumer Reports WebWatch and detailed in this report addresses the viability of integrated travel Web sites for providing viable and competitive hotel rates.

back to top

Hotel Project Testing Methodology and Parameters

This examination of hotel booking Web sites is the first independent online travel project undertaken by Consumer Reports WebWatch. (In 2002, Consumer Reports WebWatch had teamed with Consumer Reports Travel Letter to analyze independent airline booking sites.) What follows is an in-depth presentation of the testing methodology and parameters.

back to top

SELECTION OF WEB SITES

Consumer Reports WebWatch analyzed market share data and other factors before deciding which Web sites to include in this testing. Certain sites were eliminated because they are sister companies of the sites tested.

Not available at the time of testing was Travelweb.com, a Web site owned by five of the leading hotel chains — Hilton, Hyatt, Marriott, Six Continents, and Starwood — and Pegasus, a hotel technology provider. Travelweb.com was scheduled to launch as this report was being prepared.

For future reports, Consumer Reports WebWatch plans to test Travelweb.com, as well as other sites that provide hotel rates, including Hotwire, Priceline, and QuikBook.

In addition, since this testing was conducted, some of the Web sites tested may have updated or changed the booking tools available to consumers. In particular, Orbitz unveiled a new hotel-booking product after this testing was completed.

For these tests, five Web sites were chosen, including the three largest integrated travel sites:

• Expedia (www.expedia.com), formerly owned by Microsoft, is owned by USA Interactive, owner of Hotels.com and other travel distribution products. On March 19, 2003, after this testing was complete, USA Interactive announced the purchase of all remaining shares of Expedia.
  
  
• Orbitz (www.orbitz.com), which is owned by the nation's five largest airlines: American Airlines, Continental Airlines, Delta Air Lines, Northwest Airlines, and United Airlines.
  
  
• Travelocity (www.travelocity.com), which is owned by Sabre, the global distribution system (GDS) formerly owned by AMR, the parent company of American Airlines.

Also included were two of the leading hotel-specific Web sites:

• Hotels.com (www.hotels.com), a leading discount hotel Web site owned by USA Interactive. On March 19, 2003, after this testing was complete, USA Interactive announced the purchase of all remaining shares of Expedia.
  
  
• Lodging.com (www.lodging.com), a Web site purchased in 2002 by Cendant Corp., a travel conglomerate that owns multiple hotel brands, including Days Inn, Ramada, Super 8, and Travelodge. Cendant also owns Cheaptickets.com, Trip.com, and the Galileo GDS.

In addition, Consumer Reports WebWatch benchmarked each of these tests by including the following:

• Branded hotel chain and property sites from several companies, including Hilton, Hyatt, Marriott, and Sheraton.
  
  
• Sabre, the largest GDS used by travel agencies in the U.S.

back to top

METHODOLOGY

All testing conducted by Consumer Reports WebWatch was performed by trained individuals upon completion of dry-run testing. Statistical analysis provided by Consumers Union led to the creation of five separate tests, each consisting of 30 separate entries, across all seven sites, for a total of 1,050 tests. All testing was scheduled in advance and completed simultaneously in real-time. Testing times were varied throughout the course of two weeks in late February 2003.

back to top

TESTING PARAMETERS

• Each test consisted of searching for hotels in U.S. dollars in 30 cities throughout the mainland United States, Hawaii, Puerto Rico, and the U.S. Virgin Islands. These cities represented a mix of both leisure and business destinations. The cities were:

• Each test consisted of searching for a non-smoking room for two adults (with no children), with at least a queen-sized bed. NOTE: Not all sites allowed such specificity for each test, but these parameters were established in advance to ensure consistency.
  
  
• In order to simulate a variety of trips, the advance booking windows varied. The approximate times were:
  
  TEST #1: less than 7 days in advance
  TEST #2: 10 weeks in advance
  TEST #3: 14 days in advance
  TEST #4: 6 months in advance
  TEST #5: 3 months in advance
  
  
• In a further effort to simulate a variety of trips, the length of stay varied from two nights to seven nights.

back to top

SPECIFIC TESTS

Each of the five tests used a separate search criterion. These criteria were:

• TEST #1: Searching by the lowest fare. This test allowed broad parameters for the location of hotels with a minimum of three stars.
  
  
• TEST #2: Searching by a specific location within a city. These locations consisted of a variety of downtown, historic, airport, and resort locales. They included Times Square in Manhattan, Capitol Hill in Washington, O'Hare International Airport in Chicago, the French Quarter in New Orleans, Kaanapali in Maui, etc. NOTE: LODGING.COM COULD NOT PERFORM THIS FUNCTION AND WAS NOT INCLUDED IN THIS TEST ONLY.
  
  
• TEST #3: Searching by hotel "star rating" or "class." This test required searching by stars in Expedia, Orbitz, Sabre, and the branded sites; searching by category (such as "luxury") in Lodging.com and Travelocity; and using the "search by quality" function in Hotels.com.
  
  
• TEST #4: Searching by a specific hotel chain. This test required selecting brands in Expedia, Hotels.com, Orbitz, Sabre, and Travelocity and using the "view all hotels" function in Lodging.com. Each test was benchmarked with the branded site of one of four hotel chains. The specific chains used were Hyatt, Hilton, Marriott, and Sheraton.
  
  
• TEST #5: Searching by a specific hotel property. This test required selecting the property by name in Expedia, Hotels.com, Orbitz, Sabre, and Travelocity and using the "view all hotels" function in Lodging.com. Large properties were used, such as the Marriott Marquis in New York City's Times Square, the MGM Grand in Las Vegas, and the Renaissance Hollywood Hotel in Los Angeles. Web searches were conducted for all 30 hotels. In most cases, benchmarked rates were obtained through the specific property pages of the chain's Web site; in several cases, rates were obtained through individual Web sites maintained by the property itself.

back to top

INCLUSION OF MULTIPLE RESULTS

With this test of hotel sites, Consumer Reports WebWatch greatly expanded upon the methodology employed by Consumer Reports Travel Letter, which had ranked airline sites based solely on the first fares returned. Since 2000, many travel Web sites have dramatically improved upon the simple-text listings of fares and rates pioneered by the GDSs used by travel agencies. Several independent Web sites criticized Consumers Union for only judging first-returned prices. In addition, Consumer Reports WebWatch felt that there were other parameters to consider when shopping for a more complex product such as a hotel room.

Therefore, Consumer Reports WebWatch broadened the testing methodology to include multiple hotel rates provided in each test. In all cases, the best rate was culled from either the first five returns or the first full page of returns, whichever was greater. The same parameters were used for all Web sites. In addition, Consumer Reports WebWatch presented broader test results for a richer sampling. This was done by tabulating results for "close" rates, which were rates that were no higher than $5 per night compared to the lowest rate provided.

back to top

CURRENCY ISSUES

All rates were provided in U.S. dollars. None of the rates included federal, state, or local taxes or fees. In all cases, amounts were rounded to the nearest dollar. And all rankings included ties.

back to top

AVAILABILITY

Consumer Reports WebWatch did not book any of the rates provided. In all cases, the Web sites stated that the rooms and dates requested were available.

back to top

VALID TESTS

All of the Web sites were evaluated solely on valid tests. All invalid tests were eliminated due to an error on the part of Consumer Reports WebWatch. These errors included incorrect data entries, search errors, and printing errors. There were no more than five errors (3%) for any one Web site.

back to top

INVALID DATA

Through no fault of the Consumer Reports WebWatch testers, all of the sites failed to return valid data on some tests. In some cases, these failures affected their final results. There were a variety of factors involved, including:

• Results outside the specific parameters requested;
  
  
• No availability of rooms in that location or on those dates;
  
  
• The inability to provide rates outside the Continental U.S.;
  
• Technical and system failures.

back to top

SYSTEM FAILURES

Our testers experienced only one complete system failure, with Expedia during Test #1. For several minutes the tester received repeated notifications that "an internal error had occurred" and Expedia was "unable to continue processing your request at this time." It resulted in one invalid finding. Testing resumed without incident.

back to top

Questions of Potential Bias

That some integrated travel Web sites are biased in their displays of rates and fares is not in question. What remains difficult for consumers to determine is how and when such bias is in play. And it is a critical component in determining which site or sites consumers should use, both in searching for rates as well as in booking those rates.

Chairman Litman of Hotels.com, said last year: "We do not have a pay-for-placement system." Other integrated travel Web sites have issued similar statements.

Yet serious questions remain about the overall quality of the data provided by integrated travel Web sites.

back to top

THE HEDNA WHITE PAPER

A recent industry White Paper provides helpful insights into understanding online hotel bias issues.

The Hotel Electronic Distribution Network (HEDNA) is a not-for-profit trade association with worldwide membership from more than 200 companies involved in hotel distribution. This membership includes global distribution systems (GDSs), Web sites, and other "electronic means." The organization states that it "brings all segments of the hotel industry together to evolve systems and services into electronic distribution that is easy and efficient." The HEDNA mission is "to increase hotel industry revenues and profitability from electronic distribution channels and to be the foremost travel industry association advancing hotel electronic distribution."

In 2001, HEDNA published a White Paper entitled "Biasing — What You Need to Know." The 126-page document details types of hotel distribution bias, where it occurs, and how it "impacts revenue and distribution planning and how to plan for that impact."

The White Paper's Executive Summary states: "The precedent for delivery of unbiased information appears to be changing. Increasingly the hotel industry is citing examples where information delivered on travel displays is no longer non-discriminatory or neutral, but displays a preference to another hotel or group of hotels."

The White Paper further asserts: "Downward pressure on the cost of distribution has forced intermediaries, both old and new, to look for new revenue streams outside of the traditional commission-based model." The White Paper concludes: "In most cases, the use of techniques to introduce bias is considered a strategic decision with the opportunity for all to participate." And it states: "The previously protective unbiased environment in which hotels have traditionally operated is disappearing."

HEDNA's White Paper identifies four types of hotel distribution bias. The summaries below are culled from HEDNA's own descriptions:

• User-defined bias. This occurs when a reservation system offers the user (such as a consumer) the opportunity to define the order or preference of a display at the point of booking. It should be noted that Consumer Reports WebWatch does not consider this form of bias inherently harmful, provided the consumer chooses the methodology to search for a fare or rate (e.g., by date, location, company brand name, etc.).
  
  
• Bias resulting through merchandising practices. This refers to new techniques that include point-of-sale advertising and premium placement of hotels on availability displays, including options to pay for priority on-screen placement.
  
  
• Bias driven by marketing practices. This refers to negotiated rate programs between hotel companies and travel agencies, so that special rates unique to that travel agency are viewable only by that travel agency in the GDS. On-screen displays viewed by the travel agency can be constructed to reflect participating hotels first.
  
  
• Bias as a result of commercial relationships. This refers to ownership of certain distribution channels, particularly on the Internet. Such ownership may influence the number of participants that can be distributed through the channel, the cost to participate in the channel, or the ongoing premium placement of owner product in comparison to other channel participants. HEDNA states: "Opinion suggests that there may be an intrinsic bias in marketplaces created by travel players that inhibit other travel providers from participating."

HEDNA further states: "Overall, opinion points to the existence of bias that can work in the favor of hotels."

Obviously, several of the practices detailed above directly affect travel Web sites that offer integrated listings of hotels. As noted elsewhere in this report, airline listings in the GDSs are regulated by the DOT. But integrated travel Web sites are free to list airlines, hotels, and other travel products in any way they choose. The potential for biased displays is great.

Consumer Reports WebWatch's extensive testing discovered troubling patterns that raise serious questions about biased hotel displays in these integrated travel Web sites. These questions will likely remain unanswered. What follows are details of those patterns.

back to top

SEARCHING BY LOWEST RATES

Even though these integrated travel Web sites have harnessed the power of sophisticated and state-of-the-art technological tools to assist in searching for rates and availability, it's quite clear that the consumer is not being offered the maximum benefits of such technology. Specifically, these integrated Web sites are not offering consumers simple, easy-to-use functionality that would allow them to search for hotel rates in lowest-to-highest rankings.

In fact, none of these five integrated travel Web sites — Expedia, Hotels.com, Lodging.com, Orbitz, and Travelocity — offered easy functionality to obtain lowest rates during the first search. Obtaining lowest-to-highest listings by price required more advanced searches, when available. Further, these lowest-to-highest search tools were not always clearly labeled for consumers to find.

This was one of the most disturbing findings to evolve during this testing. The supposed value of an integrated Web site lies in the inherent choices it offers a consumer, rather than the limited choices offered by branded or "dealership-model" sites. Although there is little doubt that these integrated Web sites have the technological capability to deliver easy-to-use listings of lowest-to-highest hotel rates, they are not readily doing so.

Consumers may well feel that they are not controlling the interface, and that in fact these integrated travel Web sites are pushing preferred suppliers at them.

These are some of the specific problems:

• EXPEDIA. There is no "search lowest-to-highest" option on the initial hotel search page. Further, the "Expedia Special Rate" page does not automatically rank the results from lowest to highest price, so the listings are quite scattered. The consumer can obtain lowest-to-highest listings only after viewing the "Expedia Special Rate" returns. At this stage there is the option of customizing the search by using a drop-down menu on the side of the screen; the lowest-to-highest option is at the bottom of this drop-down menu.
  
  
• HOTELS.COM. The default sort mechanism for Hotels.com searches is "best value." It's not clear what this means. Hotels.com does not allow you to reorder the results of an entire search and sort by price. Instead, it sorts each page of results by price. In addition, you cannot sub-sort hotel rates by star rating or class, so during such searches (as in TEST #3), the desired properties are scattered throughout the listings.
  
  
• LODGING.COM. The site's initial hotel search options — for "U.S. Hotels" and "International Hotels" — do not allow sorting by rates. More disturbing, however, is that in order to find prices that are not "exclusive rates" you must click through to find the "check availability" function, and this is not clearly labeled. When obtained, these "other rates" are often lower than the "exclusive rates."
  
  
• ORBITZ. When a user is searching for rates, the site issues a statement: "We're searching all available rooms to find the best rates for the property you selected." However, to obtain the lowest rates first, you must search a second time, after clicking on "list hotels by price." In TEST #3, when searching by hotel star rating or class, Orbitz did not list properties in price order.
  
  
• TRAVELOCITY. The site's "Search Tips" (which are listed under "Low Rate Tips") states: "Be sure to review all rates listed as rates are often not listed in value order." However, it is not stated what criteria is used for such sortings in place of "value order." Further, the more specific the search criteria, the harder it is to obtain the intended results. For example, Travelocity offers limited character fields to search by a specific point of interest or hotel property.

It's important to reiterate that throughout this testing Consumer Reports WebWatch employed a liberal standard in evaluating the lowest rates provided by these integrated travel Web sites. The lowest rates for each query were culled from the first five returns or the first full printed page of returns, whichever was greater. And the same parameters were used for all sites. This differed from the previous travel Web site testing conducted by Consumer Reports Travel Letter, which evaluated only the first fares returned when querying airfares.

That said, it's difficult not to understate the importance of the placement of hotel rates within the onscreen displays of the integrated travel Web sites. As the HEDNA White Paper states: "For hotels that are relatively unknown or in cities that are highly competitive the importance of being on the first or second screen increases. Using bias to increase that likelihood delivers a competitive advantage to the hotel. In some cases it is believed that to be at the top of a screen implies perception as a preferred hotel."

back to top

AVAILABILITY OF RATES

Throughout this testing, the rates that were provided for specific queries were telling. But in some ways the rates that were not provided were just as telling. Specifically, there were multiple instances when these integrated travel Web sites failed to provide any rates at all, even though their competitors were offering rates for those same queries at the same time.

TEST #5 (searching by a specific hotel property) provided the clearest apples-to-apples comparison, since all five integrated travel Web sites, along with Sabre and the branded hotel Web sites, were simultaneously queried for the exact same properties. For every search, at least two of the integrated Web sites were able to provide rates. (For one city in TEST #5 [Milwaukee], Hotels.com and Travelocity were the only sites to provide rates.) Yet in multiple cases, several of the Web sites failed to return any rates at all.

Here's a breakdown of the failure to return rates for the 30 cities queried in TEST #5:

To a lesser extent, this was also borne out in TEST #4 (searching by a specific hotel chain). Once again, some of the integrated travel Web sites failed to provide rates even though there were rates available for all 30 queries.

Here's a breakdown of the failure to return rates for the 30 cities queried in TEST #4:

Sabre, which is regulated by the DOT to provide complete, fair, and accurate listings of available properties and rates, provided rates for 28 of the 30 queries in TEST #5 and 30 of the 30 queries in TEST #4.

So why, then, did some of these Web sites fail to return any rates for certain chains or properties when it was apparent that there were rates available for the dates in question? Obviously this question reflects on the inventories of hotel rooms maintained by these integrated travel Web sites. And it may reflect on the commercial agreements signed by these integrated travel Web sites and the hotel chains and/or properties in question.

There's much that is unclear about such agreements. But what is clear is that consumers are not always given a complete range of choices when searching for hotel rates on some integrated travel Web sites. Obtaining a full range of choices is available only to those consumers who extensively comparison shop.

back to top

OTHER FINDINGS

The sheer repetition inherent in this testing yielded interesting and sometimes strange findings. The issue of potential bias is a sensitive one, with serious implications for both consumers and the owners of these integrated travel Web sites. Yet any frequent user of the sites tested will be exposed to repeated reminders that these Web sites maintain strong commercial ties to the hotel properties they are listing.

• On-Site Advertising
  
  All five of these integrated travel Web sites — Expedia, Hotels.com, Lodging.com, Orbitz, and Travelocity — accept advertising from travel suppliers, including hotel chains. This advertising includes "banner ads" across the top of the screens, "pop-up" or "pop-under" ads, boxed ads, and ads imbedded into the listings of rates. These last types of ads are particularly disturbing, since some consumers may not, in fact, realize that these imbedded listings are paid advertisements.
  
  For example, when using Expedia to search for hotel rates in Orlando, each test generated an "Expedia Special Rate" with an advertisement for Walt Disney World Resort hotels (although hotel rates were not specifically quoted). This advertisement always appeared first in the listings of rates. Interestingly, this Disney ad even appeared in TEST #5, when querying a rate specifically for the Crowne Plaza property near Universal Studios.
  
  
• Promotional Rates
  
  Most integrated travel Web sites provide opportunities for hotel chains and other travel suppliers to offer promotional rates through their sites. Orbitz, for example, maintains the OrbitzSavers, a promotional rate hotel program. It promises hotel suppliers that it will "maximize sales" to achieve "higher net yields." A notice on the Orbitz site states: "Orbitz marks up, specially provides, and sells your rooms." It's unclear what "specially provides" means in this context.
  
  Throughout this testing, promotional rates were mixed into the integrated listings of lowest rates. For example, this was noted above with Lodging.com. When searching for rates in this site, if you click "check availability," Lodging.com returns listings of "exclusive rates." However, if you click through to the advanced search feature, you can select "other rates" for more detailed listings. In fact, we found that "other rates" were often lower than "exclusive rates." Unfortunately, some consumers may not realize that other options are available and will assume that the exclusive rates are always the lowest rates. In addition, it appears certain that many consumers will not thoroughly search all of the listings provided by any of the integrated Web sites if those listings are not sorted by price.
  
  Just as many traditional travel agencies maintain "preferred supplier" agreements with hotel chains and other travel companies, these types of deals have expanded to integrated travel Web sites. For example, Hilton Hotels announced in April (after this testing was completed) that it had signed a preferred hotels agreement with Expedia. Under the terms of this deal, the Web site gained direct connection to Hilton's central reservation system. In exchange, Hilton said it had "secured from Expedia the most favorable merchant model terms in the hotel industry."
  
  
• Omissions of Chains or Properties
  
  In some cases, the integrated travel Web sites did not list certain hotel chains or properties at all. For example, when searching for a Hilton property in Las Vegas, Lodging.com stated: "No properties found." However, Hilton's own branded site listed four separate properties within Las Vegas. It's not clear if Lodging.com, which is owned by the Cendant hotel chains, intentionally omitted these properties or simply did not provide them in its inventory. But consumers expecting a complete integrated list of properties are not receiving it.
  
  
• "Wacky" Properties
  
  As noted elsewhere in this report, in its testing of integrated travel Web sites for lowest airfares, Consumer Reports Travel Letter repeatedly found bargain prices for "wacky flights." These were itineraries that did not match those requested or that simply did not make sense (such as a flight between New York City and Chicago that consisted of eight legs and a one-way flight time of 11 hours, 29 minutes — without delays).
  
  The equivalent of "wacky flights" when searching for hotel rates were "wacky properties." Simply put, these were hotel properties that are outside the parameters requested and yet they were prominently listed. For example, several sites repeatedly returned rates for hotel properties in New Jersey and Long Island when specifically queried for rates in the New York City borough of Manhattan.
  
  In addition, Hotels.com did not always respond to specific requests. When queried for rates in San Francisco, the site provided properties far out in the east Bay Area, more than an hour's drive from the city. And in TEST #5, when specifically queried for rates at the Hilton San Francisco Fisherman's Wharf, Hotels.com provided rates at a Hilton property in Newark, Calif.
  
  To its credit, Orbitz provided distances in miles for specific listings in such cases. For example, when queried for a rate in Boston, it provided a hotel property in Peabody, Mass., but also noted that it was 12 miles from the city.

back to top

THE LODGING.COM OWNERSHIP ISSUE

Of the five integrated travel Web sites tested, Lodging.com is the only site that is wholly owned by a hotel company. The site is owned and operated by Cendant, which owns other travel Web sites, the Galileo GDS, car rental firms, and nine hotel chains. These brands are Amerihost Inn, Days Inn, Howard Johnson, Knights Inn, Ramada, Super 8, Travelodge, Villager, and Wingate Inn.

There were only two tests in which Lodging.com was given the opportunity to select either Cendant or non-Cendant hotel brands. These were TEST #1 (searching by the lowest rate) and TEST #3 (searching by hotel star rating or class). This was because Lodging.com was not included in TEST #2 (searching by a specific location within a city) because it could not perform this function. And the two remaining tests — TEST #4 (searching by a specific hotel chain) and TEST #5 (searching by a specific hotel property) — did not allow the site to choose the hotel brand.

There was no discernible pattern that Lodging.com provided lower rates for Cendant hotel brands over non-Cendant brands. In fact, in TEST #3, the site did not provide any of its lowest rates at Cendant hotels.

In TEST #1, Lodging.com provided lowest rates at Cendant hotels for 6 of the 30 cities (20%). By contrast, among the lowest rates provided by Sabre and the four other integrated sites for these same 7 cities, the lowest rates were provided at Cendant hotels in only 5 of 35 cases (14%). In the former 6 cases, the Lodging.com/Cendant connection proved to be an advantage for consumers in that Lodging.com provided the lowest rate overall in 3 of these 6 cities.

However, that same connection may be a disadvantage as well. As indicated by the attached charts, Lodging.com did not perform well overall in this testing. A valid but unanswered question is whether Lodging.com is a viable search alternative for all hotels or only for Cendant hotels.

back to top

Testing Results and Conclusions

The findings of Consumer Reports WebWatch's hotel Web site testing are presented below. There are three major categories of findings:

• Lowest rates
  
• Closest rates
  
• Lowest rates by test

Also detailed in this section is information on the travel Web sites' fee structures and customer service policies. Ease-of-use factors are discussed in Appendix I.

back to top

LOWEST RATES

The extensive and repetitive testing conducted by Consumer Reports WebWatch yielded rich results and revealing data. But perhaps one of the most significant findings was the performance recorded by Travelocity. In head-to-head tests with its competitors, Travelocity provided the lowest hotel rates in 29% of the tests (including ties). As Figure 1 shows, the nearest competitors were Orbitz (21%) and Expedia (17%). Hotels.com and Lodging.com did not perform as well as their deep-pocketed competitors, the technologically advanced "Big Three" of Expedia, Orbitz, and Travelocity.

Among the most important findings:

• One of the primary findings of this hotel testing was that Web rates now dominate the travel distribution landscape. In fact, the emergence and importance of "Web rates" cannot be stressed enough. Four of the integrated travel Web sites provided a higher percentage of lowest rates than Sabre, the largest GDS used by travel agencies. Only Lodging.com failed to provide more lowest rates than Sabre. Within the travel industry, there is an ongoing and heated battle over providing more Web-based fares and rates to travel agencies and corporate travel managers through other channels, such as GDSs.
  
  
• Sabre, the benchmark for these tests, performed rather poorly. It provided the lowest rates only 15% of the time. In many cases, the lowest Sabre rate was beaten by hundreds of dollars by one or more of the Web sites. This contrasts dramatically with the first travel Web test conducted by Consumer Reports Travel Letter in October 2000. In those tests, none of the four independent travel Web sites provided lower viable airline fares than the GDS (Apollo Galileo in those tests) more than 22% of the time.

The inclusion of Sabre in these tests was important because the DOT regulates Sabre and all other GDSs to ensure that airline fares and booking data are listed fairly and accurately. This unbiased approach is applied to hotel rates in the GDSs as well. However, due in large part to exclusive deals with airlines, hotels, and other travel companies, the unregulated independent travel Web sites have emerged as the masters at providing lowest rates.

An important point: Sabre's poor showing should not lead one to conclude that travel agencies cannot compete with independent travel Web sites. Concurrent with the growth of online travel sites, travel agencies have received many new booking tools as well. For example, Sabre and other GDSs now provide search engines for the Internet, and many travel agencies have secured exclusive deals with online travel Web sites as well as directly with travel suppliers. Savvy travel agents have found ways to find the lowest fares both on and off the Web.

• Overall, the branded hotel Web sites performed the worst of all, by providing the lowest rates only 10% of the time. However, there was one exception, which is noted below: The branded Web sites excelled at providing lowest rates when searching for a specific property.
  
  
• The level of competition is quite high among the integrated travel Web sites, as rates were very closely matched. Many of the Web sites provided rates that were only a few dollars lower per night than rates provided by their competitors. Expedia was particularly savvy at this; in several cases that site edged its nearest competitor by less than $1 per night.

back to top

CLOSEST RATES

This report's test results were augmented with the inclusion of closest rates, to provide a more accurate portrait of the Web sites' rate structures. These rates were within $5 per night of the lowest rate provided. These data proved enlightening, as exemplified by Figure 2. However, factoring in these rates did little to change the overall standings of the Web sites.

As noted, the order of providing the highest percentage of lowest rates was:

1. Travelocity
2. Orbitz
3. Expedia
4. Hotels.com
5. Sabre
6. Lodging.com
7. Branded hotel sites

By including the closest rates, the order of providing the highest percentage of lowest and closest rates was:

1. Travelocity
2. Orbitz
3. Hotels.com
4. Expedia
5. Sabre
6. Lodging.com
7. Branded hotel sites

The only significant change in the rankings took effect in the number three and four positions, which were swapped by Expedia and Hotels.com. In all cases, the percentages of lowest rates increased, sometimes slightly, when the closest rates were factored in as well.

Significant findings arose when the closest rates were anal