False Oracles:

Consumer Reaction to Learning the Truth About How Search Engines Work (Abstract)

Based on field research conducted by Context-Based Research Group, 120 W. Fayette Street, Suite 1100, Baltimore, MD 21201

Consumer Reports WebWatch
101 Truman Ave.
Yonkers, N.Y., USA 10703-1057

View Report   |   Download Report (PDF)   |   Download PDF Reader

Abstract

Many online consumers think of search engines as online tools that help them to quickly find Web sites most relevant to their keyword queries, particularly when they are unsure of where to surf next. This trust in search engines may make them vulnerable while online, as they are largely unaware such navigation sites often accept fees in exchange for giving advertiser Web pages prominent placement on their search results pages.

A year after the U.S. Federal Trade Commission determined there was a need for "clear and conspicuous disclosures of paid placement" on search engines, Consumer Reports WebWatch's research findings show consumers still cannot always tell what is "paid" search versus "pure" or "algorithmic" search on America's most popular search engines.

This study, which used an ethnographic approach, allowed anthropologist researchers to observe experienced Web searchers in their natural computer surroundings (home, work or school). Context-Based Research Group, a cultural anthropology research firm based in Baltimore, conducted the field research.

Four anthropologists based in the same number of U.S. metropolitan areas (Kansas City, Mo., Phoenix, Ariz., Providence, R.I., and Raleigh-Durham, N.C.) recruited 17 savvy Web consumers to participate in this unique study. Great care was taken to ensure there was diversity within the study group by gender, profession, household income and age. None of the participants knew about pay-for-placement on search engines prior to joining the study.

Once admitted to the study, each consumer performed two Web searches (one online shopping in nature, the other informational) on five pre-assigned search sites while observed and interviewed by the anthropologist. These searches, a total of 10 sessions per participant, were conducted on some of the most-trafficked search and navigation sites in the United States: About.com, AlltheWeb.com, AltaVista.com, AOL.com, Ask.com, Go.com, Google.com, InfoSpace.com, iWon.com, Kanoodle.com, LookSmart.com, Lycos.com, MSN.com, Overture.com and Yahoo.com. (A total of 163 completed searches were performed in which a participant selected at least one link from the delivered results pages.)

After performing these search sessions, the participant was "enlightened" by the anthropologist about how pay-for-placement works on each search site the participant used. The researcher led this discussion by pointing out the "disclosure" pages and "About Search" links posted on each site under evaluation and asked the participant to read through them. In addition, the anthropologists conducted pre- and post-enlightenment interviews with each participant, each averaging two hours in duration, during which information about a consumer's offline and online attitudes and behaviors was collected for later analysis.

Major Findings:

1. Most participants had little understanding of how search engines retrieve information from the Web or how they rank or prioritize links on a results page.
   
   
2. The majority of participants never clicked beyond the first page of search results as they had blind trust in search engines to present only the best or most accurate unbiased results on the first page. As a result, two-in-five links (or 41%) selected by our participants during the assigned search sessions were paid results.
   
   
3. Once enlightened about pay-for-placement, each participant expressed surprise about this search engine marketing practice. Some had negative, emotional reactions.
   
   
4. All participants said paid search links on search and navigation sites were often too difficult to recognize or find on many sites, and the disclosure information available was clearly written for the advertiser, not the consumer. Search engine sites that were perceived to be less transparent about these related disclosures lost credibility amongst this group of online consumers.