How Do People Evaluate a Web Site's Credibility?

Results from a Large Study (Full Report)

Leslie Marable
Consumer Reports WebWatch

Julianne Stanford and Ellen R. Tauber
Sliced Bread Design, LLC

Consumer Reports WebWatch
101 Truman Ave.
Yonkers, N.Y., USA 10703-1057

TABLE OF CONTENTS

Introduction
How to read this report

Background & Methods
Choosing 10 content categories
Choosing 10 Web sites in each category
How we recruited study participants
Who participated in this study
What people did in the study
Cleaning the data
How we coded and analyzed the comments
How we calculated the rankings

Results & Discussion
Part 1: Results & Discussion — Overall Analysis of Credibility Comments

1. Design Look — 46.1% Overall
2. Information Design/Structure — 28.5% Overall
3. Information Focus — 25.1% Overall
4. Company Motive — 15.5% Overall
5. Information Usefulness — 14.8% Overall
6. Information Accuracy — 14.3% Overall
7. Name Recognition and Reputation — 14.1% Overall
8. Advertising — 13.8% Overall
9. Information Bias — 11.6% Overall
10. Writing Tone — 9.0% Overall
11. Identity of Site Operator — 8.8% Overall
12. Site Functionality — 8.6% Overall
13. Customer Service — 6.4% Overall
14. Past Experience with Site — 4.6% Overall
15. Information Clarity — 3.7% Overall
16. Performance on Test by User — 3.6% Overall
17. Readability — 3.6% Overall
18. Affiliations — 3.4% Overall

Overall discussion on comments

Why is Design Look so prominent?
Areas that did not emerge in this research

Part 2: Results & Discussion — A Focus on Individual Web Site Categories
A word about the credibility-ranking results

E-Commerce Web Sites: Credibility Results
Entertainment Web Sites: Credibility Results
Finance Web Sites: Credibility Results
Health Web Sites: Credibility Results
News Web Sites: Credibility Results
Nonprofit Web Sites: Credibility Results
Opinion or Review Web Sites: Credibility Results
Search Engine Web Sites: Credibility Results
Sports Web Sites: Credibility Results
Travel Web Sites: Credibility Results

Concluding Discussion Focusing on Web Site Categories

Concluding discussion about Web site rankings
How Web site topic influences credibility assessments

How to View This Study & Other Web Credibility Research

Implications for Consumer Reports WebWatch
The Need for Education
The Need for Evaluation Assistance

References

Appendix A: Web Sites Selected for this Study

Appendix B: Coding Categories

Appendix C: Consumer Reports WebWatch Guidelines

Endnotes

Introduction

Can you trust what you find on the Web today? There's no simple answer to this question. (1)

With more than 50 percent of the U.S. population having Internet access (2), the World Wide Web has become an important channel for providing information and services. As the Web becomes a part of people's everyday lives — booking travel, finding health information, buying products — there is a growing need to help people figure out whether a Web site is credible or not (i): Can I trust the information on this site? Can I trust in the services this site describes?

To take this line of thinking to an extreme, imagine a world in which people could not reliably assess the credibility of what they find online. What would be the ultimate outcome? In our view, people would eventually stop using the Web for anything that really matters. In an extreme situation, the Web would become a channel for trivialities — for content and services that have little impact on people's lives. This would be a significant loss for institutions that benefit from being online. But we believe the loss would prove to be even greater for individuals. So far in its short lifetime, the Web has provided people with increased options for living rewarding and productive lives.

One of our goals is to help see that the viability of the Web continues. An even more ambitious goal — in fact, the essential mission of Consumer Reports WebWatch — is to help make the Web a safe and reliable channel for people who seek information and services. The study reported in this paper is a step toward achieving this larger goal. A collaboration between Consumer Reports WebWatch and Stanford University's Persuasive Technology Lab (with key contributions from Sliced Bread Design, LLC), our research investigates how people evaluate the credibility of Web sites today. Our work at this point is descriptive in nature (focusing on what people do) rather than prescriptive (what people should do). With a basic understanding of how people tend to assess the credibility of Web sites, Consumer Reports WebWatch is now in a better position to create solutions that help people evaluate the credibility of online information and services.

We believe that the future health of the Web hinges on issues relating to credibility. For this reason, we have chosen to invest time and money to understand this domain, with the realization that our reach is likely to exceed our grasp. To our knowledge this is the largest study to date on the credibility of Web sites. While our study confirms some earlier assumptions and research findings, the data from this project take our understanding of Web site credibility to a new level, offering richer insight into credibility assessments. The study also suggests new areas for Web credibility guidelines, the need for consumer education, and potential areas for future investigations.

back to top

HOW TO READ THIS REPORT

We've written the report to be a hybrid between an industry research report and an academic paper (but with a friendlier style — we hope). To be clear, this report is not a news story — you won't find the most important elements in the first few paragraphs.

The first section of our report contains an extended description of the study background, rationale, and research method (Background & Methods). For some readers this section will be important and interesting. For other people, getting through the methods section will be a tedious chore. Although some people may choose to skip or skim the methods section, understanding the strengths and weaknesses of our research method will help readers make more sense — and perhaps better personal interpretations — of our data.

Following the Background & Methods section, we present our study results and briefly discuss those results as we present them. The Results & Discussion section is one that many readers will care about most. To keep this report to a reasonable length, we do not discuss or interpret all the data presented in this report. We hope that readers will examine the data and reach some of their own conclusions, ones that we have not specifically outlined in this report.

After the Results & Discussion section, this paper presents a theory that explains how our study findings fit with previous research on Web credibility. This theory — called "Prominence-Interpretation Theory" — is not difficult to understand. For some readers these three pages about theory may be the most enlightening part of our report, particularly for those who seek to understand how people assess credibility of Web sites. Some readers may want to jump ahead to the theory part of our report first ("How to View This Study & Other Web Credibility Research") and then return back to our Background & Methods section and continue reading. This nonlinear approach will give readers more insight into our research throughout the report. (After some debate, we decided to put the theory section toward the end of this document because we didn't want to burden all readers with theory in the first few pages of our report. We want you to keep reading!)

The final section of the paper interprets the study results in light of the current Consumer Reports WebWatch Web credibility guidelines. This section will be most important for those interested in the Consumer Reports WebWatch mission: to make the Web a safe and reliable channel for people who seek information and services.

The final pages of this document contain references, a set of appendices, and a collection of endnotes.

back to top

Footnotes
(i) In this paper we adhere to the definition of credibility outlined by Fogg and Tseng (1999), with the following discussion drawing largely from this work. In their view, credibility can be defined as believability. Credible information is believable information. It's important to note that credibility is a perceived quality. It is not a property of a Web site, such as how many words the site contains or how many links are on the page. Instead, when one discusses credibility, it is always from the perspective of the observer's perception. It's also important to understand that people perceive credibility by evaluating multiple dimensions simultaneously. In general, these dimensions can be categorized into two key components: trustworthiness and expertise. The trustworthiness component refers to the goodness or morality of the source and can be described with terms such as well intentioned, truthful, or unbiased. The expertise component refers to perceived knowledge of the source and can be described with terms such as knowledgeable, reputable, or competent. People combine assessments of both trustworthiness and expertise to arrive at a final credibility perception.

Background & Methods

After three years of researching Web credibility, the Stanford Persuasive Technology Lab began developing a new online method for comparing the credibility of live Web sites. To do this we performed iterative design and pilot testing, including more than 200 people over the course of six months. The results were encouraging. Not only did the data produce a relative ranking of the Web sites that seemed to have high external validity, but also the participants contributed brief evaluations about the Web sites that struck us as frank and insightful. We concluded from these two pilot studies that (1) our online research method of paired comparison would work on a larger scale, for a wider audience; (2) the relative rankings of Web sites that resulted from the research was interesting and at times provocative, but these rankings had little practical or theoretical value; and (3) the most valuable information from the pilot studies were the comments people made about the sites they evaluated.

After refining the research method, the Stanford team began talking to people at Consumer Reports WebWatch, a nonprofit project of Consumers Union, publisher of Consumer Reports. Consumer Reports WebWatch commissioned the study covered in this report. This collaboration made sense, since the goal of Consumer Reports WebWatch is to investigate, inform, and improve the credibility of information published on the World Wide Web (ii). (Note: Consumer Reports WebWatch is supported by grants from The Pew Charitable Trusts, the John S. and James L. Knight Foundation, and the Open Society Institute.)

back to top

CHOOSING 10 CONTENT CATEGORIES

Choosing the categories to study was a collaborative task, including researchers from the Stanford Persuasive Technology Lab, Consumer Reports WebWatch, and Sliced Bread Design, LLC. We selected categories that (1) are prominent on the Web today, (2) had relevance to Web credibility, and (3) had a consumer focus (created for ordinary Web users, as opposed to specialists).

The categories we selected were:

• E-Commerce
  
  
• Entertainment
  
  
• Finance
  
  
• Health
  
  
• News
  
  
• Nonprofit
  
  
• Opinion or Review
  
  
• Search Engines
  
  
• Sports
  
  
• Travel

back to top

CHOOSING 10 WEB SITES IN EACH CATEGORY

Choosing 10 Web sites to test in each category was not an easy task. To explore a range of sites and, later, to narrow the field, we sought input from the Consumer Reports WebWatch advisers and researchers at Stanford, among others. We focused on sites that were created for ordinary Web users, not experts (for example, health sites for consumers rather than for medical professionals). We sought to include a wide range of Web sites in each category, and in making our selections we paid attention to issues such as name recognition, popularity, visual design, best practices, and so on. In the selection process, we tried to avoid selecting two sites that were quite similar (such as two online bookstores) unless there was a compelling difference between the two, such as Amazon (purely an online brand) and Barnes and Noble (a brick-and-mortar brand in addition to an online one).

We knew the choice of Web sites would be important. The final rankings of the sites in each category would be a direct result of the sites we chose. For example, if we chose only top-quality sites within one category, then even a very good site could end up on the bottom end of the final rankings. Even more important, the types of comments we would collect from participants during the study would hinge on the sites we chose. If sites didn't offer enough variety, the comments from participants would also lack variety.

After almost two of month of deliberations, we finally arrived at our final 100 Web sites for this study. A complete list of all the sites within 10 categories is included in Appendix A.

With the 100 sites selected and programmed into the Web-based research system, we were ready to begin recruiting participants.

back to top

HOW WE RECRUITED STUDY PARTICIPANTS

We began recruiting participants in May 2002 by contacting nonprofit groups, such as the Children's Brain Tumor Foundation, and offering a $5 donation for each supporter who completed the study. The nonprofit groups then let their supporters know through e-mail or a link on their Web pages. We drew on diverse charity groups to get a broad range of participants. Our goal was to work with charities that had supporters from all over the U.S. and whose supporters would represent a diversity of ages, income levels, political leanings, and more (the screen shot in Figure 5 shows the nonprofit groups that participated). From June 15th to August 15th, we collaborated with 10 nonprofit groups, leading to over 2,600 people participating in the study. Although the charity recruiting method does not provide a representative sample of Web users (iii), this recruitment method is entirely adequate for the purposes of this study, which are fundamentally exploratory. In our view, this method is superior to other tractable alternatives (e.g., offering money directly to people, setting up a contest, spamming.). We also believe that people who participate to help a charity group will do a better job than people who are doing the study for other motives, such as personal interest in winning a contest.

back to top

WHO PARTICIPATED IN THIS STUDY

A total of 2,684 people completed the study. We did not require people to leave demographic information, yet 60.1% of the participants did so. From the information gathered, we constructed a demographic picture of the people who participated:

• Female: 58.1%; Male: 41.9%
  
  
• Average age: 39.9
  
  
• Average use of Web: 19.6 hours/week

The respondents who contributed demographic information came from 47 states (no one claimed to be from Idaho, North Dakota, or Wyoming) and the District of Columbia. The vast majority of participants live in the U.S., although people from 29 other countries also participated.

back to top

WHAT PEOPLE DID IN THE STUDY

We provide a more detailed description of what people did in this study below. The following seven steps summarize the participants' experience:

After being contacted by a nonprofit group or a friend, people interested in helping with our study:

1. Logged on to www.mostcredible.org
2. Were welcomed and introduced to the study
3. Were randomly assigned to one of 10 Web site content categories (such as health or news)
4. Were given two live Web sites to evaluate for credibility
5. Reviewed the two sites assigned to them
6. Ranked the two sites according to credibility
7. Left comments about each site's credibility

A more detailed description of the participants' experience follows:

After being contacted by nonprofit group or a friend, people interested in helping with the study would log on to www.mostcredible.org and begin. The first page welcomed people to the study, outlined the three steps, and reminded them that they could later select a nonprofit group to receive a donation. (iv)

After participants read about the study, they would select "Click here to begin" to go to the next page.

At this point, the Web-based research engine would randomly assign the participant to one of 10 content categories (health, travel, finance, etc.) and randomly select two sites for evaluation from that category.

The Web page listed the category (such as "Finance Web Sites") and listed two Web sites by name and URL. The text on the page asked participants to visit the two sites, return and rank which site was the more credible of the two, and share their comments. Participants could click on the site name or URL to have a new browser window open containing a live version of that Web site.

After participants examined the two Web sites, they returned to the main page to rank which of the two sites site they found more credible. Next, they shared comments about their decision. The system required people to put in a ranking but did not require them to leave comments; however, most people left comments.

After entering comments about the sites, participants submitted this information. They were then taken to a page that asked for demographic information (again, not required, but most people cooperated) and asked which nonprofit should receive a $5 donation.

Participants concluded their role in the study by submitting the page containing demographics and their nonprofit selection. They then saw a "thank you" screen that provided contact information about the study.

back to top

Footnotes
(ii) B.J. Fogg is an adviser to the Consumer Reports WebWatch project. He receives no compensation for advising CWW, and he received no compensation for being involved with this study. This project was part of his academic work at Stanford University.

(iii) In an ideal world, this type of large-scale research would draw on a representative sample of Web users. However, in the past we've found that obtaining a truly representative sample is not possible — or at least not possible without an enormous budget. As an alternative, in this study we used a recruiting process that has worked well for us before: charity collaborations. In our view, this recruitment method is better than other methods often used, such as spamming, offering discounts at online retailers, or entering people in sweepstakes.

(iv) Nowhere in the study did we mention the participation of Consumer Reports WebWatch or its affiliates. We suspected that the influential reputation of Consumers Union would increase the likelihood that companies or people would log on to the study many times and skew the results.

back to top

CLEANING THE DATA

We asked participants to complete the study only once. We believe that the vast majority of people honored this request. However, there was no foolproof way to assure that someone would not repeat the study in order to earn more money for a nonprofit group or, for whatever motive, to manipulate the study findings. To take steps toward ensuring the integrity of the data, we designed our Web-based system to record the IP addresses of each participant. Finding duplicate IP addresses in the data file would be a reasonable — though not certain — indicator that one person had participated in the study multiple times. There were a number of cases where one IP address is the source for multiple evaluations (in one case, as many as 20). We examined each of these cases, realizing that two people in one household could have completed the study using the same IP address. We also looked for unusual patterns in the data to find evidence of tampering: a Web site that had lots of evaluations in a short period of time, a single category that was evaluated significantly more than others, and so on. After inspecting these cases, we removed data that showed compelling evidence of someone completing multiple sessions or tampering with our randomization scheme.

back to top

HOW WE CODED AND ANALYZED THE COMMENTS

The most valuable data from this research are the comments people made about the Web sites they evaluated. During the course of the study, the database stored these comments, along with the ID number for the site under evaluation, the participants' demographics, and other particulars. These comments and associated information became the focus of analysis once data collection was complete.

The first step in analyzing the data was to code the comments according to content. The study generated 2,440 comments about Web credibility. Some of these comments were brief and others were lengthy. Some were trivial and others were insightful. We believe that this collection of comments about Web credibility is the largest to date and offers many opportunities for analysis. We present one type of analysis here; other analyses and interpretations of the data are possible.

Two independent coders went through the participant comments and assigned codes to mark what was said in the comment. A third coder then went through the data to resolve discrepancies. Each comment could receive more than one code. For example, the comment below would be coded in two categories: design look and information bias.

• "This Web site looks more professional than the other, but I believe it is also more biased."

Described more in the Appendix B, the categories for coding came from two sources-the current version of the Consumer Reports WebWatch guidelines (retrieved from http://www.consumerwebwatch.org/consumer-reports-webwatch-guidelines.cfm on August 15, 2002), and from the emerging themes in the consumer comments themselves (visual design, previous experience with the site, etc.).

After coding each comment, we tallied the frequency for each code category. In other words, we calculated how often a specific issue was mentioned. For example, we found that information bias was mentioned in 283 of the 2,440 comments — 11.6% of the time. This frequency score gave an indication of what criteria people used — or said they used — to make their credibility evaluations of the sites they saw. (v) The Results & Discussion section of this report has more information about how we analyzed the comments and what we found.

back to top

HOW WE CALCULATED THE RANKINGS

The ranking system is straightforward. When a user evaluated a site as more credible than the other site listed on the page, the more credible site won a point (+1 went into the database for that site) and the less credible site lost a point (-1 went into the database). Over the course of the study, each site was evaluated many times, gaining and losing points along the way. At the end of the study, each site received a final score, which was the average (mean) of all the scores it had received (the total number of points divided by the total number of times the site was ranked). So if a site were to win all of the credibility pairings, it would have a score of +1.0. If it won half of the time, the score would be 0.0. And if it won 75% of the time, the score would be +0.50.

The 10 sites in each category were then ranked according to their mean scores, highest to lowest. This ranking gives a general idea about which sites people in this study found most and least credible. Small differences in means between two sites in the same category are not practically significant.

We should note here that study participants did not know our plans for this research. They did not know we were studying 10 categories of Web sites and that we were ranking sites within each category. Participants simply evaluated two Web sites. Had participants known we were compiling data to create a credibility ranking of Web sites, we suspect that some people may have tried to manipulate our results.

back to top

This section presents the results of our study and discusses the findings we judge as the most notable. For the sake of clarity, we present the Results & Discussion section in two parts, as shown below:

Part 1: Results & Discussion — An Overall Analysis of Credibility Comments

Part 2: Results & Discussion — A Focus on Individual Web Site Categories

In Part 1 we share our analysis of the 2,440 comments that participants made about the credibility of the Web sites. Our analysis found 18 types of comments relating to credibility with incidence over 3 percent. We present the data and discuss each type of comment, from most to least common.

In Part 2 we shift our focus from the types of comments people made to the types of Web sites in the study, presenting and discussing the results for each Web site category one by one in the following order:

• E-Commerce
  
  
• Entertainment
  
  
• Finance
  
  
• Health
  
  
• News
  
  
• Nonprofit
  
  
• Opinion or Review
  
  
• Search Engines
  
  
• Sports
  
  
• Travel

back to top

Part 1: Results & Discussion —
Overall Analysis of Credibility Comments

As explained in the Methods section, participants were given the opportunity to leave brief comments about the credibility of the two sites assigned to them. Discussed in more detail earlier, we coded 2,440 comments according to categories described in the Appendix B. The following pages share the results of our content analysis, starting with the overview in Table 1.

Table 1 presents an overall picture of our content analysis for comments about all 100 sites in this study. This table shows 18 types of comments, from "design look" to "affiliations." The percentages in the table represent how often a comment on that topic appeared in the entire set of comments. For example, participants in our study mentioned something about the "design look" of the site in 46.1% of the 2,440 comments.

Table 1: How often participants commented on various issues when evaluating the credibility of Web sites.

While the percentages in Table 1 show the types and frequency of comments about Web credibility, the table alone does not give rich information. Below we provide explanations and examples for each type of comment, along with the incidence for each category of Web site.

back to top

1. DESIGN LOOK — 46.1% OVERALL

When evaluating the credibility of a Web site, participants commented on the design look of the site more often than any other Web site feature, with 46.1% of the comments addressing the design look in some way. When coding for comments on design look, researchers included comments on many elements of the visual design, including layout, typography, white space, images, color schemes, and so on. The comments could be either positive or negative. Some of the comments coded in this category are as follows:

• This site is more credible. I find it to be much more professional looking. — M, 38, Washington
  
  
• More pleasing graphics, higher-quality look and feel — F, 52, Tennessee
  
  
• Just looks more credible. — M, 24, New Jersey
  
  
• Actually, despite the subject of the Web site, it looks very credible. This may be due to the subdued color scheme and the font used on the left-hand side of the page. — F, 29, California
  
  
• I know this is superficial, but the first thing that struck me is the color difference. The … site is a soothing green (sort of like money) while the [other] site is a jarring purple. — M, 56, Virginia
  
  
• The design is sloppy and looks like some adolescent boys in a garage threw this together. — F, 48, California
  
  
• Not very professional looking. Don't like the cheesy graphics. — F, 33, Washington
  
  
• Looks childish and like it was put together in 5 minutes. — F, 25, Maryland

Design Look comments, by site category
As shown in Figure 6, nearly half of all site evaluators (46.1%) used visual cues, such as the site's overall design or look, to assess a site's credibility. This happened slightly more frequently with finance (54.6%), search engines (52.6%) and travel (50.5%) categories, and less frequently with the health (41.8%) and news (39.6%) categories.

Figure 6: Percentage of comments relating to design look, by category

Our results about the connection between Design Look and perceived credibility suggests that creating Web sites with quality information alone is not enough to win credibility in users' minds. In most cases Web site designers need also to focus on the impression that the visual design will make, creating a site that achieves what many of our participants described as "a polished, professional look." But the connection between visual design and credibility may not be so simple. Slick-looking Web sites frequently received negative comments. Participants seemed to make judgments about the people behind the site on the basis of the Design Look. Many comments were indicative of this attitude: "It looks like it's designed by a marketing team, and not by people who want to get you the information that you need."

Based on the comments we've read from this study, we speculate that once a site is above a user's personal threshold to qualifying as having a "professional look," then other aspects of the Web site come into the credibility equation. In other words, the visual design may be the first test of a site's credibility. If it fails on this criterion, Web users are likely to abandon the site and seek other sources of information and services.

We discuss the topic of Design Look in more depth at the end of this section.

back to top

2. INFORMATION DESIGN/STRUCTURE — 28.5% OVERALL

After Design Look, the next category that people commented on in assessing credibility was the structure of the site's information, being mentioned in 28.5% of the total comments. The participant comments discussed how well or poorly the information fit together, as well as how hard it was to navigate the site to find things of interest. While information structure is often associated with usability, the comments here show how information structure has implications for credibility. Sites that were easy to navigate were seen as being more credible. Some sample comments are below:

• This site is very well organized, which lends to more credibility. — M, 33, Illinois
  
  
• This one is more credible because it is more organized. — F, 57, Maryland
  
  
• Horrible site, information badly presented. They try to put everything on the front page, instead of having multiple layers of navigation. This to me suggests that they developed this thing on a whim. — M, 42, Canada

Information Design/Structure comments, by site category
As shown in Figure 7, more than a quarter (28.5%) of all site evaluations made reference to issues of site information design — that is, the site's overall information structure or organization. This occurred much more frequently in the search engines (42.6%), finance (33.0%), and travel (31.8%) categories, and less frequently in the opinion or review (24.3%), sports (22.3%), and nonprofit (18.2%) categories.

Figure 7: Percentage of comments relating to information design by category.

What we find notable about Information Structure
That information design affects credibility should come as no surprise. A well-organized site is the antithesis to a site that purposely confuses and misleads a user toward advertisements and other promotions.

Online usability research has made it clear that information structure is critical for task success on the Web, and ease of use has been shown to contribute to credibility perceptions in previous research (Fogg et al., 2000; Fogg et al., 2001; Fogg et al., 2002). The reason behind this consistent finding isn't completely clear. One might speculate that by providing a clear information structure, a Web design team demonstrates expertise to the users. Users may then assume this expertise extends to the quality of information on the site.

back to top

3. INFORMATION FOCUS — 25.1% OVERALL

In 25.1% of the comments about credibility, people in this study talked about the focus of information on the site. The comments varied in content. At times a focused site was seen as more credible, other times a narrow focus hurt credibility. What's clear is that many people in this study relied on information focus to determine whether a site was credible or not. Sample comments are below:

• Credible because of the breadth of information available. — M, 35, California
  
  
• I find this site trustworthy because it offers a simple message to a very targeted community. — F, 34, Massachusetts
  
  
• This Web site is filled with too much crap. I feel as though part of the reason it seems less credible is the fact that the crap they fill it with is taking attention away from their own Web site. — F, 23, Illinois
  
  
• Broad categories, but shallow reviews and comparisons. — M, 35, California
  
  
• This site seems focused on body image. They have articles about feeling good naked, the perfect swimsuit for every body type, and toning exercises. Not a lot of solid health information. — F, 22 Minnesota

Information Focus comments, by site category
As Figure 8 shows, about one in four site evaluations (25.1%) made comments about issues of information focus, that is, the perceived scope or focus of the site. This generally occurred far more frequently in the information-heavy categories (health: 33.0%; news: 31.9%; sports: 30.8%), and occurred less frequently in the more transactional-heavy categories (e-commerce: 24.7%; search engines: 24.5%; finance: 18.9%).

Figure 8: Percentage of comments relating to information focus, by category.

The other notable finding about information focus is how much this issue varied depending on the type of site, with information focus being most prominent when evaluating health and news sites and least prominent when evaluating nonprofit sites. The data suggest that people have clearer expectations about the focus of certain types of Web sites. We speculate that the expectations about site focus are higher for the types of information-rich sites people know best (e.g., health, news, sports).

back to top

4. COMPANY MOTIVE — 15.5% OVERALL

We found that 15.5% of the comments in this study addressed the perceived underlying motive of the site or the institution sponsoring the site. These comments often referred to how Web sites lost credibility when the only purpose of a site seemed to be selling things or getting money from users. In other cases, Web sites won credibility by conveying motives that people found to be admirable. Sample comments are below:

• The fact that this site has a global conscience impressed me and made me feel it was more credible. — F, 40, New Jersey
  
  
• This site looks like its goal is to help you find what you are looking for. — F, 55, California
  
  
• I would trust this site because it's run by a religious denomination whose aim is socially responsible investing. — F, 54, New York
  
  
• Seems too "commercial" and therefore less objective. — M, 52, Texas
  
  
• This site says to me "Give us your money and get out." — F, 29, British Columbia
  
  
• Doesn't seem credible when they give a product a good review and give you a link to order it too. — F, 38, Texas

Company Motive comments, by site category
As shown in Figure 9, a sizable percentage (15.5%) of all site evaluations commented on the perceived motive of a company or organization behind the site. This occurred with far greater frequency in opinion or review (22.1%), finance (21.0%), nonprofit (20.2%) categories, and with less frequency in the search engines (14.2%), travel (12.8%), and news (5.9%) categories.

Figure 9: Percentage of comments relating to underlying company motive by category

What we find notable about Company Motive
Generally, people commented positively on company motive if they felt the site provided useful information or services without a commercial purpose. Ads or endorsements made the picture more complicated. A good number of comments made it clear that at least some people recognized a link between endorsements and company profit. Positive reviews of a product or firm, for example, were met with skepticism if the organization behind the site might have something to gain from such good reviews.

back to top

5. INFORMATION USEFULNESS — 14.8% OVERALL

When evaluating Web site credibility, people in this study commented on the usefulness of the site's information 14.8% of the time. As one might expect, useful information led people to see the Web site as more credible. Below are sample comments we found in this category:

• This Web site provided useful and interesting knowledge about events in sports. — F, 30, New Jersey
  
  
• Liked it because it is something that would be useful to me and other family members. — F, 18, Illinois
  
  
• I searched for a particular scientific term, and this search engine came up with more useful Web sites than the other one. — F, 40, Washington

Information Usefulness comments, by site category
As Figure 10 illustrates, less than one-in-six site evaluations (14.8%) commented on information usefulness, that is, how useful a user perceived of the site's information. This happened with far greater frequency in the niche-news categories of health (20.5%), entertainment (19.5%), and opinion or review (17.1%) categories, and with slightly less frequency in the travel (11.5%), sports (11.4%), and nonprofit (11.1%) categories.

Figure 10: Percentage of comments relating to information usefulness, by category.

Category	Percent of Site Evaluations
Health	20.5%
Entertainment	19.5%
Opinion or Review	17.1%
E-Commerce	16.3%
Search Engines	15.6%
All Sites	14.