Assessing the Quality of Medical Web Sites

Quackery and misinformation on the Internet may become a matter of life and death. As a growing number of patients and health professionals consult medical Web resources, concerns have been raised about their quality and reliability. The free flow of information inevitably brings such difficulties.

Patients, health professionals, and health policymakers need reliable medical knowledge as a basis for their decisions. Patients need to know when to seek help, what relevant questions to ask, what treatment alternatives there are, and why they should follow caregivers' advice. Health professionals need to keep up to date with the scientific frontier in order to suggest the best available diagnostic and treatment methods. Policymakers need a solid basis for guideline development and organizational decisions.

All this information is available on the World Wide Web today. According to one estimate (Eysenbach et al. 1999), there were 100,000 medical Web sites in 1999. It has been noted that many provide misleading or fraudulent reporting students, general practitioners, specialist doctors, researchers), type of source and its trustworthiness, authors' degree of medical/scientific qualification, type of review (peer review, editorial control, readers feedback, none), source of funding (non-profit, minor commercial influence, major commercial influence), and degree of distinction between commercial and other contents.

Another initiative is the U.K. project OMNI (Organizing medical Networked Information) which aims to provide high quality biomedical resources that have been filtered, evaluated, described, indexed, and classified.

Investigators from OMNI have outlined the following important questions to ask when using an evaluation site or rating scale on the Web (http://omni.ac.uk/agec/iolim96.hrml#Ref5):

* Does the site clearly stare the criteria it uses for the selection and inclusion of resources?

* Is the evaluation site selective in the types of resources included?

* If resources are given a rating (e.g., stars or points), are the criteria for the ratings stated?

* Do resource descriptions/evaluations include the name of the evaluator?

* Is there any indication of the expertise/profession of the evaluator?

* Do resource descriptions/evaluations include the date on which the evaluation was written?

* Is there a mechanism for the regular updating of reviews?

* Are the URLs for resources included in the database regularly checked for changes, either automatically by link-checking software, or by a human?

* Are "dead" links identified and removed promptly?

* Does the site include a search facility?

A whole range of similar instruments rate medical Web sites. However, most of these have been criticized for being incomplete. For example, researchers at McMaster University, Ontario, affiliated with the Canadian Cochrane Centre, have questioned the use of such rating scales (Jadad et al. 1998). Their critique is based on a systematic evaluation of tools, showing that out of forty-seven rating instruments identified, only fourteen provided description of the criteria behind these ratings, and no more than five gave instructions for using the criteria.

Furthermore, it was observed that the organizations behind these instruments failed to inform users whether ratings are consistent when applied by different raters and whether the instruments actually measure what they claim to be measuring.

Some aspects of quality may be difficult to measure. Dr. Gunter Eysenbach of Erlangen University Hospital, Germany, as pointed out that not only the accuracy of the statements made on a Web site, but also the context in which they are presented, contribute to its quality (Eysenbach et al. 1998).

Apart from the question of how to best rate quality is the issue of who should perform the rating. Eysenbach and coworkers describe centralized and decentralized options. A decentralized system, as suggested by the authors, would mean that doctors, medical societies, and associations would critically appraise Internet information and rate the value and trustworthiness of information by putting electronic evaluative and descriptive "tags" on it. The authors speculate that Web browsers could be customized by the user to filter the information on the basis of these quality tags. The browser would automatically discard low-quality Web resources.

However, while such tools could probably facilitate searching, they would not alter the fact that Web users need to use their own critical thinking. As has been discussed elsewhere (Levi 1999), rating systems, quality filtering, and approval systems can only complement, and will never replace, critical content analysis on the Web users' part. At best, such tools may compare with the peer review system of medical journals, a system that has also been criticized for its shortcomings.

Pitfalls to Consider

In summary; Web users must always make an independent judgment of the quality of the information posted on Web sires. Many aspects should be considered regarding the clarity, relevance, and reliability of the information.