Online citations, reference lists, and bibliographies.
← Back to Search

Empirical Studies Assessing The Quality Of Health Information For Consumers On The World Wide Web: A Systematic Review.

G. Eysenbach, J. Powell, O. Kuss, Eun-Ryoung Sa
Published 2002 · Medicine

Save to my Library
Download PDF
Analyze on Scholarcy
CONTEXT The quality of consumer health information on the World Wide Web is an important issue for medicine, but to date no systematic and comprehensive synthesis of the methods and evidence has been performed. OBJECTIVES To establish a methodological framework on how quality on the Web is evaluated in practice, to determine the heterogeneity of the results and conclusions, and to compare the methodological rigor of these studies, to determine to what extent the conclusions depend on the methodology used, and to suggest future directions for research. DATA SOURCES We searched MEDLINE and PREMEDLINE (1966 through September 2001), Science Citation Index (1997 through September 2001), Social Sciences Citation Index (1997 through September 2001), Arts and Humanities Citation Index (1997 through September 2001), LISA (1969 through July 2001), CINAHL (1982 through July 2001), PsychINFO (1988 through September 2001), EMBASE (1988 through June 2001), and SIGLE (1980 through June 2001). We also conducted hand searches, general Internet searches, and a personal bibliographic database search. STUDY SELECTION We included published and unpublished empirical studies in any language in which investigators searched the Web systematically for specific health information, evaluated the quality of Web sites or pages, and reported quantitative results. We screened 7830 citations and retrieved 170 potentially eligible full articles. A total of 79 distinct studies met the inclusion criteria, evaluating 5941 health Web sites and 1329 Web pages, and reporting 408 evaluation results for 86 different quality criteria. DATA EXTRACTION Two reviewers independently extracted study characteristics, medical domains, search strategies used, methods and criteria of quality assessment, results (percentage of sites or pages rated as inadequate pertaining to a quality criterion), and quality and rigor of study methods and reporting. DATA SYNTHESIS Most frequently used quality criteria used include accuracy, completeness, readability, design, disclosures, and references provided. Fifty-five studies (70%) concluded that quality is a problem on the Web, 17 (22%) remained neutral, and 7 studies (9%) came to a positive conclusion. Positive studies scored significantly lower in search (P =.02) and evaluation (P =.04) methods. CONCLUSIONS Due to differences in study methods and rigor, quality criteria, study population, and topic chosen, study results and conclusions on health-related Web sites vary widely. Operational definitions of quality criteria are needed.
This paper references
Internet Availability of Prescription Pharmaceuticals to the Public
B. Bloom (1999)
Direct sale of sildenafil (Viagra) to consumers over the Internet.
K. Armstrong (1999)
Evaluating the Source and Content of Orthopaedic Information on the Internet: The Case of Carpal Tunnel Syndrome*
P. Beredjiklian (2000)
Quality of consumer drug information provided by four Web sites.
C. L. Hatfield (1999)
Prospecting and evaluation of the anatomy sites on the internet
E. Voiglio (1999)
Commentary: measuring quality and impact of the world wide web
J. Wyatt (1997)
Health information during a week of television.
F. A. Smith (1972)
Orthopdie im World Wide Web: Universitts-Homepages der G7-Staaten
I. Butzke (2000)
Irish health care Web sites: a review.
B. O'Mahony (1999)
Review Of Internet Health Information Quality Initiatives
A. Risk (2001)
The Health On the Net Code of Conduct for medical and health Websites.
C. Boyer (1998)
It Is Not Just a Matter of Ethics: A Survey of the Provision of Health Disclaimers, Caveats, and Other Health-related Alerts in Consumer Health Information on Eating Disorders on the Internet
F. Tu (2000)
Grade Level ranged from 6.2 to 14.5 years (median 10.5
Flesch Kinkaid (2000)
Published criteria for evaluating health related web sites: review
P. Kim (1999)
Information about age-related macular degeneration on the Internet.
Stone Tw (2001)
Evaluation of the quality of self-education mammography material available for patients on the Internet.
E. Tamm (2000)
What's on the Internet? Services for women affected by HIV and AIDS.
C. Mallory (1997)
Japanese sites of medical institutions (hospitals, clinics) not having a privacy policy 1145 1147 99
Comparison of search strategies and quality of medical information of the Internet: a study relating to ankle sprain.
D. Groot (2001)
Quality of care information for consumers on the Internet.
M. Oermann (2000)
Review and evaluation of anatomy sites on the Internet (updated 1999)
D. Frasca (2000)
A critical review of internet information about depression.
T. L. Lissman (2001)
Health information on the Internet: accessibility, quality, and readability in English and Spanish.
G. Berland (2001)
Quality of medical information about menorrhagia on the worldwide web
P. Latthe (2000)
Quality of web based information on treatment of depression: cross sectional survey
K. Griffiths (2000)
Quality of information on female sterilisation on the Internet.
P. Latthe (2000)
Accessibility of information on the web
S. Lawrence (1999)
Documents:Mean Flesch Reading Ease score was 57.0 Mean Fry Formula was 12.0 (12th grade, 0 months of schooling) Mean SMOG was 12.2 Mean Flesch-Kincaid grade level
D’Alessandro (2001)
An Ontology of Quality Initiatives and a Model for Decentralized, Collaborative Quality Management on the (Semantic) World Wide Web
G. Eysenbach (2001)
Applicability and quality of information for answering clinical questions on the Web.
W. Hersh (1998)
Drug Information Libraries on the Internet
M. Bogenschutz (2000)
Can Examination of WWW Usage Statistics and other Indirect Quality Indicators Help to Distinguish the Relative Quality of Medical websites?
A. Hernández-Borges (1999)
Sharing decisions with patients: is the information good enough?
A. Coulter (1999)
Comparison of information obtained from a Usenet newsgroup and from drug information centers.
J. A. Seaboldt (1997)
Readability levels of patient education material on the World Wide Web.
M. Graber (1999)
Usability engineering
J. Nielsen (1997)
Evaluating printed health information for consumers.
L. Dalton (1981)
Management advice for children with chronic fatigue syndrome: a systematic study of information from the internet
B. Wright (1999)
Is current information available useful for patients and their families?
H. Ellamushi (2001)
The internet as a pediatric neurosurgery information resource.
Maugans Ta (1998)
The readability of pediatric patient education materials on the World Wide Web.
D. D'Alessandro (2001)
Quality evaluation of orthodontic information on the World Wide Web.
Y. Jiang (2000)
Varicose veins on the internet.
G. Libertiny (2000)
Problems and risks of unsolicited e-mails in patient-physician encounters in travel medicine settings.
A. Sing (2001)
Anticoagulant Patient Information Material Is Written at High Readability Levels
C. A. Estrada (2000)
No disclaimer (“not a substitute for professional care
Pandolfini (2000)
Pharmacy benefit management sites not providing a " clear confidentiality statement
Kihlstrom (2001)
Quality of Breast Cancer Sites on the World Wide Web
L. Hoffman-Goetz (2000)
The Content and Accessibility of Sex Education Information on the Internet
M. Smith (2000)
DISCERN: an instrument for judging the quality of written consumer health information on treatment choices.
D. Charnock (1999)
Analysis of Internet Sites for Headache
S. Peroutka (2001)
Quality of websites: kitemarking the west wind
T. Delamothe (2000)
Index, subjective marks for writing style (no results reported)
Gunning Fog (1999)
The low availability of metadata elements for evaluating the quality of medical information on the World Wide Web
J. Shon (1999)
A critical appraisal of the use of Internet for calculating cardiovascular risk
P. Gillois (1999)
The Internet as a source of information on breast augmentation.
J. B. Gordon (2001)
Sites about hysterectomy where " confidentiality of data relating to individual patients was not respected " (eg, showing patients' photographs and full names) 7 26 26
Quality of information in mass media: a barrier to the dental health education of the public.
P. Frazier (1974)
Labeling and filtering of medical information on the Internet.
G. Eysenbach (1999)
Coverage and quality of oral cancer information in the popular press: 1987-98.
M. T. Canto (1998)
How reliable is science information on the web?
E. Allen (1999)
Rx medication information for the public and the WWW: quality issues.
P. Doupi (1999)
Information epidemics, economics, and immunity on the internet
E. Coiera (1998)
The Internet as a source for current patient information.
H. Mcclung (1998)
The measurement of observer agreement for categorical data.
J. Landis (1977)
Accuracy of telephone reference service in health sciences libraries.
Paskoff Bm (1991)
Web sites with misinformation about illicit drugs.
E. Boyer (2001)
Assessing the quality of newspaper medical advice columns for elderly readers.
F. Molnar (1999)
Quality of information on emergency contraception on the Internet.
M. Latthe (2000)
Validation of an index of the quality of review articles.
A. Oxman (1991)
Slimming on the Internet
J. Miles (2000)
Online Prescribing of Sildanefil (Viagra) on the World Wide Web
G. Eysenbach (1999)
Urology and the Internet: an evaluation of Internet use by urology patients and of information available on urological topics
G. Hellawell (2000)
A systematic review of patient information leaflets for hypertension
D. Fitzmaurice (2000)
A metadata vocabulary for self- and third-party labeling of health web-sites: Health Information Disclosure, Description and Evaluation Language (HIDDEL)
G. Eysenbach (2001)
Evaluation of pediatric surgery information on the Internet.
C. Corpron (2001)
An analysis of the readability and cultural sensitivity of information on the National Cancer Institute's Web site: CancerNet.
F. Wilson (2000)
Ethical issues in qualitative research on internet communities
G. Eysenbach (2001)
Consumer health informatics
G. Eysenbach (2000)
Accessibility and reliability of cutaneous laser surgery information on the World Wide Web.
J. Bykowski (2000)
Evaluation of cancer information on the Internet
J. S. Biermann (1999)
The quality of dietary information on the World Wide Web.
K. Davison (1997)
HMO sites not providing information on monthly cost to consumers on their Web site 17 21 81

This paper is referenced by
Evaluation of the Source and Quality of Information Regarding Cervical Disc Herniation on Websites
Y. C. Hong (2020)
Do adverts increase the probability of finding online cognitive behavioural therapy for depression? Cross-sectional study
R. Jones (2012)
Evaluating the Quality of Malaria-Related Health Information in the Nigerian Internet Context
Trout Kate (2014)
Interacción con los padres a través de internet de las sociedades pediátricas españolas y las integradas en la ALAPE
C. A. Vázquez (2004)
Evaluation of metabolic syndrome related health information on internet in Indian context.
A. Joshi (2013)
Cyberchondria: towards a better understanding of excessive health-related Internet use
V. Starcevic (2013)
Exploring West Virginia diabetes educators' perceptions of and experiences with integrating Internet technology in their practice
E. Quintana (2008)
Challenges Confronting Rural Dwellers in Accessing Health Information in Ghana: Shai Osudoku District in Perspective
P. P. Sokey (2017)
Avaliação de sites de saúde em questão: a Aids nos sites brasileiros de Organizações Não Governamentais (ONG) de Lésbicas, Gays, Bissexuais, Travestis e Transexuais (LGBT)
André Pereira Neto (2013)
Evaluation of the Quality of Otolaryngology Information on Persian Websites
A. Kouhi (2020)
Public Library Responses to a Consumer Health Inquiry in a Public Health Crisis: The SARS Experience in Ontario
Roma Harris (2005)
Consumer Health Informatics and Telehealth
P. Brennan (2006)
The initial development of the WebMedQual scale: Domain assessment of the construct of quality of health web sites
Mélanie Provost (2006)
What do patients with prostate or breast cancer want from an Internet site? A qualitative study of information needs.
L. Rozmovits (2004)
Ophthalmology on the internet: what do our patients find?
A. Kahana (2004)
Advice available on the Internet for people with coeliac disease: an evaluation of the quality of websites.
C. England (2004)
Content Analysis of Selected Health Information Websites - Final Report
T. Petch (2004)
Infodemic in the Global Coronavirus Crisis
Vahideh Zareh Gavgani (2020)
[What does the Internet teach the obstetric patient about labor analgesia?]
M. Weiss (2018)
Online health information seeking by adults hospitalized for acute coronary syndromes: Who looks for information, and who discusses it with healthcare providers?
M. Waring (2018)
Content analysis of web-based norovirus education materials targeting consumers who handle food: An assessment of alignment and readability
H. Evans (2016)
Medical Tourism Development in Croatia
Oliver Kesar Katarina Rimac (2011)
An Online Resource of Digital Stories About Cancer Genetics: Qualitative Study of Patient Preferences and Information Needs
R. Iredale (2011)
An investigation of consumer's choice of a healthcare facility when user-generated anecdotal information is integrated into healthcare public reports
Sruthy Agnisarman (2018)
Quality of anaesthesia-related information accessed via Internet searches.
S. Caron (2007)
The Internet and the oral healthcare professionals: potential and challenges of a new era.
N. Mattheos (2007)
An Assessment of the Currency of Free Science Information on the Web
C. Chen (2007)
Credibility: A multidisciplinary framework
Soo Young Rieh (2007)
User-Centered Evaluation of the Quality of Blogs
Sutthinan Chuenchom (2011)
Rodolfo Paolucci (2012)
Readability of Menopause Web Sites: A Cross-Sectional Study
Deborah H. Charbonneau (2012)
Ana and her web?: an investigation into Internet use in adults with an eating disorder
Faye Yuill (2013)
See more
Semantic Scholar Logo Some data provided by SemanticScholar