Online citations, reference lists, and bibliographies.
← Back to Search

Empirical Studies Assessing The Quality Of Health Information For Consumers On The World Wide Web: A Systematic Review.

G. Eysenbach, J. Powell, O. Kuss, Eun-Ryoung Sa
Published 2002 · Medicine

Save to my Library
Download PDF
Analyze on Scholarcy
Share
CONTEXT The quality of consumer health information on the World Wide Web is an important issue for medicine, but to date no systematic and comprehensive synthesis of the methods and evidence has been performed. OBJECTIVES To establish a methodological framework on how quality on the Web is evaluated in practice, to determine the heterogeneity of the results and conclusions, and to compare the methodological rigor of these studies, to determine to what extent the conclusions depend on the methodology used, and to suggest future directions for research. DATA SOURCES We searched MEDLINE and PREMEDLINE (1966 through September 2001), Science Citation Index (1997 through September 2001), Social Sciences Citation Index (1997 through September 2001), Arts and Humanities Citation Index (1997 through September 2001), LISA (1969 through July 2001), CINAHL (1982 through July 2001), PsychINFO (1988 through September 2001), EMBASE (1988 through June 2001), and SIGLE (1980 through June 2001). We also conducted hand searches, general Internet searches, and a personal bibliographic database search. STUDY SELECTION We included published and unpublished empirical studies in any language in which investigators searched the Web systematically for specific health information, evaluated the quality of Web sites or pages, and reported quantitative results. We screened 7830 citations and retrieved 170 potentially eligible full articles. A total of 79 distinct studies met the inclusion criteria, evaluating 5941 health Web sites and 1329 Web pages, and reporting 408 evaluation results for 86 different quality criteria. DATA EXTRACTION Two reviewers independently extracted study characteristics, medical domains, search strategies used, methods and criteria of quality assessment, results (percentage of sites or pages rated as inadequate pertaining to a quality criterion), and quality and rigor of study methods and reporting. DATA SYNTHESIS Most frequently used quality criteria used include accuracy, completeness, readability, design, disclosures, and references provided. Fifty-five studies (70%) concluded that quality is a problem on the Web, 17 (22%) remained neutral, and 7 studies (9%) came to a positive conclusion. Positive studies scored significantly lower in search (P =.02) and evaluation (P =.04) methods. CONCLUSIONS Due to differences in study methods and rigor, quality criteria, study population, and topic chosen, study results and conclusions on health-related Web sites vary widely. Operational definitions of quality criteria are needed.
This paper references
10.7326/0003-4819-131-11-199912070-00005
Internet Availability of Prescription Pharmaceuticals to the Public
B. Bloom (1999)
10.1056/NEJM199910283411810
Direct sale of sildenafil (Viagra) to consumers over the Internet.
K. Armstrong (1999)
10.2106/00004623-200011000-00004
Evaluating the Source and Content of Orthopaedic Information on the Internet: The Case of Carpal Tunnel Syndrome*
P. Beredjiklian (2000)
10.1093/AJHP/56.22.2308
Quality of consumer drug information provided by four Web sites.
C. L. Hatfield (1999)
10.1007/s00276-999-0065-2
Prospecting and evaluation of the anatomy sites on the internet
E. Voiglio (1999)
10.1136/bmj.314.7098.1879
Commentary: measuring quality and impact of the world wide web
J. Wyatt (1997)
10.1056/NEJM197203092861005
Health information during a week of television.
F. A. Smith (1972)
10.1055/S-2000-10170
Orthopdie im World Wide Web: Universitts-Homepages der G7-Staaten
I. Butzke (2000)
Irish health care Web sites: a review.
B. O'Mahony (1999)
10.2196/JMIR.3.4.E28
Review Of Internet Health Information Quality Initiatives
A. Risk (2001)
10.1016/S0010-4825(98)00037-7
The Health On the Net Code of Conduct for medical and health Websites.
C. Boyer (1998)
10.1006/IILR.2000.0135
It Is Not Just a Matter of Ethics: A Survey of the Provision of Health Disclaimers, Caveats, and Other Health-related Alerts in Consumer Health Information on Eating Disorders on the Internet
F. Tu (2000)
Grade Level ranged from 6.2 to 14.5 years (median 10.5
Flesch Kinkaid (2000)
10.1136/bmj.318.7184.647
Published criteria for evaluating health related web sites: review
P. Kim (1999)
10.1097/00007611-200194010-00004
Information about age-related macular degeneration on the Internet.
Stone Tw (2001)
10.1016/S1076-6332(00)80113-0
Evaluation of the quality of self-education mammography material available for patients on the Internet.
E. Tamm (2000)
10.1080/07399339709516283
What's on the Internet? Services for women affected by HIV and AIDS.
C. Mallory (1997)
Japanese sites of medical institutions (hospitals, clinics) not having a privacy policy 1145 1147 99
(2000)
10.1016/S0020-1383(01)00019-5
Comparison of search strategies and quality of medical information of the Internet: a study relating to ankle sprain.
D. Groot (2001)
10.1097/00001786-200007000-00008
Quality of care information for consumers on the Internet.
M. Oermann (2000)
10.1007/s00276-000-0107-2
Review and evaluation of anatomy sites on the Internet (updated 1999)
D. Frasca (2000)
10.1176/APPI.PS.52.8.1046
A critical review of internet information about depression.
T. L. Lissman (2001)
10.1001/JAMA.285.20.2612
Health information on the Internet: accessibility, quality, and readability in English and Spanish.
G. Berland (2001)
10.1111/j.1471-0528.2000.tb11577.x
Quality of medical information about menorrhagia on the worldwide web
P. Latthe (2000)
10.1136/bmj.321.7275.1511
Quality of web based information on treatment of depression: cross sectional survey
K. Griffiths (2000)
10.1080/01443610062968
Quality of information on female sterilisation on the Internet.
P. Latthe (2000)
10.1038/21987
Accessibility of information on the web
S. Lawrence (1999)
Documents:Mean Flesch Reading Ease score was 57.0 Mean Fry Formula was 12.0 (12th grade, 0 months of schooling) Mean SMOG was 12.2 Mean Flesch-Kincaid grade level
D’Alessandro (2001)
10.2196/JMIR.3.4.E34
An Ontology of Quality Initiatives and a Model for Decentralized, Collaborative Quality Management on the (Semantic) World Wide Web
G. Eysenbach (2001)
10.1001/JAMA.280.15.1307
Applicability and quality of information for answering clinical questions on the Web.
W. Hersh (1998)
10.1080/02791072.2000.10400447
Drug Information Libraries on the Internet
M. Bogenschutz (2000)
10.2196/JMIR.1.1.E1
Can Examination of WWW Usage Statistics and other Indirect Quality Indicators Help to Distinguish the Relative Quality of Medical websites?
A. Hernández-Borges (1999)
10.1136/BMJ.318.7179.318
Sharing decisions with patients: is the information good enough?
A. Coulter (1999)
10.1093/AJHP/54.15.1732
Comparison of information obtained from a Usenet newsgroup and from drug information centers.
J. A. Seaboldt (1997)
Readability levels of patient education material on the World Wide Web.
M. Graber (1999)
10.1007/1-4020-4653-7_4
Usability engineering
J. Nielsen (1997)
Evaluating printed health information for consumers.
L. Dalton (1981)
10.1017/S0790966700005218
Management advice for children with chronic fatigue syndrome: a systematic study of information from the internet
B. Wright (1999)
Is current information available useful for patients and their families?
H. Ellamushi (2001)
10.1159/000028647
The internet as a pediatric neurosurgery information resource.
Maugans Ta (1998)
10.1001/ARCHPEDI.155.7.807
The readability of pediatric patient education materials on the World Wide Web.
D. D'Alessandro (2001)
10.1067/MOD.2000.104492
Quality evaluation of orthodontic information on the World Wide Web.
Y. Jiang (2000)
10.1053/EJVS.2000.1179
Varicose veins on the internet.
G. Libertiny (2000)
10.2310/7060.2001.24423
Problems and risks of unsolicited e-mails in patient-physician encounters in travel medicine settings.
A. Sing (2001)
10.1161/01.STR.31.12.2966
Anticoagulant Patient Information Material Is Written at High Readability Levels
C. A. Estrada (2000)
No disclaimer (“not a substitute for professional care
Pandolfini (2000)
Pharmacy benefit management sites not providing a " clear confidentiality statement
Kihlstrom (2001)
10.1007/BF03404290
Quality of Breast Cancer Sites on the World Wide Web
L. Hoffman-Goetz (2000)
10.1177/109019810002700603
The Content and Accessibility of Sex Education Information on the Internet
M. Smith (2000)
10.1136/jech.53.2.105
DISCERN: an instrument for judging the quality of written consumer health information on treatment choices.
D. Charnock (1999)
10.1046/j.1468-2982.2001.00137.x
Analysis of Internet Sites for Headache
S. Peroutka (2001)
10.1136/bmj.321.7265.843
Quality of websites: kitemarking the west wind
T. Delamothe (2000)
Index, subjective marks for writing style (no results reported)
Gunning Fog (1999)
The low availability of metadata elements for evaluating the quality of medical information on the World Wide Web
J. Shon (1999)
A critical appraisal of the use of Internet for calculating cardiovascular risk
P. Gillois (1999)
10.1097/00006534-200101000-00028
The Internet as a source of information on breast augmentation.
J. B. Gordon (2001)
Sites about hysterectomy where " confidentiality of data relating to individual patients was not respected " (eg, showing patients' photographs and full names) 7 26 26
10.1111/J.1752-7325.1974.TB04018.X
Quality of information in mass media: a barrier to the dental health education of the public.
P. Frazier (1974)
10.1055/S-0038-1634181
Labeling and filtering of medical information on the Internet.
G. Eysenbach (1999)
10.1111/J.1752-7325.1998.TB03000.X
Coverage and quality of oral cancer information in the popular press: 1987-98.
M. T. Canto (1998)
10.1038/45370
How reliable is science information on the web?
E. Allen (1999)
10.1080/146392399298375
Rx medication information for the public and the WWW: quality issues.
P. Doupi (1999)
10.1136/BMJ.317.7171.1469
Information epidemics, economics, and immunity on the internet
E. Coiera (1998)
10.1542/PEDS.101.6.E2
The Internet as a source for current patient information.
H. Mcclung (1998)
10.2307/2529310
The measurement of observer agreement for categorical data.
J. Landis (1977)
Accuracy of telephone reference service in health sciences libraries.
Paskoff Bm (1991)
10.1056/NEJM200108093450619
Web sites with misinformation about illicit drugs.
E. Boyer (2001)
Assessing the quality of newspaper medical advice columns for elderly readers.
F. Molnar (1999)
Quality of information on emergency contraception on the Internet.
M. Latthe (2000)
10.1016/0895-4356(91)90160-B
Validation of an index of the quality of review articles.
A. Oxman (1991)
10.1177/014107680009300510
Slimming on the Internet
J. Miles (2000)
10.2196/JMIR.1.2.E10
Online Prescribing of Sildanefil (Viagra) on the World Wide Web
G. Eysenbach (1999)
10.1046/j.1464-410x.2000.00831.x
Urology and the Internet: an evaluation of Internet use by urology patients and of information available on urological topics
G. Hellawell (2000)
10.1038/sj.jhh.1001003
A systematic review of patient information leaflets for hypertension
D. Fitzmaurice (2000)
A metadata vocabulary for self- and third-party labeling of health web-sites: Health Information Disclosure, Description and Evaluation Language (HIDDEL)
G. Eysenbach (2001)
10.1053/JPSU.2001.25751
Evaluation of pediatric surgery information on the Internet.
C. Corpron (2001)
An analysis of the readability and cultural sensitivity of information on the National Cancer Institute's Web site: CancerNet.
F. Wilson (2000)
10.1136/bmj.323.7321.1103
Ethical issues in qualitative research on internet communities
G. Eysenbach (2001)
10.1136/bmj.320.7251.1713
Consumer health informatics
G. Eysenbach (2000)
10.1067/MJD.2000.103044
Accessibility and reliability of cutaneous laser surgery information on the World Wide Web.
J. Bykowski (2000)
10.1002/(SICI)1097-0142(19990801)86:3<381::AID-CNCR4>3.0.CO;2-G
Evaluation of cancer information on the Internet
J. S. Biermann (1999)
The quality of dietary information on the World Wide Web.
K. Davison (1997)
HMO sites not providing information on monthly cost to consumers on their Web site 17 21 81
(2000)



This paper is referenced by
10.4184/jkss.2020.27.2.77
Evaluation of the Source and Quality of Information Regarding Cervical Disc Herniation on Websites
Y. C. Hong (2020)
10.1136/bmjopen-2011-000800
Do adverts increase the probability of finding online cognitive behavioural therapy for depression? Cross-sectional study
R. Jones (2012)
10.4236/AID.2014.41008
Evaluating the Quality of Malaria-Related Health Information in the Nigerian Internet Context
Trout Kate (2014)
10.1016/S1695-4033(04)78351-4
Interacción con los padres a través de internet de las sociedades pediátricas españolas y las integradas en la ALAPE
C. A. Vázquez (2004)
10.3233/THC-120706
Evaluation of metabolic syndrome related health information on internet in Indian context.
A. Joshi (2013)
10.1586/ern.12.162
Cyberchondria: towards a better understanding of excessive health-related Internet use
V. Starcevic (2013)
10.33915/etd.2865
Exploring West Virginia diabetes educators' perceptions of and experiences with integrating Internet technology in their practice
E. Quintana (2008)
10.3390/SOCSCI6020066
Challenges Confronting Rural Dwellers in Accessing Health Information in Ghana: Shai Osudoku District in Perspective
P. P. Sokey (2017)
10.3395/RECIIS.V7I1.451
Avaliação de sites de saúde em questão: a Aids nos sites brasileiros de Organizações Não Governamentais (ONG) de Lésbicas, Gays, Bissexuais, Travestis e Transexuais (LGBT)
André Pereira Neto (2013)
10.22038/ijorl.2019.31379.2031
Evaluation of the Quality of Otolaryngology Information on Persian Websites
A. Kouhi (2020)
Public Library Responses to a Consumer Health Inquiry in a Public Health Crisis: The SARS Experience in Ontario
Roma Harris (2005)
10.1007/0-387-36278-9_14
Consumer Health Informatics and Telehealth
P. Brennan (2006)
10.1016/j.ijmedinf.2005.07.034
The initial development of the WebMedQual scale: Domain assessment of the construct of quality of health web sites
Mélanie Provost (2006)
10.1016/S0738-3991(03)00116-2
What do patients with prostate or breast cancer want from an Internet site? A qualitative study of information needs.
L. Rozmovits (2004)
10.1001/ARCHOPHT.122.3.380
Ophthalmology on the internet: what do our patients find?
A. Kahana (2004)
10.1111/J.1365-277X.2004.00561.X
Advice available on the Internet for people with coeliac disease: an evaluation of the quality of websites.
C. England (2004)
Content Analysis of Selected Health Information Websites - Final Report
T. Petch (2004)
10.34172/doh.2020.01
Infodemic in the Global Coronavirus Crisis
Vahideh Zareh Gavgani (2020)
10.1016/j.bjan.2017.12.003
[What does the Internet teach the obstetric patient about labor analgesia?]
M. Weiss (2018)
10.1016/j.pec.2018.06.016
Online health information seeking by adults hospitalized for acute coronary syndromes: Who looks for information, and who discusses it with healthcare providers?
M. Waring (2018)
10.1016/J.FOODCONT.2016.01.003
Content analysis of web-based norovirus education materials targeting consumers who handle food: An assessment of alignment and readability
H. Evans (2016)
Medical Tourism Development in Croatia
Oliver Kesar Katarina Rimac (2011)
10.2196/jmir.1735
An Online Resource of Digital Stories About Cancer Genetics: Qualitative Study of Patient Preferences and Information Needs
R. Iredale (2011)
10.1016/J.ERGON.2018.03.003
An investigation of consumer's choice of a healthcare facility when user-generated anecdotal information is integrated into healthcare public reports
Sruthy Agnisarman (2018)
10.1093/BJA/AEM117
Quality of anaesthesia-related information accessed via Internet searches.
S. Caron (2007)
10.1111/J.1601-5037.2007.00248.X
The Internet and the oral healthcare professionals: potential and challenges of a new era.
N. Mattheos (2007)
10.1007/978-3-540-77010-7_48
An Assessment of the Currency of Free Science Information on the Web
C. Chen (2007)
10.1002/ARIS.144.V41:1
Credibility: A multidisciplinary framework
Soo Young Rieh (2007)
User-Centered Evaluation of the Quality of Blogs
Sutthinan Chuenchom (2011)
CURSO DE ESPECIALIZAÇÃO EM INFORMAÇÃO CIENTÍFICA E
Rodolfo Paolucci (2012)
10.1080/08952841.2012.708574
Readability of Menopause Web Sites: A Cross-Sectional Study
Deborah H. Charbonneau (2012)
Ana and her web?: an investigation into Internet use in adults with an eating disorder
Faye Yuill (2013)
See more
Semantic Scholar Logo Some data provided by SemanticScholar