Skip Navigation Links.
Collapse <span class="m110 colortj mt20 fontw700">Volume 12 (2024)</span>Volume 12 (2024)
Collapse <span class="m110 colortj mt20 fontw700">Volume 11 (2023)</span>Volume 11 (2023)
Collapse <span class="m110 colortj mt20 fontw700">Volume 10 (2022)</span>Volume 10 (2022)
Collapse <span class="m110 colortj mt20 fontw700">Volume 9 (2021)</span>Volume 9 (2021)
Collapse <span class="m110 colortj mt20 fontw700">Volume 8 (2020)</span>Volume 8 (2020)
Collapse <span class="m110 colortj mt20 fontw700">Volume 7 (2019)</span>Volume 7 (2019)
Collapse <span class="m110 colortj mt20 fontw700">Volume 6 (2018)</span>Volume 6 (2018)
Collapse <span class="m110 colortj mt20 fontw700">Volume 5 (2017)</span>Volume 5 (2017)
Collapse <span class="m110 colortj mt20 fontw700">Volume 4 (2016)</span>Volume 4 (2016)
Collapse <span class="m110 colortj mt20 fontw700">Volume 3 (2015)</span>Volume 3 (2015)
Collapse <span class="m110 colortj mt20 fontw700">Volume 2 (2014)</span>Volume 2 (2014)
Collapse <span class="m110 colortj mt20 fontw700">Volume 1 (2013)</span>Volume 1 (2013)
American Journal of Public Health Research. 2013, 1(7), 183-190
DOI: 10.12691/AJPHR-1-7-7
Original Research

Development and Implementation of the Standards for Evaluating and Reporting Epidemiologic Studies on Chronic Disease Incidence or Prevalence

Tatyana Shamliyan1, 2, , Mohammed T. Ansari3, Gowri Raman4, Nancy Berkman5, Mark Grant6, Gail Janes7, Margaret Maglione8, David Moher3, Mona Nasser9, Karen Robinson10, Jodi Segal10 and Sophia Tsouros3

1Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis

20Elsevier Clinical Solutions, Senior Director, Quality assurance

3Clinical Epidemiology Methods Centre, Ottawa Health Research Institute, Ottawa

4Tufts University Medical Center, Boston

5RTI International – University of North Carolina, Chapel Hill

6Blue Cross and Blue Shield Association, Chicago

7Centers for Disease Control and Prevention, Atlanta

8Southern California EPC; RAND Corporation, Santa Monica

9University of Plymouth, Peninsula Dental School, Plymouth, UK

10Johns Hopkins University, Baltimore

Pub. Date: August 16, 2013
Full Text PDF

Cite this paper

Tatyana Shamliyan, Mohammed T. Ansari, Gowri Raman, Nancy Berkman, Mark Grant, Gail Janes, Margaret Maglione, David Moher, Mona Nasser, Karen Robinson, Jodi Segal and Sophia Tsouros. Development and Implementation of the Standards for Evaluating and Reporting Epidemiologic Studies on Chronic Disease Incidence or Prevalence. American Journal of Public Health Research. 2013; 1(7):183-190. doi: 10.12691/AJPHR-1-7-7

Abstract

We aimed to develop quality checklists for observational non-therapeutic studies. Based on a systematic review of current practices of quality assessment of observational studies, collaborating co-authors from Evidence-based Practice Centers and the Centers for Disease Control and Prevention developed a new checklist for studies examining incidence and prevalence of chronic conditions, evaluated face and content validity, and discrimination validity to distinguish reporting from methodological quality. This new checklist is available in text format or as a relational database to produce standardized reports with flaws in reporting quality, external (six criteria), and internal (five criteria) validity of the studies. Study and hypotheses (subgroups) level analyses are possible with predetermined in protocol templates criteria of major and minor flaws. Consensus around justified research specific methodological standards and reliability tests should precede quality evaluation of primary studies to assure confidence in quality assessment. To be effective, policy decisions should be made based on comprehensive systematic evidence reviews that include transparent, standardized quality appraisals. Implementation of the developed checklists would increase transparency and quality of research leading to effective informed decisions in health care.

Keywords

risk factors, morbidity, reproducibility of results, validation studies, bias (epidemiology), quality control, review literature as topic

Copyright

Creative CommonsThis work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

References

[1]  Institute of Medicine (U.S.). Finding What Works in Health Care: Standards for Systematic Reviews. Heidelberg, Neckar: National Academies Press; 2011.
 
[2]  Bero LA, Jadad AR. How consumers and policymakers can use systematic reviews for decision making. Annals of internal medicine. Jul 1 1997;127(1):37-42.
 
[3]  Briss PA, Brownson RC, Fielding JE, Zaza S. Developing and using the Guide to Community Preventive Services: lessons learned about evidence-based public health. Annual review of public health. 2004;25:281-302.
 
[4]  Briss PA, Zaza S, Pappaioanou M, et al. Developing an evidence-based Guide to Community Preventive Services--methods. The Task Force on Community Preventive Services. American journal of preventive medicine. Jan 2000;18(1 Suppl):35-43.
 
[5]  Chan KS, Morton SC, Shekelle PG. Systematic reviews for evidence-based management: how to find them and what to do with them. The American journal of managed care. Nov 2004;10(11 Pt 1):806-812.
 
[6]  National Center for Chronic Disease Prevention and Health Promotion. Chronic Disease Overview. http://www.cdc.gov/NCCdphp/overview.htm. 2013;Accessed August 2013.
 
[7]  Fox DM. Evidence of evidence-based health policy: the politics of systematic reviews in coverage decisions. Health affairs. Jan-Feb 2005;24(1):114-122.
 
[8]  Lavis J, Davies H, Oxman A, Denis JL, Golden-Biddle K, Ferlie E. Towards systematic reviews that inform health care management and policy-making. Journal of health services research & policy. Jul 2005;10 Suppl 1:35-48.
 
[9]  Chapman DP, Williams SM, Strine TW, Anda RF, Moore MJ. Dementia and its implications for public health. Preventing chronic disease. Apr 2006;3(2):A34.
 
[10]  Launer LJ, Andersen K, Dewey ME, et al. Rates and risk factors for dementia and Alzheimer's disease: results from EURODEM pooled analyses. EURODEM Incidence Research Group and Work Groups. European Studies of Dementia. Neurology. Jan 1 1999;52(1):78-84.
 
[11]  Aschengrau A SG. Essentials of Epidemiology in Public Health. Sudbury, Mass. 2003;Jones and Bartlett.
 
[12]  West S, King V, Carey TS, et al. Systems to rate the strength of scientific evidence. Evidence report/technology assessment. Mar 2002(47):1-11.
 
[13]  Higgins JP, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. Bmj. 2011;343:d5928.
 
[14]  Viswanathan M, Ansari MT, Berkman ND, et al. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Methods Guide for Effectiveness and Comparative Effectiveness Reviews. Rockville (MD)2008.
 
[15]  Viswanathan M, Berkman ND. Development of the RTI item bank on risk of bias and precision of observational studies. J Clin Epidemiol. Feb 2012;65(2):163-178.
 
[16]  16.Shamliyan T, Kane RL, Dickinson S. A systematic review of tools used to assess the quality of observational studies that examine incidence or prevalence and risk factors for diseases. J Clin Epidemiol. Oct 2010;63(10):1061-1070.
 
[17]  Shamliyan T, Kane RL, Jansen S. Quality of systematic reviews of observational nontherapeutic studies. Preventing chronic disease. Nov 2010;7(6):A133.
 
[18]  Shamliyan T, Kane RL, Jansen S. Systematic reviews synthesized evidence without consistent quality assessment of primary studies examining epidemiology of chronic diseases. J Clin Epidemiol. Jun 2012;65(6):610-618.
 
[19]  Centers for Disease Control and Prevention (U.S.). Principles of epidemiology in public health practice : an introduction to applied epidemiology and biostatistics. Atlanta, GA: U.S. Dept. of Health and Human Services, Centers for Disease Control and Prevention (CDC), Office of Workforce and Career Development. 2006;3rd ed.
 
[20]  Loney PL, Chambers LW, Bennett KJ, Roberts JG, Stratford PW. Critical appraisal of the health research literature: prevalence or incidence of a health problem. Chronic diseases in Canada. 1998;19(4):170-176.
 
[21]  Woodbury MG, Houghton PE. Prevalence of pressure ulcers in Canadian healthcare settings. Ostomy/wound management. Oct 2004;50(10):22-24, 26, 28, 30, 32, 34, 36-28.
 
[22]  Macfarlane TV, Glenny AM, Worthington HV. Systematic review of population-based epidemiological studies of oro-facial pain. Journal of dentistry. Sep 2001;29(7):451-467.
 
[23]  Lundh A, Gotzsche PC. Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies. BMC medical research methodology. 2008;8:22.
 
[24]  DuRant RH. Checklist for the evaluation of research articles. The Journal of adolescent health : official publication of the Society for Adolescent Medicine. Jan 1994;15(1):4-8.
 
[25]  Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. Journal of epidemiology and community health. Jun 1998;52(6):377-384.
 
[26]  Kaplin AI, Williams M. How common are the "common" neurologic disorders? Neurology. Jul 24 2007;69(4):410; author reply 410-411.
 
[27]  Hirtz D, Thurman DJ, Gwinn-Hardy K, Mohamed M, Chaudhuri AR, Zalutsky R. How common are the "common" neurologic disorders? Neurology. Jan 30 2007;68(5):326-337.
 
[28]  Feinstein AR. Clinimetrics. New Haven: Yale University Press. 1987.
 
[29]  Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. International journal of epidemiology. Jun 2007;36(3):666-676.
 
[30]  Hulley SB. Designing clinical research: an epidemiologic approach. Philadelphia: Lippincott Williams & Wilkins. 2001;2nd ed.
 
[31]  BMJ Publishing Group Limited. Clinical evidence. clinicalevidence.bmj.com. 2013;Accessed in August 2013.
 
[32]  National Library of Medicine (U.S.). NIoHUS. PubMed Central. Bethesda, MD. 2013.
 
[33]  Higgins J GS. Cochrane handbook for systematic reviews of interventions. Cochrane Collaboration. 2011;Chichester, West Sussex(Hoboken NJ: John Wiley & Sons).
 
[34]  Atkins D, Briss PA, Eccles M, et al. Systems for grading the quality of evidence and the strength of recommendations II: pilot study of a new system. BMC health services research. Mar 23 2005;5(1):25.
 
[35]  Atkins D, Eccles M, Flottorp S, et al. Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group. BMC health services research. Dec 22 2004;4(1):38.
 
[36]  Shamliyan TA, Kane RL, Ansari MT, et al. Development quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists. J Clin Epidemiol. Jun 2011;64(6):637-657.
 
[37]  Efron B TR. An introduction to the bootstrap. New York: Chapman & Hall. 1993.
 
[38]  R: A language and environment for statistical computing. Foundation for Statistical Computing. Vienna, Austria. 2013.
 
[39]  King JE. Software solutions for obtaining a kappa-type statistic for use with multiple raters. The annual meeting of the Southwest Educational Research Association. 2004; Dallas, TX.
 
[40]  Gwet K. Inter-rater reliability: dependency on trait prevalence and marginal homogeneity. Statistical Methods for Inter-Rater Reliability Assessment Series. 2002;2:1-9.
 
[41]  Gwet K. Computing inter-rater reliability with the SAS system. Stat Methods Inter-rater Reliability Assess. 2002;3:1-16.
 
[42]  Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. The British journal of mathematical and statistical psychology. May 2008;61(Pt 1):29-48.
 
[43]  Denny CH, Holtzman D, Goins RT, Croft JB. Disparities in chronic disease risk factors and health status between American Indian/Alaska Native and White elders: findings from a telephone survey, 2001 and 2002. American journal of public health. May 2005;95(5):825-827.
 
[44]  Collins JG. Prevalence of selected chronic conditions: United States, 1990-1992. Vital and health statistics. Series 10, Data from the National Health Survey. Jan 1997(194):1-89.
 
[45]  Blumberg SJ, Welch EM, Chowdhury SR, Upchurch HL, Parker EK, Skalland BJ. Design and operation of the National Survey of Children with Special Health Care Needs, 2005-2006. Vital and health statistics. Ser. 1, Programs and collection procedures. Dec 2008(45):1-188.
 
[46]  Mack KA, Ahluwalia IB. Observations from the CDC: Monitoring women's health in the United States: selected chronic disease indicators, 1991-2001 BRFSS. Journal of women's health. May 2003;12(4):309-314.
 
[47]  Wilt TJ, Shamliyan T, Taylor B, et al. Comparative Effectiveness of Therapies for Clinically Localized Prostate Cancer. AHRQ Comparative Effectiveness Reviews. 2008.
 
[48]  Nelson R, Norton N, Cautley E, Furner S. Community-based prevalence of anal incontinence. JAMA : the journal of the American Medical Association. Aug 16 1995;274(7):559-561.
 
[49]  Teunissen TA, van den Bosch WJ, van den Hoogen HJ, Lagro-Janssen AL. Prevalence of urinary, fecal and double incontinence in the elderly living at home. International urogynecology journal and pelvic floor dysfunction. Jan-Feb 2004;15(1):10-13; discussion 13.
 
[50]  Johanson JF, Lafferty J. Epidemiology of fecal incontinence: the silent affliction. The American journal of gastroenterology. Jan 1996;91(1):33-36.
 
[51]  Fleming DT, McQuillan GM, Johnson RE, et al. Herpes simplex virus type 2 in the United States, 1976 to 1994. The New England journal of medicine. Oct 16 1997;337(16):1105-1111.
 
[52]  Shamliyan T, Wyman J, Bliss DZ, Kane RL, Wilt TJ. Prevention of urinary and fecal incontinence in adults. Evidence report/technology assessment. Dec 2007(161):1-379.
 
[53]  Jacob K, Talwar S, Copplestone A, Gilbert TJ, Haywood GA. Activation of coagulation occurs after electrical cardioversion in patients with chronic atrial fibrillation despite optimal anticoagulation with warfarin. International journal of cardiology. May 2004;95(1):83-88.
 
[54]  U.S. Congress OoTA. The Continuing Challenge of Tuberculosis. Washington, DC: U.S. Government Printing Office. 1993;OTA-H-574(http://ota-cdn.fas.org/reports/9347.pdf):Accessed in August 2013.
 
[55]  55.Nagelkerke NJ, Borgdorff MW, Kalisvaart NA, Broekmans JF. The design of multi-stage tuberculin surveys: some suggestions for sampling. The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease. Apr 2000;4(4):314-320.
 
[56]  Davidow AL, Katz D, Reves R, Bethel J, Ngong L, Tuberculosis Epidemiologic Studies C. The challenge of multisite epidemiologic studies in diverse populations: design and implementation of a 22-site study of tuberculosis in foreign-born people. Public health reports. May-Jun 2009;124(3):391-399.
 
[57]  Shaghaghi A, Matlabi H. Reporting of Health Promotion Research: Addressing the Quality Gaps in Iran. Health Promotion. 2012;2(1):48-52.
 
[58]  Kleijn SA, Aly MF, Knol DL, et al. A meta-analysis of left ventricular dyssynchrony assessment and prediction of response to cardiac resynchronization therapy by three-dimensional echocardiography. European Heart Journal–Cardiovascular Imaging. 2012;13(9):763-775.
 
[59]  Malboosbaf R, Hosseinpanah F, Mojarrad M, Jambarsang S, Azizi F. Relationship between goiter and gender: a systematic review and meta-analysis. Endocrine. 2012:1-9.
 
[60]  Slattery J, Morgan A, Douglas J. Early sucking and swallowing problems as predictors of neurodevelopmental outcome in children with neonatal brain injury: a systematic review. Developmental Medicine & Child Neurology. 2012;54(9):796-806.
 
[61]  Robroek SJ, Reeuwijk KG, Hillier FC, Bambra CL, van Rijn RM, Burdorf A. The contribution of overweight, obesity, and lack of physical activity to exit from paid employment: a meta-analysis. Scandinavian journal of work, environment & health. 2013;39(3):233-240.
 
[62]  Manfredini D, Restrepo C, Diaz‐Serrano K, Winocur E, Lobbezoo F. Prevalence of sleep bruxism in children: a systematic review of the literature. Journal of oral rehabilitation. 2013.
 
[63]  Khalesi M, Whiteman DC, Doi SA, Clark J, Kimlin MG, Neale RE. Cutaneous markers of photo-damage and risk of basal cell carcinoma of the skin: A meta-analysis. Cancer Epidemiology Biomarkers & Prevention. 2013.
 
[64]  Edmondson D, Richardson S, Fausett JK, Falzon L, Howard VJ, Kronish IM. Prevalence of PTSD in Survivors of Stroke and Transient Ischemic Attack: A Meta-Analytic Review. PLOS ONE. 2013;8(6):e66435.
 
[65]  Sale JE, Beaton D, Posen J, Bogoch E. Medication initiation rates are not directly comparable across secondary fracture prevention programs: reporting standards based on a systematic review. Journal of clinical epidemiology. 2013;66(4):379-385. e374.
 
[66]  Hoy D, Brooks P, Woolf A, et al. Assessing risk of bias in prevalence studies: modification of an existing tool and evidence of interrater agreement. J Clin Epidemiol. Sep 2012;65(9):934-939.
 
[67]  Groenwold RH, Rovers MM. The Catch-22 of appraisals on the quality of observational studies. J Clin Epidemiol. Oct 2010;63(10):1059-1060.
 
[68]  Kane RL, Shamliyan T. Be specific and dare to generalize: do we need a rating form for every disease? J Clin Epidemiol. Sep 2012;65(9):921-923.
 
[69]  Booth A, Clarke M, Dooley G, et al. PROSPERO at one year: an evaluation of its utility. Systematic reviews. 2013;2:4.
 
[70]  Van der Wees P, Qaseem A, Kaila M, Ollenschlaeger G, Rosenfeld R, Board of Trustees of the Guidelines International N. Prospective systematic review registration: perspective from the Guidelines International Network (G-I-N). Systematic reviews. 2012;1:3.
 
[71]  Qaseem A, Forland F, Macbeth F, et al. Guidelines International Network: toward international standards for clinical practice guidelines. Annals of internal medicine. Apr 3 2012;156(7):525-531.