Technical ReviewAn Independent AGREE Evaluation of the Occupational Medicine Practice Guidelines
Introduction
The information age has brought us technology which allows the sharing of data quickly and easily around the world. A physician now has the option of obtaining information regarding clinical practice management from many sources at any time.
It has long been the purpose of practice guidelines to organize the huge amount of relevant raw data for easier understanding and application in a clinical setting. Today, a physician can access a number of guidelines pertaining to the same condition or topic. Often, recommendations among guidelines vary, and in some cases they are conflicting. The question of how to select the best clinical guideline then arises. Health-care professionals need to critically evaluate guidelines to understand whether they are well constructed and representative of the preponderance of evidence. The guideline development process should be precise and rigorous to ensure that the results are reproducible and not vague [1], [2], [3].
Clinical practice guidelines impact the manner in which patient care is assessed by peer review and often serve as a basis for payer decision making regarding the delivery of spine care to patients. The American College of Occupational and Environmental Medicine (ACOEM) Guidelines have been adopted by several compensation systems as a standard for evaluation and management of work injuries. In California, where low back injuries comprised 39% of new worker's compensation claims in 2003 [4], the ACOEM Guidelines are legislatively mandated as “presumptively correct” for evaluation and management of all musculoskeletal injuries in the worker's compensation system.
Section snippets
Purpose
It is generally accepted that peer review and testing of health-care guidelines should be performed prior to their acceptance as being valid and their subsequent utilization in clinical practice [5], [6]. The purpose of this paper is to report the results of an analysis of the quality of the Occupational Medicine Practice Guidelines, Second Edition, published by the ACOEM [7]. The analysis employed the standardized AGREE (Appraisal of Guidelines Research and Evaluation) instrument to assesses
Study design/setting
Four appraisers with prior guideline evaluation experience volunteered to rate the ACOEM Guidelines using the AGREE instrument. All the participants have advanced training in the critical assessment of clinical data, and all are familiar with the AGREE instrument. Three are chiropractic physicians, one is a medical physician. One has a PhD, one a master's degree. Three appraisers have advanced certification in orthopedics, one in occupational and environmental medicine as well as in emergency
Instrument
The AGREE guideline evaluation instrument was selected as it has been assessed as reliable and valid for the purpose of guideline evaluation [10]. The AGREE instrument is a generic tool developed by the AGREE Collaboration in 1998. It was designed for the European Union under the Biomedicine and Health Research (BIOMED 2) Program to help guideline developers and users assess the methodological quality of clinical practice guidelines. Several additional countries, including Canada, have since
Results
Inter-examiner agreement for item scores was very high for the medical and chiropractic assessors. For most items across all chapters, agreement was 100%. When disagreement in the form of different item scores occurred, the disagreement was in degree of criterion satisfaction rather than whether or not the criterion had been satisfied. The medical reviewer's item score was consistently within the 95% confidence limits of the mean of the three chiropractic evaluators for that item when there was
Conclusions
The ACOEM Guidelines, Second Edition appears to have content consistent with their stated objectives, but the reporting of the guidelines construction process, particularly the rigor of recommendation development, is flawed, and the recommendations may not be valid because of possible evidence selection deficiencies. The reader should consider these flaws and limitations when using the guidelines. The reader should consider using guidelines of higher quality when possible. Future guidelines
References (21)
- et al.
Evaluating the quality of clinical practice guidelines
J Manip Physiol Therap
(2001) - et al.
An independent assessment of chiropractic practice guidelines
J Manipulative Physiol Ther
(2003) - et al.
Independent guideline appraisal summary report for guidelines for chiropractic quality assurance and practice parameters (Mercy)
J Chiropract Med
(2002) - et al.
Independent guideline appraisal summary report for vertebral subluxation in chiropractic practice (CCP) guidelines
J Chiropract Med
(2002) Development, use, and evaluation of clinical practice guidelines
J Occupa Environ Med/Am College Occupa Environ Med
(1997)- et al.
Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada
Int J Technol Assess Health Care
(2003) [Clinical practice guidelines: a reader's guide]
Revue Des Maladies Respiratoires
(2003)- California Workers' Compensation Institute 2004. AB749: An Analysis of Medical Utilization Trends. Available at...
- et al.
Users' guides to the medical literature. VIII. How to use clinical practice guidelines
JAMA
(1995) Occupational medicine practice guidelines: evaluation and management of common health problems and functional recovery of workers
(2004)
Cited by (55)
Workplace mental health: An international review of guidelines
2017, Preventive MedicineCitation Excerpt :Exploring the development process of the recommendations has been found to be key to successful implementation (Hulshof and Hoenen, 2007). Several reviews have evaluated the quality of occupational guidelines and highlighted that a lack of consultation with relevant professional and population groups, a narrow review of the literature and a lack of extra resources to assist in the implementation of recommendations are common (Cates et al., 2006; Hulshof and Hoenen, 2007; Kinnunen-Amoroso et al., 2009; Manchikanti et al., 2008; Staal et al., 2003). These shortcomings may explain why employers and relevant health professionals often fail to engage with recommendations.
Improvement evident but still necessary in clinical practice guideline quality: a systematic review
2017, Journal of Clinical EpidemiologyCitation Excerpt :In accordance with previous reviews of CPG quality [28,31,42–44], the present review found that domain scores were lowest in the Applicability and Editorial independence. These areas of CPG development have historically been weak and continue to be an area in which great improvement is possible [10,11,28,31,43–45]. Guideline development groups can improve their domain scores in Editorial independence by providing transparent and comprehensive information regarding funding sources and including a direct statement declaring the presence or the absence of each author's conflicts of interest.
Appraisal of guidelines developed by the World Health Organization
2014, Public HealthCitation Excerpt :Delineation of guidelines' scope and purpose, and clarity of presentation were generally better done, however, stakeholder engagement, editorial independence, and applicability were often poorly reported and/or executed. Our findings are similar to those reported in other studies examining cohorts of guidelines using the AGREE II instrument9,15–22 including several cohorts of WHO guidelines.8,9 Scores for the domains of scope and purpose, and clarity of presentation tend to be higher across these various studies, while scores for stakeholder involvement, editorial independence and applicability are often low.
Prevalence of Internet Addiction among Iranian University Students: A Systematic Review and Meta-Analysis
2022, Cyberpsychology, Behavior, and Social Networking
FDA device/drug status: not applicable.
Nothing of value received from a commercial entity related to this manuscript.