Critical Appraisal of Clinical Practice Guidelines

Evidence-based clinical practice guidelines (CPGs) are summaries of best evidence, ideally from rigorous systematic reviews, which are then delineated as recommendations for practice. CPGs are a big help for busy clinicians – IF the CPG is based on evidence that is reliable and valid! This post will provide a “How To” of the questions to ask when you are conducting a critical appraisal of clinical practice guidelines. Explanations of why the specific questions are important to ask are included.

Quick Reminder: What is Critical Appraisal, Again?
A critical appraisal is basically a detailed examination of published research for the purpose of making a decision about scientific merit and, therefore, for making a decision about use of the evidence in practice.
Okay, so remember from my previous posts about critical appraisal (what is critical appraisal? Are the results valid? Are the results important? Can I apply the results?) that the purpose of doing a critical appraisal is to make a decision about a study’s scientific merit.

Why is evaluating scientific merit important? Because you only want to use research findings in practice that are generated from valid research methods.

You learn the basics of the nursing research process in your Nursing Research classes so that you can understand the reasoning behind questions you would ask for a research critique (AKA critical appraisal).

You can do your own critical appraisal of clinical practice guidelines by answering the critiquing questions in a nursing research textbook or by using a critical appraisal tool specific to the type of study you are reading.

However, the easiest way to make a decision about scientific merit about evidence sources is to use validated preappraised evidence. CPGs are near the top of the hierarchy of evidence pyramid and considered preappraised evidence because the developers of the CPG would have had to systematically gather, evaluate, and summarize the evidence regarding the topic of the CPG (e.g., acute coronary syndrome, arthritis, cystic fibrosis, etc.) to come up with the practice recommendations.

To review the purpose and role of critical appraisal in evidence-based practice, check out my series of posts on critical appraisal, starting with, What does “Critical Appraisal” Mean in Evidence-Based Practice? Also, read up on What does “Preappraised Evidence” Mean in Evidence-Based Practice?

And if you haven’t downloaded this yet, my free Evidence-Based Practice cheat sheet includes an intro to EBP and the major questions you need to ask when performing a critical appraisal for any type of study.

How are Clinical Practice Guidelines Developed?
CPGs condense a large amount of evidence into a practical source for busy clinicians.
Evidence-based CPGs are summaries of best evidence. The Summary level of evidence is the highest form of evidence that is readily accessible to clinicians. Because a CPG is a summary, it includes all the sources of evidence from the levels below the summary level. (See my post on the 6S Pyramid for details about the hierarchy of evidence levels.)

Because the information used to produce the CPG comes from a variety of sources (e.g., professional standards, quality assurance data, original research, expert consensus). Thus, CPGs are usually broader in scope than other evidence sources such as systematic reviews or meta-analyses. However, ideally, most of the evidence should come from rigorous systematic reviews (AGREE, 2017; IOM, 2011).

The evidence is then summarized and is delineated as recommendations for practice. Practice recommendations are succinct points for easy translation into practice. Each practice recommendation should have a level of evidence designation (to identify the quality of the studies that the recommendation is based on) and also be graded (as to the strength or scope of evidence that supports the recommendation).

The evidence-based recommendations should address all issues important in the clinical decision-making process. Though the CPG is a “guideline” thus implying flexibility in use, the point of the CPG is to influence practice in a definitive manner.

Because CPGs provide guidelines about a variety of patient care outcomes, CPG development should follow a rigorous and structured process. The critical questions to ask when doing a critical appraisal of clinical practice guidelines are related to whether the research results are valid and important: these answers will help you decide whether a CPG’s recommendations are trustworthy or not.

Critical Appraisal of Clinical Practice Guidelines: Where Do You Start?
If you are a graduate student or an advanced practice nurse (APN), in particular, you need to be familiar with how to critically appraise all kinds of study types – because staff will expect you to help guide them through the critical appraisal process. Undergraduate students and staff nurses should recognize the importance of critical appraisal to evidence-based practice and have a beginning understanding of how to appraise a research study.

A critical appraisal of clinical practice guidelines starts with asking questions related to validity and reliability of the methods used to produce the CPG. Once satisfied that the CPG is valid, you can look at how important and meaningful the CPG recommendations are before deciding to actually translate this evidence into practice by using the CPG to care for your patients.

There are tools available to specifically appraise CPGs. One is the AGREE tool, an acronym for “Appraisal of Guidelines for Research and Evaluation.” The original AGREE tool was developed in 2003; its purpose was to provide a quality standard for clinical practice guidelines. The newest revision of the tool is called AGREE II (AGREE, 2017). The AGREE II is a valid and reliable tool, with international support, that can guide authors in the development and reporting of CPGs, as well as how to appraise a CPG for quality and rigor.

Critical Appraisal of Clinical Practice Guidelines: How To
Critical Appraisal of Clinical Practice Guidelines
Critical Appraisal of Clinical Practice Guidelines Needs to Happen Before Implementation! Photo By shahreen/Shutterstock

After determining whether the CPG is necessary, a critical appraisal of clinical practice guidelines consists of asking the same three major critical appraisal questions used for any study to determine whether a CPG is acceptable for use.

I wrote about the major critical appraisal questions in previous posts. Is the Study Valid? (in this case, the CPG you are appraising) Are the Results Important? (i.e., the Practice Recommendations), and Can I apply the CPG to Practice?

For each CPG you appraise, be sure to identify which level of evidence scale was used to rate the quality of the study design and which grading tool the reviewers used to grade their practice recommendations.

What Questions Help Determine, Are the Results of the CPG Valid?

To ascertain whether the CPG should be used in practice, the following critical appraisal of clinical practice guidelines questions for determining validity should be asked:

Did the recommendations consider all relevant patient groups, management options, and possible outcomes? Were all important options and outcomes clearly specified? All groups and subgroups to whom these guidelines apply should be delineated, as well as costs, the effects of the interventions on morbidity and mortality, and specific outcome measures.
Was an explicit and sensible process used to identify, select, and combine evidence? Is there a systematic review of evidence linking options to outcomes for each relevant question? Every step in the methodological process of guideline development should be reported in detail and make sense (incl. systematic search strategies, inclusion/exclusion criteria for evidence selection, criteria for scientific quality determination, methods for conflict resolution, the process for recommendation formulation, external review process, etc.).
Processes that do not include structured methods are more likely to be biased and thus either underestimate or overestimate (more likely) the treatment effects and/or exaggerate or ignore consequences of the intervention (DiCenso et al., 2005).
Were there representatives from all relevant professional disciplines included in the guideline development group or involved in the parts of the development process?
What sources were used? Were systematic reviews used in the development of the guidelines? How strong was the evidence in the systematic reviews? Are any of the recommendations associated with patient values or ethical principles?
How were the recommendations graded?
Is the guideline likely to account for important recent developments? Is the guideline outdated by the time it is published? Compare the dates of the most recent evidence in the reference list and the publication date of the CPG. Do the authors note studies in progress? (DiCenso et al., 2005).
Has the guideline been subject to peer review and testing? Is there documentation that the CPG recommendations have been deemed acceptable by independent reviewers or clinicians in practice? Was the CPG validated (tested; piloted) in the clinical setting? Were changes made to the CPG as a result of testing and peer review?
Did the funding body influence the content of the CPG? Are conflicts of interest among the guideline developers made explicit?
Research is expensive; however, funding for the CPG development work should not influence how the evidence was selected, reviewed, or incorporated into the CPG. Look for a statement in the CPG that explicitly states that the “views or interests of the funding body have not influenced the final recommendations” (AGREE, 2017, p. 38).
Conflicts of interests among the guideline developers may also influence how the evidence was selected, reviewed, or incorporated into the CPG. Acknowledgement of potential conflicts of interests among the guideline developers (e.g., member of a speaker’s bureau or board member for a professional group) may be found in the CPG methods section, a list of guideline developers, a conflict of interest list, or the appendix).
What Questions Help Determine, Are the Results of the CPG Important?
The “results” in these critical appraisal of clinical practice guidelines questions refer to the practice recommendations made as a result of the synthesis of the evidence sources selected for this CPG.

Are practical, clinically important, recommendations made? Are the recommendations specific and easily identified? What is the intent of the CPG: to give recommendations about diagnosis, prognosis, prevention, screening, treatment, or palliation? Are the benefits greater than the risks?
How strong are the recommendations? Were the recommendations formally graded? What was the tool or process used and is it clear to you how to interpret the grading system?
What is the impact of uncertainty associated with the evidence and values used in the guidelines? How weak was the evidence? Was a sensitivity analysis reported? (FYI – a sensitivity analysis has nothing to do with test sensitivity — it’s asking if we change some of the information/ assumptions, how different will the outcomes be?)
What Questions Help Determine, Can I Apply the CPG to Practice?
Finally, once you are satisfied that the CPG is valid and important, questions related to practical application are the focus of these critical appraisal of clinical practice guidelines questions:

Is the primary objective of the guideline consistent with your objective? Does the CPG meet your needs for patient care?
Are the recommendations applicable to your patients? Was the target population clearly described so you can determine if your patient population is similar? Do the recommendations fit your patient population in terms of similar prevalence of disease, etc.?
Was the perspective of the target population sought and considered in the development of the recommendations?
Are recommendations made in enough detail that you understand how to implement them? What type of information do you need to provide to your patients for them to make an informed decision about their care? Do you need to individualize the guidelines or depart from their recommendations?
Bottom line: Clinical practice guidelines are preappraised evidence that you should be able to implement immediately in practice IF you are sure that the methods are transparent and rigorous, that the results (recommendations) are important, and that the target population matches your patient population.
How to Cite this Blogpost in APA*: Thompson, C. J. (2018, February 13). Critical appraisal of clinical practice guidelines [Web Log Post]. Retrieved from https://nursingeducationexpert.com/critical-appraisal-of-clinical-practice-guidelines *Citation should have hanging indent
References
AGREE Next Steps Consortium. (2017). The AGREE II Instrument [Electronic version]. Retrieved from http://www.agreetrust.org.

DiCenso, A., Guyatt, G., & Ciliska, D. (Eds.). (2005). Evidence-based nursing: A guide to clinical practice (pp. 155-171). St. Louis: Elsevier Mosby.

Institute of Medicine. (2011). Clinical practice guidelines we can trust. Washington, DC: The National Academies Press. https://doi.org/10.17226/13058.