Critical Appraisal Questions for Evidence-Based Practice: Can I Apply the Results to Practice?

Using Evidence to Make Clinical Decisions
The basic process of EBP was devised for the purpose of clinical decision making about individual patients. The unit of study you are interested in may not be patients, but whether it’s hospital units or nursing staff or healthcare systems or students – the endpoint is the same – Can you use the evidence from the literature to change or improve your clinical/administrative/teaching/or advanced practice?

The process is the same regardless of practice setting or discipline.

After deciding that the study is worth your time, you determine whether you can believe the results by examining the study methodology for bias. Only when you are satisfied that the study was conducted rigorously do you care about the results. First, decide whether the results are important and significant. If the study ticks those boxes, you can finally decide if you can apply the results to your patient or patient population.

So let’s look at how to make those evidence-based decisions about patient care (or your topic of study).

Generalizability and Transferability of the Research Findings
“Research evidence can be used only if it has some relevance to settings and people outside of the contexts studied” (Polit & Beck, 2010, p. 1452).
Anytime we consult the literature to answer a clinical question, we must evaluate the quality of the study, the importance of the problem studied, and the applicability of the findings to our patient population (Newman & Roberts, 2002). The point of research is to, ultimately, use the research findings in practice. Practice in one specialty may differ from practice in another specialty, but the goal is the same — to translate research so that the findings can be used in practice.

Critical appraisal of research is the important first step to basing your practice on evidence. The last step is deciding whether your patient or patient population is similar to those studied so you can apply the results to practice.

Another name for application of research findings to practice is generalizability or external validity in quantitative research and transferability in qualitative research (Polit & Beck, 2010). These terms simply mean, can we generalize (or transfer) the results from this study sample to that of the greater population? Can we apply the results to our patients or clinical practice?

Glasziou, Vandenbroucke, and Chalmers (2004) posed a variety of issues to think about when making decisions about applying evidence to your own practice setting. Briefly:

The clinician needs to understand that research studies have limitations as to the type of questions they can answer; a randomized controlled trial is good to ascertain the effect of interventions, but not as good to determine prognosis or the patient’s lived experience;
Research should be interpreted with other studies on the same topic. Systematic reviews of well-conducted, high-quality studies should be sought.
Evidence grading should be more than just identifying study type;
A variety of studies should be searched for, examined, and collated to come up with a comprehensive answer to a clinical question;
Clinicians should understand how to critically appraise research.
“For interventions, the best available evidence for each outcome of potential importance to patients is needed” (Glasziou et al., 2004, p. 40).
What Subquestions Help Determine, Can You Apply the Results in Practice?
Now that you’ve determined that the study is methodologically sound and that the results are important, you need to consider the patient’s situation and your patient’s preferences (the third piece of the EBP “pie”) to determine if you can apply the study results to your individual patient (or unit, or hospital or students – whatever the unit of study or variables studied were).

How Can I Apply the Results to Patient Care?
DiCenso and Guyatt (2005a) stated that there are three issues the clinician must examine to ensure that characteristics of the study group do not differ significantly from the patients you are hoping to apply the results to: biologic issues, socioeconomic issues, and epidemiologic issues. I’ll discuss these as we talk about the subquestions for an intervention study.

Were the study patients similar to the patients in my clinical setting? (Substitute if needed: Were the study units similar or Was the hospital system similar or Were the students similar… etc.)
Where do you find this information? Look at the beginning of the research results and see how the researchers described their sample. Consider gender, age range, diagnosis, comorbidities, ethnicity, geography, economic status, education level — how did the researchers describe their average subject? Look for other information about the patients throughout the study, especially in the discussion and implications sections of the article.

Biologic Issues: Consider if there are pathophysiologic differences related to the condition being studied or patient differences that might cause a reduced or weakened response to the treatment. Internal stressors and responses (think pharmacokinetics and drug metabolism, immune and circulatory function, organ function, etc.) and environmental factors may be different for different groups of people and affect the patient’s response to the intervention.

Socioeconomic Issues: Consider if there are socioeconomic differences related to the condition being studied that might affect patient adherence. Are adherence factors discussed in the study? Were there resource limitations in one group versus the other? Are prognostic factors unbalanced between the groups?

Were the providers influenced by adherence factors such as resource limitations (lack of skills or providers, etc.), equipment shortages, or technological expertise?

Epidemiologic Issues: When you are satisfied that the aforementioned issues have not influenced the patients’ responses to the intervention, “consider patient characteristics that could influence the magnitude of the benefits or risks … (and, thus, the tradeoff between the two)” (p. 485). This question has to do with comorbidities that could affect response to treatment. Certain conditions can mimic others or can “shift the balance between benefit and risk” (p. 485).

Were all important outcomes considered? Are substitute endpoints valid? Substitute endpoints or outcomes are also known as surrogate endpoints or outcomes — these terms are interchangeable. These are outcomes that are used in place of an outcome that is directly important to the patient when circumstances (such as data or sample availability) preclude the researcher from using direct outcomes. Outcomes of direct interest to the patient include things like infection rate, length of stay, cardiovascular event (e.g., MI, stroke), etc.
Surrogate outcomes: “Outcomes that are not in themselves important to patients, but are associated with outcomes that are important to patients, (e.g., bone density for fracture, cholesterol for myocardial infarction, and blood pressure for stroke” (DiCenso et al., 2005b, p. 570; emphasis added).

Surrogate outcomes need to make sense. The clinician should be able to see the link between the surrogate outcome and the condition of interest.

Are the likely benefits worth the potential risks? This question is pretty straightforward — what is the patient’s risk of harm or adverse events? First, ask is my patient’s baseline risk similar to that of the patients in the study? Second, ask how is my patient’s risk of harm related to the magnitude of the effect (think risk measures, number-needed-to-treat/harm).
Where do you find this information? Look for information in the Results section about the risks and note any specific risks to patient subgroups that are elucidated. Prognostic studies are good sources of information to identify risks of patient subgroups. For example, according to the MONICA project, death from coronary heart disease is lower in East Asians than in Americans and Western Europeans (Dicenso & Guyatt, 2005a).

Baseline risks can vary based on a number of factors – socioeconomic status, country of origin and residence, genetic predisposition, etc. So that means you need to know a bit about the patients you are treating or caring for, right?

Remember that your patient is likely to have some differences from the study participants. The bottom line is to determine if the differences are “important enough” to diminish or negate the benefits of the intervention that the study patients enjoyed or to increase the risk of harm. Sometimes, you’ll be able to implement an intervention by adjusting the dose or modifying it in some way – sometimes, not.

Is it feasible for me to implement this intervention in my practice setting? Of course, even if the intervention passes all of the other critical appraisal questions, this is a question that always has to be answered before translating research into practice. Depending on what the intervention is, you have to think about whether resources are available to pay for or secure the treatment, store it, implement it, monitor and measure it, troubleshoot, maintain, etc. Are you allowed to implement this treatment intervention?
If your patient meets the inclusion criteria for the study being appraised and has no exclusionary characteristics (and the answers to the previous critical appraisal questions is satisfactory) you could be confident that the intervention you are applying will have the desired results (DiCenso and Guyatt, 2005a). DiCenso and Guyatt (2005b) suggested that “you ask whether there is some compelling reason that the results should not be applied to the patient” (p. 65).

Conclusion

Again, these subquestions are specific to a patient care intervention, but the general principles work for most areas of interest.

For example, let’s say you are a nurse manager and you want to implement a new staffing model based on the evidence. The questions of generalizability to nursing staff would be similar to that of patient care.

Are your nurses similar to the study participants? Are there significant differences between your staff and the study participants that would change how the model will be viewed? accepted? implemented? Were important outcomes related to nurse staffing considered and measured? Are the benefits of the model greater than the potential risks (poor patient care, nurses quitting, recruitment problems, etc.)? Do you have the authority (are you allowed) to change the current staffing model?

Bottom line: If your patient (unit, hospital, etc.) is similar to the patients sampled in the study you are appraising, outcomes important to patients were studied, the benefits are greater than the risks, and the intervention is feasible to perform in your setting, you could consider applying the study results to your patient.
How to Cite this Blogpost in APA*: Thompson, C. J. (2017, November 28). Critical appraisal questions in evidence-based practice: Can I apply the results to practice? [Blogpost]. Retrieved from https://nursingeducationexpert.com/critical-appraisal-apply-the-results *Citation should have hanging indent
References
DiCenso, A., & Guyatt, G. (2005a). Applying results to individual patients. In A. DiCenso, G. Guyatt, and D. Ciliska (Eds.), Evidence-based nursing: A guide to clinical practice (pp. 481-489). St. Louis, MO: Elsevier Mosby.

DiCenso, A., Guyatt, G., & Ciliska, D. (Eds.). (2005b). Evidence-based nursing: A guide to clinical practice. St. Louis, MO: Elsevier Mosby.

Glasziou, P., Vandenbroucke, J., & Chalmers, I. (2004). Assessing the quality of research. British Medical Journal, 328(7430), 39-41.

Newman, M., & Roberts, T. (2002). Critical appraisal I: Is the quality of the study good enough for you to use the findings? In J. V. Craig and R. L. Smyth (Eds.), The evidence-based practice manual for nurses (pp. 86-113). Edinburgh, Scotland: Churchill Livingstone.

Polit, D. F., & Beck, C. T. (2010). Generalization in quantitative and qualitative research: Myths and strategies. International Journal of Nursing Studies, 47, 1451–1458. doi:10.1016/j.ijnurstu.2010.06.004