Critical Appraisal of Systematic Reviews and Meta-analyses
*Corresponding Author:
Leonardo Roever, Department of Clinical Research, Av Pará, 1720 - Bairro Umuarama, Uberlândia-MG-CEP 38400-902, Brazil, Tel: +553488039878, Email: leonardoroever@hotmail.com
Received: 08-Dec-2015 / Accepted Date: 15-Dec-2015 / Published Date: 22-Dec-2015 DOI: 10.4172/2471-9919.1000e106
Introduction
A systematic review is a form of research using as a data source literature on a particular topic, are particularly useful for integrating information from a number of studies carried out separately about certain therapeutics/intervention, which may present conflicting results and/ or coincident, and to identify issues that need evidence, by applying explicit methods and systematic search, critical assessment and synthesis of the selected information. Meta-analysis is a statistical method used in systematic reviews to integrate the results of the included studies and increase the statistical power of the study. The Table 1 shows the checklists needed to make a critical analysis of a systematic reviews and meta-analysis [1-19].
| Appraisal questions |
|---|
| The research question is clearly defined and the inclusion/exclusion criteria must be listed in the paper. A comprehensive literature search is carried out. |
| Did the review address a clearly focused issue? Was there enough information on: The population studied The intervention given The outcomes considered Did the authors look for the appropriate sort of papers? Have an appropriate study design. |
| Do you think the important, relevant studies were included? Look for Which bibliographic databases were used Follow up from reference lists Personal contact with experts Search for unpublished as well as published studies Search for non-English language studies |
| Did the review’s authors do enough to assess the quality of the included studies? The authors need to consider the rigor of the studies they have identified. |
| Is the clinical question clearly focused with regard to: the population? The intervention? The outcome measures? Are the criteria for the selection of the studies to be included in the review in accordance with: the specifications of the foregoing question in regard to populations, interventions and results? The type of research design that will be chosen? Is the literature search method clearly specified? Is there a high probability that some relevant studies may have been omitted? Have the identified studies been evaluated for methodological quality? Was the methodological quality evaluation carried out by more than one person independently, and the degree of agreement between them established? |
| At least two people should have selected studies. At least two people should have extracted data. |
| Is this a systematic review of randomised trials? |
| Does it include a methods section that describes: Finding and including all relevant trials? Assessing their individual validity? |
| Were the results consistent from study to study? Were the individual patient data used in the analysis (or aggregate data)? |
| The status of publication was not used as an inclusion criterion. The excluded studies are listed. The relevant characteristics of the included studies are provided. The scientific quality of the included studies was assessed and reported. |
| Was the scientific quality of the included studies used appropriately? Appropriate methods are used to combine the individual study findings. The likelihood of publication bias was assessed appropriately. |
| Are the valid results of this systematic review important? Can you apply this valid, important evidence from a systematic review in caring for your patient? Is your patient so different from those in the study that its results cannot apply? |
| If the results of the review have been combined, was it reasonable to do so? Consider whether · The results were similar from study to study The results of all the included studies are clearly displayed The results of the different studies are similar The reasons for any variations are discussed |
| What is the overall result of the review? Consider If you are clear about the reviews ‘bottom line’ results What these are (numerically if appropriate) How were the results expressed (NNT, odds ratio, etc.) |
| Are the results presented with confidence intervals? Were the results consistent from one study to another? What were the overall results of the review? How precise were the results? |
| What are your patient’s potential benefits and harms from the therapy? Method I: In the OR tables above, find the intersection of the closest odds ratio from the systematic review and your patient’s expected event rate (PEER) Method II: To calculate the NNT from any OR and PEER: |
| Are your patient’s values and preferences satisfied by the regimen and its consequences? Do you and you patient have a clear assessment of their values and preferences? Are they met by this regimen and its consequences? |
| Should you believe apparent qualitative differences in the efficacy of therapy in some subgroups of patients? Do they really make biologic and clinical sense? Is the qualitative difference both clinically (beneficial for some but useless or harmful for others) and statistically significant? Was this difference hypothesized before the study began (rather than the product of dredging the data), and has it been confirmed in other, independent studies? |
| Can the results be applied to the local population? Consider whether The patients covered by the review could be sufficiently different from your population to cause concern Your local setting is likely to differ much from that of the review Are my patients similar to the patients included in the original studies? Is the intervention feasible in my setting? Have all the clinically relevant results been taken into consideration? Do the benefits outweigh the potential harm? Were all important outcomes considered? Are the benefits worth the harms and costs? Even if this is not addressed by the review, what do you think? Conflicts of interest are declared. |
| What is your overall assessment of the methodological quality of this review? Are the results of this study directly applicable to the patient group targeted by this guideline? |
Rate the overall methodological quality of the study, using the following as a guide:
High quality(++): Majority of criteria met. Little or no risk of bias.
Low quality (-): Either most criteria not met, or significant flaws relating to key aspects of study design.
Reject (0): Poor quality study with significant flaws. Wrong study type. Not relevant to guideline
High quality(++): Majority of criteria met. Little or no risk of bias.
Low quality (-): Either most criteria not met, or significant flaws relating to key aspects of study design.
Reject (0): Poor quality study with significant flaws. Wrong study type. Not relevant to guideline
Table 1: Critical appraisal of prognostic studies.
Use this checklist can improve the evaluation of prognostic studies.
References
- Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, et al. (2010) Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology 7:10.
- Critical Appraisal Skills Programme (CASP), Public Health Resource Unit, Institute of Health Science, Oxford.
- Clegg, Andrew, Hewitson, Paul, Milne, et al. (2004) Explicit and reproducible: how to assess the quality of the evidence in a systematic review. StudentBMJ 8:24.
- Heneghan, Carl,Badenoch, Douglas. Evidence-based Medicine Toolkit, 2nd edition. Oxford: Blackwell Publishing.
- Systematic Review Appraisal Sheet. Centre for Evidence Based Medicine. University of Oxford.
- Guyatt G. Rennie D. Meade MO, Cook DJ (2008) Users’ Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice.
- Greenalgh T (1997) How to read a paper: Papers that summarise other papers: systematic reviews and meta-analysis. British Medical Journal 315:672-675.
- Jadad AR, Moher D, Klassen TP (1998) Guides for reading and interpreting systematic reviews: II. How did the authors find the studies and assess their quality. Archives of Pediatricand Adolescent Medicine 152:812-817.
- Oxman AD, Guyatt GH (1993) The science of reviewing research. Ann N Y AcadSci 703: 125-133.
- Oxman AD, Guyatt GH (1991) Validation of an index of the quality of review articles. J ClinEpidemiol 44: 1271-1278.
- Mulrow CD (1987) The medical review article: state of the science. See comment in PubMed Commons below Ann Intern Med 106: 485-488.
- Light RJ, Pillemer DB (1984) Summing Up: The Science of Reviewing Research. Cambridge: Harvard University Press.
- Linde K, Willich SN (2003) How objective are systematic reviews? Differences between reviews on complementary medicine. J R Soc Med 96:17-22.
- Akobeng AK (2005) Understanding systematic reviews and meta-analysis. Arch Dis Child 90: 845-848.
- Sacks HS, Reitman D, Pagano D, Kupelnick B (1996) Meta-analysis: an update. Mt Sinai J Med 63: 216-224.
- Zhou A, Obuchowski N, McClish D (2002) Issues in meta-analysis for diagnostic tests. Statistical methods in diagnostic medicine. New York.
- Alderson PGS, Higgins JPT (2004) Cochrane Reviewers' Handbook 4.2.2, Chichester, UK.
- Moher DCD, Eastwood S, Olkin I, Rennie D, Stroup DF (1999) Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet 354:1896-1900.
Citation: Roever L, Zoccai GB (2015) Critical Appraisal of Systematic Reviews and Meta-analyses. Evidence Based Medicine and Practice 1: e106. Doi: 10.4172/2471-9919.1000e106
Copyright: © 2015 Roever L, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Share This Article
Open Access Journals
Article Tools
Article Usage
- Total views: 13259
- [From(publication date): 0-2016 - Apr 23, 2024]
- Breakdown by view type
- HTML page views: 12195
- PDF downloads: 1064
