Appraisal sheets

HOME INDEX SEARCH DATABASE WHAT IS A CAT? WHAT CAN YOU DO? INTRODUCTION HBO EXPLAINED

 

The tables below are standardised critical appraisal sheets I use to help write the CATs on this site. I find that the answers to each of these questions helps me to avoid missing a fatal flaw in the paper, and directly assists the correct information getting into the final CAT.

 

HBO EVIDENCE CRITICAL APPRAISAL WORKSHEET For RCTs (and non-random comparative trials)

 

 

Important information that should be in the paper	Potential related problems	Threats to the internal and external validity of the study[1]
1.1aWhat is the research question posed in the paper? 1.1b Is it about the impact of a therapy, causality, evaluation of a diagnostic test or about the magnitude of a health problem?[2]	1.2Is this question relevant to the practice of diving or hyperbaric medicine?
2.1What is the study type?[3]	2.2Is the study type appropriate to the research question?	2.3If not, how useful are the results likely to be?
3.1aDefine the population in which the authors are interested. Are the study subjects representative of this population? 3.1bIf assigned to groups, how was this accomplished? 3.1c How many reached final follow-up?	3.2aAre there selection biases?[4] 3.2b Was allocation random? Was allocation made after a decision to enter the trial?	3.3aAny threat to external validity? Any threat to internal validity? 3.3b Any threat to internal validity? 3.3c Does this proportion threaten internal validity?
4.1What is being studied (study factor)? How is it measured?[5]	4.2Is there any likely measurement error (differential or non-differential)?[6]	4.3Is there any likely important cause of bias? (Beware differential error with Case-Control Studies).
5.1What outcomes are being assessed (outcome factors)?	5.2Any important outcomes missed? Any likely measurement error (differential or non-differential)?	5.3Do missed outcomes reduce the applicability of this study? Is there any likely source of bias?
6.1aWhat potential confounders are considered? 6.1b How were they dealt with?	6.2aAny important confounders missing? 6.2b Were they dealt with adequately, or subject to measurement error?	6.3How likely is confounding to be a significant source of bias?
7.1aIs a point estimate of effect given?[7] 7.1b Are confidence intervals given? If not in a study with statistically non-significant findings, is power given?	7.2aIs it reasonable to accept these results are not due to chance? 7.2b Are the differences reported clinically significant? Was the sample size sufficient to detect a clinically significant difference?	7.3Is this study useful or inconclusive in answering the research question?
8.1What are the authors’ conclusions?	8.2Have the authors correctly interpreted the results?	8.3Have the authors considered study limitations in their conclusions?

 

 

CRITICAL APPRAISAL WORKSHEET for Meta-analysis and systematic reviews[8]

 

Important information that should be in the paper	Potential related problems	Threats to the internal and external validity of the study
1.1aDoes the review address a focused clinical question? 1.1b Is it about the impact of a therapy, causality, evaluation of a diagnostic test or about the magnitude of a health problem?	1.2Does this question address the clinical problem?
2.1 Were the criteria used to select studies for inclusion appropriate?	2.2 Are the study types appropriate to the research question?	2.3 If not, how useful are the results likely to be?
3.1a Is it likely that important and relevant studies were missed? 3.1b Was the validity of the included studies appraised?	3.2a Could this alter the overall result?	3.3a Could this reduce the validity of the review conclusions? 3.3b Any threat to internal validity?
4.1 Are all clinically important outcomes considered?		4.3 Do any missed outcomes seriously affect the usefulness of this review?
5.1a What are the overall results of the review? 5.1b How precise are the results?	5.1a Are the results statistically significant? 5.2b Are the results clinically significant?
6.1 Can the results be applied to my patients?	6.2 Are the benefits worth the harms and costs?

 

 

NOTES TO ACCOMPANY THE WORKSHEET

1. Internal validity - how well is the study done within itself, are the methods used reliable and trustworthy so that we can believe the results? External validity - how applicable are these results (whether or not you believe them) to the kinds of situation or patients you would like to apply them to? For example, a diving equipment trial that enrolled Navy SEALS might not be directly applicable to a recreational diver).

2. These are the four main 'types' of question posed in papers. For the purposes of this wiki, nearly all papers are about the impact of a therapy or intervention of some sort (eg, a new breathing apparatus for diving).

3. If not a randomised trial of some type, then it is not relevant for this wiki.

4. Selection bias is not usually a problem with RCTs, but look for reassurances that any of the subjects could have gone into any of the arms of the study.

5. The study factor is simply the new drug or equipment or whatever is being assessed. For example, the Clarke study assesses HBOT for the treatment of radiation proctitis - so HBOT is the study factor.

6. Are you happy that they really gave whatever drug or treatment they said they did? Not usually a problem for us in this field.

7. The point estimate of effect is the measured difference between groups in the main outcome. For example, the difference in the proportion of ulcers healed in a study of chronic wounds.

8. This sheet is for meta-analyses rather than individual studies.

BACK