Appraisal sheets

HOME INDEX SEARCH DATABASE WHAT IS A CAT? WHAT CAN YOU DO? INTRODUCTION HBO EXPLAINED  

The tables below are standardised critical appraisal sheets I use to help write the CATs on this site. I find that the answers to each of these questions helps me to avoid missing a fatal flaw in the paper, and directly assists the correct information getting into the final CAT.

 

HBO EVIDENCE CRITICAL APPRAISAL WORKSHEET

 

 

Important information that should be in the paper	Potential related problems	Threats to the internal and external validity of the study[1]
1.1a What is the research question posed in the paper? 1.1b Is it about the impact of a therapy, causality, evaluation of a diagnostic test or about the magnitude of a health problem?[2]	1.2 Is this question relevant to the practice of diving or hyperbaric medicine?
2.1 What is the study type?[3]	2.2 Is the study type appropriate to the research question?	2.3 If not, how useful are the results likely to be?
3.1a Define the population in which the authors are interested. Are the study subjects representative of this population? 3.1b If assigned to groups, how was this accomplished? 3.1c How many reached final follow-up?	3.2a Are there selection biases?[4] 3.2b Was allocation random? Was allocation made after a decision to enter the trial?	3.3a Any threat to external validity? Any threat to internal validity? 3.3b Any threat to internal validity? 3.3c Does this proportion threaten internal validity?
4.1 What is being studied (study factor)? How is it measured?[5]	4.2 Is there any likely measurement error (differential or non-differential)?[6]	4.3 Is there any likely important cause of bias? (Beware differential error with Case-Control Studies).
5.1 What outcomes are being assessed (outcome factors)?	5.2 Any important outcomes missed? Any likely measurement error (differential or non-differential)?	5.3 Do missed outcomes reduce the applicability of this study? Is there any likely source of bias?
6.1a What potential confounders are considered? 6.1b How were they dealt with?	6.2a Any important confounders missing? 6.2b Were they dealt with adequately, or subject to measurement error?	6.3 How likely is confounding to be a significant source of bias?
7.1a Is a point estimate of effect given?[7] 7.1b Are confidence intervals given? If not in a study with statistically non-significant findings, is power given?	7.2a Is it reasonable to accept these results are not due to chance? 7.2b Are the differences reported clinically significant? Was the sample size sufficient to detect a clinically significant difference?	7.3 Is this study useful or inconclusive in answering the research question?
8.1 What are the authors’ conclusions?	8.2 Have the authors correctly interpreted the results?	8.3 Have the authors considered study limitations in their conclusions?

 

 

CRITICAL APPRAISAL WORKSHEET for Meta-analysis and systematic reviews[8]

 

Important information that should be in the paper	Potential related problems	Threats to the internal and external validity of the study
1.1a Does the review address a focused clinical question? 1.1b Is it about the impact of a therapy, causality, evaluation of a diagnostic test or about the magnitude of a health problem?	1.2 Does this question address the clinical problem?
2.1 Were the criteria used to select studies for inclusion appropriate?	2.2 Are the study types appropriate to the research question?	2.3 If not, how useful are the results likely to be?
3.1a Is it likely that important and relevant studies were missed? 3.1b Was the validity of the included studies appraised?	3.2a Could this alter the overall result?	3.3a Could this reduce the validity of the review conclusions? 3.3b Any threat to internal validity?
4.1 Are all clinically important outcomes considered?		4.3 Do any missed outcomes seriously affect the usefulness of this review?
5.1a What are the overall results of the review? 5.1b How precise are the results?	5.1a Are the results statistically significant? 5.2b Are the results clinically significant?
6.1 Can the results be applied to my patients?	6.2 Are the benefits worth the harms and costs?

 

NOTES TO ACCOMPANY THE WORKSHEET

1. Internal validity - how well is the study done within itself, are the methods used reliable and trustworthy so that we can believe the results? External validity - how applicable are these results (whether or not you believe them) to the kinds of situation or patients you would like to apply them to? For example, a diving equipment trial that enrolled Navy SEALS might not be directly applicable to a recreational diver).

2. These are the four main 'types' of question posed in papers. For the purposes of this wiki, nearly all papers are about the impact of a therapy or intervention of some sort (eg, a new breathing apparatus for diving).

3. If not a randomsied trial of some type, then it is not relevant for this wiki.

4. Selection bias is not usually a problem with RCTs, but look for reassurances that any of the subjects could have gone into any of the arms of the study.

5. The study factor is simply the new drug or equipment or whatever is being assessed. For example, the Clarke study assesses HBOT for the treatment of radiation proctitis - so HBOT is the study factor.

6. Are you happy that they really gave whatever drug or treatment they said they did? Not usually a problem for us in this field.

7. The point estimate of effect is the measured difference between groups in the main outcome. For example, the difference in the proportion of ulcers healed in a study of chronic wounds.

8. This sheet is for meta-analyses rather than individual studies.

BACK