Literature Appraisal

last authored:
last reviewed:

 

 

Introduction

Is this the same as evidence-based medicine? what can this section be called?

 

Effective health care relies on a solid knowledge and evaluation of existing research to make the best evidence-based decision whenever possible.

 

 

 

Assessing Methodological Quality

When analyzing a paper for its methological value, a number of questions may be considered:

 

1) Was the study original?

• is the study bigger, longer, or more substantial in other ways?
• is the population different than previous studies?

2) Who is the study about?

• how were subjects recruited?
• who was included?
• who was excluded?
• were subjects studied in 'real-life' circumstances?

3) Was the study design sensible?

• what intervention was being considered, and what was it being compared with?
• what outcome was measured, and how?

4) Was clinical bias avoided or minimized?

• groups should be as similar as possible, with the exception of the particular difference being measured
• selection bias, performance bias, exclusion bias, detection bias

5) Was assessment 'blind'?

 

6) Were preliminary statistical questions dealt with?

• sample size: a trial should be big enough to detect as statistically significant a worthwhile effect if it exists and be reasonably sure it doesn't exist if not found
• duration of follow-up: a study must be long enough to provide evidence of the intervention effect
• completeness of follow-up: subjects who withdraw are more likely to have experienced side-effects and missed check-ups and less likely to have followed trial protocol. Ignoring people who have withdrawn from a trial will bias results, usually in favour of the intervention. As such, standard practice is to analyze results on an 'intention-to-treat' basis

 

 

 

Study Validity

There are many things that can erode a study's validity.

• accuracy
• bias
• confounding
• standardization
• event modifiers

return to top

 

 

Understand Internal and External Validity

Internal validity refers to the validity within the study (did the researchers use valid methods to test the sample size within the experiment), whereas external validity is the degree to which the results of the experiment can be generalized to a larger population outside the sample used in the experiment.

Understand basics of bias and confounding and how to control each

Bias is a systematic error in a study that can lead to erroneous results (something is wrong with the study that skews the results away from the truth).

The main two categories of bias according to MacPherson are (although in reality there are many forms of bias that do not fit neatly in either category):

1. Selection bias: a bias that occurs in the way the sample was selected Example: if you attempted to uncover the average amount of beer consumed by Nova Scotia residents but only sent the survey out to students at Dalhousie – who may drink more than the average Nova Scotian.

2. Measurement bias: an error inherent in the method of measurement itself. Example: asking a person to tell you how many sex partners they’ve had in the past month – if it’s Steve may be inclined to play down the numbers so that Moeller doesn’t get jealous – this is called recall bias. Or if it’s Moeller interviewing Steve he may ignore evidence that Steve may have some action on the side, this is called interviewer bias.

To control for bias you must set up your experiment as best possible (this is mostly common sense). As a general rule, to avoid selection bias you randomize participants to different study groups. To avoid measurement bias use standardized and objective measurements.

Confounding variables (a.k.a. Confounder) are another source of bias. “Confounders” are a variable other than the variable being examined that could explain for all or part of the results. Example: An experiment may try and examine if there is a correlation between coffee drinking and lung cancer, and finds a high correlation. However, the confounding variable of smoking in fact explains the finding (i.e. many coffee drinkers smoke, and this is why many coffee drinkers end up with lung cancer).

Controlling for confounding variables is the same as bias (common sense): Randomize participants to treatments, and assess possible factors that could skew results if you cannot randomize completely (i.e. if you are looking over records you can’t go back in time and reassign a person to a different category).

Understand when direct and indirect standardization are used to compare rates

Standardization is used to remove confounding variables when comparing rates between two groups.

Direct standardization converts the sample population to the same configuration (usually for age) of some larger population, which thereby allows you to compare it to the larger population.

Indirect standardization is used when the sample populations rates are not stable or unknown. So the larger populations rates are reconfigured to match the sample populations configuration (usually for age)

Measures of Association

RR = Relative risk is the ration of the outcome risk is the exposed group as compared to the unexposed group (slide 4 in risks and rates continued)

OR = Odds Ratio (seen above)

RD = Risk difference (seen above)

NNT = 1/RD = Number needed to treat (seen above)

return to top

 

 

Resources and References

Sullivan I. et al, 2004. Framingham...

NHS Evidence - Health Information Resources

BMJ Qualitative Research papers -Tricia Greenalgh

return to top