Meta-analysis
 

aim of science = weave together understanding of world from different studies

 

• replication = bedrock of science - can same result be observed by different researchers?

reviewing research to cumulate knowledge

 

• summarize/synthesize what is known about some research question

• degree of relationships/ differences, estimates of population parameters, variability across studies

• plan next steps in research

approaches to reviewing research

 

• narrative literature review

• reviewer gives verbal, somewhat subjective summary and appraisal of studies

• vote counting review

• counting up how many studies showed statistically significant results

• if all/nearly show significance, then conclude some relationship/difference

• if some significant and some not,  then conclude results are mixed or conflicting

• if nearly all nonsignificant, then conclude no relationship/difference

• level of hyperactivity in childhood and violence in adolescence/young adulthood (Hawkins et al.)
• 4 longitudinal studies

study	n	r	p
A	61	+.17	.19
B	44	+.22	.15
C	189	+.12	.10
D	725	+.13	< .001

• meta-analysis

• goal: summarize results across studies quantitatively

• focus on magnitude of difference/relationship/association ("effect size")

• statistical significance irrelevant

• procedures

• select studies for inclusion - whether published, substantively meaningful distinctions, quality

• hyperactivity & violence - PsycInfo, MEDLINE, etc. search, bibliographies of articles

• compare or combine results from different studies

• decide whether studies similar enough to combine

• meta-analysis not pooling all of the data

• rather, analysis of separate studies' results

• e.g., what is the average RR, correlation, etc. in these studies?

• e.g., which type of studies have a higher or lower RR, correlation, mean value, etc.?

weighting by sample size common

• hyperactivity & violence -

• weighted mean r for all 4 studies = .13

• median r = .15

 
• meta-analysis shows markedly different conclusion than vote- counting approach

• studies show level of childhood hyperactivity consistenty but weakly associated with youth violence

• can't conclude hyperactivity causes violence - only observational evidence

• other points

 
• interpretation of meta-analytic summaries similar to interpretation of original studies - research design

 

• summary of observational studies - cannot make causal conclusions

• retrospective vs. prospective (cross-sectional vs. longitudinal)

 

• factors related to opiate addicts' continued opiate use during/after treatment (Brewer et al.)

• how strongly is patients' cocaine use associated with relapse to opiates?

• retrospective - weighted mean r = .28 (4 studies)

• prospective - weighted mean r = -.03 (3 studies)

 

• summary of experimental studies - can make causal inferences

• depression causes continued use?

• three experiments - opiate addicts randomly assigned to psychotherapy or no psychotherapy

all showed that psychotherapy reduced patients' depression

• all experiments showed psychotherapy increased opiate use

• thus, depression not causally linked to continued opiate use

 

• meta-analytic summary consistent: depression very weakly correlated with continued opiate use (weighted mean r = .10)

criticisms of meta-analysis

 

 
 
 
 
 
 

• including biased or flawed original studies - can use screening criteria to exclude or weight/stratify results by study quality

 

• file drawer problem - not including unpublished studies