The Importance of Research Synthesis Methods
No single study settles anything. In psychology, education, health, and the social sciences, we now publish more research on any given question than one person could reasonably read in a career, and much of it appears to disagree. Research synthesis is how a field turns that sprawling pile of individual studies into cumulative knowledge that can actually inform theory, practice, and policy. For decades the default approach was the narrative review, often accompanied by an informal box score: five studies found a significant effect, three didn’t, so the evidence is “mixed.” The trouble is that vote counting throws away nearly everything that matters. It ignores the magnitude and precision of the effects; it ignores study quality and power; and it is sensitive to which studies the author happened to cite, leading to potentially misleading conclusions.
Contemporary best practice is to conduct a formal research synthesis consisting of two phases, each designed to increase the scientific rigor and validity of the results. The first phase is to conduct a systematic review of the literature including a pre-specified protocol, documented and reproducible search strategy, explicit inclusion and exclusion criteria, independent screening, structured coding of study characteristics, and a formal assessment of study quality and risk of bias. Done well, this phase is transparent enough that another researcher could repeat it and arrive at the same set of studies. The second phase is to perform a quantitative meta-analysis of study findings: converting results to a common effect size metric, fitting fixed- or random-effects models, quantifying and then explaining heterogeneity through moderator analyses, handling dependency among effect sizes drawn from the same study, and evaluating publication bias and small-study effects. Conducting a research synthesis with careful attention to each phase provides novel scientific insights not previously possible.
Because each phase of a research synthesis is critically important we we offer Systematic Review and Meta-Analysis as separate, complementary workshops, each designed and taught by Dr. Mariola Moeyaert, whose research program sits squarely at the intersection of evidence synthesis and applied methodology. Each class can be taken independently, for researchers just starting versus those arriving with a completed review in hand ready to embark on a meta-analysis. Even better, our Research Synthesis Workshop Bundle pairs the two courses at a reduced rate and walks you through the full arc, from framing your question to completing your meta-analysis. And because all of our workshops are evergreen, you always get lifetime access to the most recent versions of the class materials so you can be confident that you are incorporating the most cutting-edge methods in your own work.
Building a cumulative science requires more than a box score. Let CenterStat help.