Measurement is far from a new problem. Indeed, psychology, education, and related fields have developed a rich tradition of psychometric thinking about construct validity, dimensionality, reliability, score precision, item functioning, measurement invariance, and test bias that spans more than a century. Yet in much contemporary substantive research, measurement is treated at most as a brief preliminary hurdle rather than an ongoing scientific responsibility, either ignored entirely or addressed merely through the routine reporting of Cronbach’s alpha (which hardly counts). Structural validity evidence in particular is often underreported and researchers frequently rely on prior use of a scale as justification for current use, which is far from sufficient. Despite the extraordinary knowledge that exists about measurement, researchers rarely make full use of rigorous psychometric tools in their day-to-day modeling practice, and this in turn can drastically limit what can be learned from our data.

Validity and Reliability

Two key issues at hand are validity, the extent to which our measures actually reflect their intended constructs, and reliability, the extent to which our measures capture true construct variance relative to noise, or measurement error.  Nearly all statistical models proceed under the assumption that the observed measures are both valid and perfectly reliable (error free). The consequence of using invalid measures is relatively intuitive to work out. If a variable doesn’t actually represent the construct you think it does (e.g., a measure thought to represent impulsivity actually captures risk tolerance), then the obtained results obviously won’t provide accurate information about the intended construct. The consequences of unreliability are equally troubling: estimates obtained from a model assuming perfect reliability will be biased in the presence of measurement error, sometimes quite badly. In ordinary least squares regression, for example, predictors are typically treated as fixed and error-free for purposes of estimation. Absent this, the coefficient estimates will be biased.

The assumption of perfect reliability generalizes to a host of other statistical models ranging from mixture models to growth curve models to machine learning and beyond. But our predictors are often scale scores derived from a set of items, or even a single item, that can contain substantial measurement error. If present but ignored the analysis is no longer operating on the construct itself; it is operating on an imperfect proxy for the construct. The simple textbook story is that unreliability causes the coefficient estimates to shrink, that is, we obtain downwardly biased estimates pulled toward zero. Sometimes, this problem is dismissed with the justification that results are therefore just “conservative” because the true effects are actually even bigger. However, this textbook story only holds under the simplified scenario of model with just one predictor. With multiple predictors (so every single model fit in the real world), bias due to unreliability can be much more complicated and propagate throughout the entire model to not only attenuate but sometimes inflate estimates and standard errors.

Thus, the credibility of the entire scientific enterprise hinges on rigorous construct measurement. Fortunately, we have two extremely powerful and well-developed psychometric frameworks within which to assess measurement: factor analysis and item response theory (IRT). Both are latent variable models, but they tend to differ in their emphasis.

Factor Analysis

Item Response Theory

Whereas factor analysis is commonly applied with continuous response variables (frequently scale-level data), item-response theory (IRT) adds another layer of insight by focusing on the creation of scales from individual items. With item-level data, the response options are typically categorical (e.g., yes/no, correct/incorrect, never/sometimes/often or strongly disagree to strongly agree), necessitating models that consider how the probability of each response varies over the range of the latent trait (see below). For instance, the probability of answering an algebra item correctly should increase with math ability. Whereas factor analysis prioritizes defining the construct space by considering multiple factors, IRT is designed for the process of item selection and scale construction for a single underlying factor. IRT models thus allow researchers to evaluate which items discriminate most sharply, whether guessing is present, where along the latent continuum items are most informative, and how much measurement precision the test provides at different trait levels. IRT is a powerful yet often underused approach for measurement, especially because it yields information about both item functioning and score precision. This can be a major advantage over simply adding responses together and treating the result as if every point on the scale were equally reliable.

Connections Between Factor Analysis and IRT

Though these two psychometric frameworks developed side-by-side for somewhat different purposes, they are in fact highly related. A conventional factor analysis model can be considered an item response theory model for continuous items, and can likewise be used principally for scale construction purposes. Conversely, the IRT model can be considered a factor analysis model generalized to discrete items, with multidimensional IRT models offering a similar focus on structural validity. Knowledge of both factor analysis and IRT and their interconnections therefore offers both a rich understanding and wealth of tools for rigorously evaluating construct measurement.

Measurement in Practice

None of this means every study needs to begin with a full-scale psychometric redevelopment project. Researchers should, however, be much more cautious about treating common scales as if they were transparent windows onto the constructs they are intended to measure. A scale that worked well in one sample, at one time, for one purpose is not automatically adequate in another setting. Good measurement practice is therefore not a one-time citation to an old validation study. It is an ongoing process of testing whether the present data support the interpretation we want to make. For social, behavioral and health scientists, that is the larger message: measurement is not a nuisance to be dispatched before the “real” modeling begins. It is a fundamental part of the “real” modeling.

CenterStat is Here to Help

Here at CenterStat, we are deeply committed to measurement in all forms; indeed, both Dan and Patrick hold faculty positions in the L.L. Thurstone Psychometric Lab at the University of North Carolina, and Thurstone was arguably one of the greatest measurement experts to have ever lived. Measurement plays a key role within many of our workshops, but we offer two training opportunities that explicitly focus on these topics. The first is taught by Wes Bonifay (University of Missouri) and is titled Foundations of Item Response Theory. The second is co-taught by Patrick Curran (University of North Carolina) and Greg Hancock (University of Maryland) and is titled Exploratory and Confirmatory Factor Analysis. Each of these classes provides a broad treatment of IRT and FA starting from a basic introduction and moving to powerful contemporary applications of these models in practice. Each can be accessed separately, but we also offer a deeply discounted tuition on a measurement bundle that includes both classes.

Regardless of whether you learn measurement modeling from us or from someone else, it is absolutely critical that measurement be treated with the deep respect it deserves in every published study. This in turn increases the reliability, validity, and reproducibility of findings that we all desire.

Suggested Readings

Anastasi, A. (1950). The concept of validity in the interpretation of test scores. Educational and Psychological Measurement, 10, 67-78

Bandalos, D. L. (2018). Measurement theory and applications for the social sciences. Guilford Publications.

Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53, 605-634.

Bock, D. (1997). A brief history of item response theory. Educational Measurement: Issues and Practice, 16, 21-33.

Bollen, K. A., & Bauldry, S. (2011). Three Cs in measurement models: Causal indicators, composite indicators, and covariates. Psychological Methods, 16, 265-284.

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281-301.

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. New York: Psychology Press.

Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999). Evaluating the use of exploratory factor analysis in psychological research. Psychological Methods, 4, 272–299.

Hambleton, R. K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 147–200).

Jöreskog, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183-202.

Novick, M. R. (1966) The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3, 1-18.

Reise, S. P., & Waller, N. G. (2003). How many IRT parameters does it take to model psychopathology items? Psychological Methods, 8, 164–184.

Sellbom, M., & Tellegen, A. (2019). Factor analysis in psychological assessment research: Common pitfalls and recommendations. Psychological Assessment, 31, 1428–1441.

Smith, G. T. (2005). On construct validity: Issues of method and measurement. Psychological Assessment, 17, 396-408

Spearman, C. (1904). “General intelligence”, objectively determined and measured. American Journal of Psychology, 15, 201-293.

Thissen, D. E., & Wainer, H. E. (2001). Test scoring. Lawrence Erlbaum Associates Publishers.

Yong, A. G., & Pearce, S. (2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in quantitative methods for psychology, 9, 79-94.

The post Why Measurement Matters appeared first on CenterStat.

Welcome. Dan Bauer and Patrick Curran here. We are the founders of CenterStat, and we want to take a few minutes to briefly describe to you our unique strengths as a provider of quantitative and qualitative methods training. As you have undoubtedly noticed, an ever-increasing number of online businesses will happily take your money in exchange for training in statistics, methods, and research design. CenterStat is one of them. However, we believe that there a number of features of our training program that makes us unique among our peers. We’d like to highlight a few of these with you.

A Pop-and-Pop Outfit

First, instead of a faceless online company, CenterStat is an old-school pop-and-pop outfit founded by the two of us 15 years ago. We first became close friends a quarter of a century ago, and as time unfolded we both joined the L.L. Thurstone Psychometric Lab as faculty members of the Department of Psychology and Neuroscience at the University of North Carolina. We started teaching together by offering a single class in the questionable opulence of a Hampton Inn ballroom, with complimentary cold coffee and green bananas, and from there we’ve grown into what CenterStat is today. From our very first class to today, our training program has been designed and run by the two of us alone. We don’t relegate important tasks to AI, and we make every major decision together. You can also reach out to us directly. Do you have a question about the content of a class you took? We will respond to your email. Do you want to discuss options for on-site training? We will meet with you via Zoom. We are deeply committed to teaching and mentorship and simply refuse to allow an LLM to run our business.

A Deliberately Curated Curriculum

Second, we’re intimately involved in all the workshops that CenterStat provides, selecting and developing these with intentionality rather than simply platforming a hodgepodge of classes with unclear distinctions or organization. For the past two decades, one or the other of us served as Director of the Quantitative Psychology Graduate Program at UNC (curiously making each of us the other’s boss). A key responsibility of this position was to develop a curriculum in quantitative methods and research design for all student of our own department as well as students in allied fields. We have drawn on this extensive experience to create a similarly integrated curriculum at CenterStat. Whereas some providers offer a laundry list of dozens if not hundreds of alphabetized classes taught by anyone willing to upload a video, we instead curate classes from internationally recognized leaders in their fields within five integrated, curricular pathways: foundational topics, measurement and latent variables, longitudinal design and analysis, data science and machine learning, and qualitative research methods. Classes within each pathway are inter-connected with one another, enabling participants to build comprehensive knowledge in each area. Indeed, classes can be chosen to mimic the level of training one would receive over multiple years in a doctoral program with the highest quality course offerings available in the world.

The Highest Quality Materials Available

Third, our workshops consistently deliver the highest quality materials available anywhere, designed to follow best practices in pedagogy so that you gain both a deep understanding of the underlying principles of the methodology and the ability to confidently apply it in your own work. We want you to understand what you’re doing, not just how to do it (potentially incorrectly) in software program X, Y, or Z (or even R!). For all courses, we provide detailed notes to accompany instructional videos, both for lectures and software demonstrations, as well as code and data files for all examples, often in multiple software programs. Our materials follow a consistent format across classes, ensuring that you know exactly what you will receive when purchasing a new class. Check out some examples here. Many other organizations provide little to no quality control, leaving instructors to their own devices, with the result that classes differ markedly from one another in level and quality and may be either excessively software-centric or inexplicably software free.

Your Access Never Expires and Your Materials are Never Out of Date

Our Commitment to Price Transparency and Lowering Barriers to Access

Finally, we are deeply committed to maintaining a transparent pricing structure that is clearly defined and consistent across classes and instructors and aspires to lower barriers to access where possible. Some online training providers seem to go out of their way to intentionally obfuscate precisely what you are getting for your money. At CenterStat, every course is priced at precisely the same rate per hour of lecture time, and we provide a single summary table of the length and cost of every workshop we offer. Further, we are dedicated to maximizing the access of high-quality quantitative training to as broad an audience as possible and offer significant discounts to current students, as well as students and researchers working and residing in low and lower-middle income countries. And for everyone else, you can join our email list to receive occasional further discounts and promotions. We still need to pay the bills to keep the servers on, but we aspire to work towards disseminating high-quality online training to as broad a global audience as possible.

Our Commitment to You

We sincerely respect anyone who is working to help others improve the quality and replicability of our science and we are glad our competitors exist. At the same time, it is clear that not all online experiences are equal. We founded CenterStat with the commitment of offering the best possible training experience possible at the most affordable price, and we believe that we have achieved this goal. We hope you do as well.

The post What Makes Us Unique: A Message from Dan & Patrick appeared first on CenterStat.

Why LatentGOLD? First, it is a versatile latent variable modeling software program that can be used to estimate all of the models considered in the class, so why not? Second, not long ago we added an entire chapter to the workshop on approaches for robustly examining relationships between latent classes and external variables (e.g., predictors of class membership or distal outcomes predicted by class membership), and these approaches are exceptionally well implemented in LatentGOLD. This comes as no suprise given the primary developer of LatentGOLD, Jeroen Vermunt, also played a key role in the innovation of these methods. Third, Statistical Innovations Europe now offers a free academic license to LatentGOLD and they have partnered with us to provide a 90-day free license to non-academic workshop participants as well.

The LatentGOLD demonstrations complement prior demonstrations in Mplus and R, providing users with a variety of choices when fitting finite mixture models, including applications of latent profile analysis and latent class analysis.

If you previously enrolled in the class, then you have instant access to these materials as a benefit of our evergreen content model. Otherwise, enroll now to take advantage of this added software flexibility!

The post LatentGOLD demonstrations now included in Mixture Modeling and Latent Class Analysis workshop appeared first on CenterStat.

For decades, we have been able to expand growth models to include time-specific (and even lagged) effects for predictors that themselves change over time, often called time-varying covariates (or TVCs). TVC models allow us to transition to testing within-person hypotheses because the model does not just ask whether people change but whether within-person fluctuations in predictors are related to within-person deviations in the outcome. This is our focus here.

A Brief Review of Growth Modeling

Growth modeling is a massive topic that cannot be fully covered here. However, we have prior blog posts, a free online lecture series on different approaches to growth modeling, and we offer several full-length workshops on growth modeling using multilevel models (MLMs), structural equations models (SEMs), and models for intensive longitudinal data (ILD).

Briefly, the core motivating goal of growth modeling, whether estimated using the MLM or the SEM, is to summarize a set of repeated measures with a smoothed underlying trajectory that parsimoniously represents how the outcome changes over time. Individual trajectories can obtain a variety of functional forms (e.g., linear, quadratic, piecewise, exponential, etc.) or even display no systematic change over time (and thus necessitate only an intercept term).

A key outcome of the model is the estimation of the means (sometimes called fixed effects) and the variances (sometimes call random effects) of each defined growth component. For example, say we fit a linear trajectory to six repeated measures of aggressive behavior in children. We could obtain a mean starting point and rate of change reflecting the average trajectory for the full sample, and a variance in starting point and rate of change (and covariance between the two) reflecting child-to-child variability in the trajectory parameters. This model may be of theoretical interest on its own, but we often expand this model to include one or more covariates of theoretical interest. There are two types of covariates that are commonly considered.

Time-invariant vs. Time-varying Covariates

Time-invariant covariates (TICs) are assumed to have a constant value across measurement occasions. Examples include sex assigned at birth, baseline SES, treatment condition, or childhood adversity at the start of kindergarten. In growth models, TICs typically predict differences in trajectory parameters and assess questions such as does baseline SES predict the intercept (i.e., starting point) or does treatment condition predict the slope (i.e., rate of change)? An example finding might indicate that there are no treatment group differences in depression at baseline but that, on average, those who received treatment reported a significant decrement in depressive symptomatology over time compared to those who did not. An exemplar path diagram of a latent curve model with two TICs (x₁ and x₂) predicting trajectory intercepts and slopes (η₁ and η₂) for five repeated measures (y₁–y₅) is:

In contrast, time-varying covariates (TVCs) have the potential to differ in value at each measurement occasion. This is a critical distinction from TICs. Whereas baseline SES is by definition invariant to the passage of time, other constructs, such as daily stress, nightly sleep, current caregiver status, or weekly anxiety, change from one time point to the next. Whereas TICs predict the growth factors themselves (e.g., we regress the intercept and slope trajectory components on treatment group membership), TVCs directly predict the time-specific repeated assessments above-and-beyond the underlying growth process. Although a seemingly modest change to the structure of the model, this actually has profound implications for the types of questions we can ask, particularly those focused on within-person process. An exemplar path diagram of a latent curve model with a TVC (z₁–z₅ measured at time 1 to 5) is:

Note that TVCs can have both within- and between-person effects.  For example, suppose we wish to evaluate the relation between substance use and anxiety over a set of repeated measures.  The between-person effect would reflect average differences between people: Do people who report higher average levels of anxiety over time also report higher average levels of substance use over time? This contrasts from the within-person effect: When a person is more anxious than usual at a given time point do they tend to engage in more substance use than they typically do? Often with TVCs we are particularly interested in modeling these within-person processes, which can be extended to examine lead-lag relations, and dynamic systems in ways not possible with a standard TIC model.

In the spirit that you don’t get something for nothing, inclusion of TVCs introduces natural complexities that must be addressed, particularly in the management of the data, specification of the model, and substantive interpretation of the findings. However, these issues are all well understood and one needs to simply turn to existing resources that illustrate best practices.

Incorporating TVCs into Growth Models

One point of common confusion is how within- and between-person effects of TVCs are estimated and interpreted within the SEM versus MLM frameworks. Within the SEM growth model, the TVCs are incorporated in their raw metric as exogenous predictors of the repeated measures. Because the TVCs freely covary with the latent growth factors (or at least should; be sure your software package is estimating these relations), the resulting regression parameters represent pure estimates of within-person effects. The standard SEM specification does not provide pure estimates of between-person effects of TVCs; however, these can be obtained by expanding the model in particular ways (see Curran et al., 2012, for details).

In contrast, if the TVCs are incorporated into the MLM in their raw metric, the resulting regression coefficients inextricably combine the within-person and between-person effects (see Raudenbush & Bryk, 2002, for details). A well-known solution to this problem is to person-mean center, that is subtract the person mean of the TVC from each observed value of the TVC, and then use the person-mean-centered TVC as the predictor at Level 1; this captures the same pure within-person effect estimate as the SEM. However, the person mean of the TVC itself can also be incorporated as an additional predictor at Level 2 of the model to capture the between-effect.

Thus, under a broad set of conditions, the MLM and SEM will provide precisely the same (or nearly the same) estimates of the within and between effects of TVCs, even though this is accomplished in quite different ways.

Limitations

Both the SEM and MLM approaches to modeling TVCs include a number of assumptions, but two are of most interest here. First, using either of the methods described above, we make a fundamental assumption that although the distal outcome may be growing systematically over time, the TVC most decidedly is not. Think about this logically: we deviate the person mean from each measured TVC and the person mean is by definition constant over time. In other words, there is no growth trajectory for the TVC. Significant estimation and interpretation problems can be encountered if the TVC itself is growing over time and this is not adequately represented in the model (see Curran & Bauer, 2011, for detailed examples, and Wang & Maxwell, 2015, for exceptions). To be clear, methods exist for handling such conditions, but additional work is necessary to represent systematic co-occurring individual change in the model (Curran et al., 2014).

The second assumption relates to the strength of relation between the TVC and the outcome over time. Consider three possible scenarios: (1) the magnitude of the relation is equal at all time points; (2) the magnitude of the relation systematically increases or decreases with the passage of time; or (3) the magnitude of the relation obtains a unique value at every single time point. All three of these can be incorporated in both the SEM and the MLM, and formal tests are available to determine which condition best represents the sample data at hand. However, although these methods offer powerful tests of the nature of the relation between the TVC and the outcome, there is a well-developed yet little used method that provides more flexible insight into these over-time relations, and this is called the time-varying effects model, or TVEM.

Time-Varying Effects Models

Time-varying effects models offer a semi-parametric approach to the more traditional TVC model within the MLM framework. Whereas the MLM provides formal tests of structured relations between the TVC and the outcome (e.g., a bilinear “product” interaction between the TVC and time), the TVEM uses a spline method of estimation to approximate complex nonlinear relations that might wax and wane in magnitude over time. Confidence regions can then be plotted to show “sensitive periods” during which a TVC is significantly related to the outcome versus periods when it is not. These relations are primarily graphical in nature and lovely plots can be created that demonstrate the potentially complex nature of the relation between the TVC and the outcome as a function of time. The TVEM offers many advantages when trying to understand complex relations over time, yet these methods remain quite underutilized in practice. See Lanza and Linden-Carmichael (2021) for a thorough introduction to the estimation and interpretation of TVEMs within the social sciences.

Conclusion

In sum, time-varying covariate growth models allow for the introduction of powerful tests of within-person dynamics over time that are not present in a more standard TIC. These methods are well developed and widely used and can easily be incorporated in your own data applications. A variety of resources exist, and we cite several of these below. CenterStat also offers full workshops that provide detailed instruction on TVC models, including Multilevel Models for Longitudinal Data, Longitudinal Structural Equation Modeling, and Analysis of Intensive Longitudinal Data. Including TVCs in your models requires care but can also yield important new insights.

Suggested Readings

Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583-619.

Curran, P. J., Howard, A. L., Bainter, S. A., Lane, S. T., & McGinley, J. S. (2014). The separation of between-person and within-person components of individual change over time: a latent curve model with structured residuals. Journal of consulting and clinical psychology, 82(5), 879.

Curran, P.J., Lee, T.H., Howard, A.H., Lane, S.T., & MacCallum, R.C. (2012). Disaggregating within-person and between-person effects in multilevel and structural equation growth models. In: Hancock, G., editor. Advances in longitudinal methods in the social and behavioral sciences. Charlotte, NC: Information Age; p. 217-253.

Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. (2015). A critique of the cross-lagged panel model. Psychological Methods, 20(1), 102.

Lanza, S. T., & Linden-Carmichael, A. N. (2021). Time-varying effect modeling for the behavioral, social, and health sciences. Springer.

Raudenbush, S.W. & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods. Advanced Quantitative Techniques in the Social Sciences Series/SAGE.

Shiyko, M.P., Burkhalter, J., Li, R., & Park, B. J. (2014). Modeling nonlinear time-dependent treatment effects: an application of the generalized time-varying effect model (TVEM). Journal of Consulting and Clinical Psychology, 82, 760.

Wang L.P. & Maxwell S.E. (2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychological Methods, 20, 63-83.

The post The Strengths and Limitations of Time-Varying Covariate Growth Models appeared first on CenterStat.

Parametric Inference and the Sampling Distribution

In classical statistics, inference depends on specifying a model for the data-generating process and the idea of repeated sampling. For instance, suppose we take a sample of size N and calculate the sample mean. Ultimately, we’d like to make a conclusion about the mean in the population from which we drew our sample. However, we need to account for sampling error. That is, had we drawn a different sample of the same size, we would have obtained a different sample mean. In fact, there are infinitely many different samples of size N that we might have drawn from our population, each of which would yield a somewhat different mean, simply depending on who happened to get into the sample. The collection of means across all of these hypothetical samples has its own distribution, known as a sampling distribution. The sampling distribution reflects our uncertainty in the estimation of the population mean (i.e., variation due to sampling error) and it underpins confidence intervals, hypothesis tests, and other inferential procedures.

For instance, under the assumption that the we draw independent and identically distributed observations from a normal distribution (or by appealing to the Central Limit Theorem at large sample sizes), the sample mean follows a normal distribution with mean equal to the population mean and standard error equal to the standard deviation divided by the square root of N. If the standard deviation for the population is known, then we can simply divide our sample mean by the standard error and reference this to the standard normal distribution (z-distribution) to make inferences. More typically, however, we aren’t privy to this knowledge and must plug in our sample standard deviation to get the standard error. The sampling distribution of the mean will then deviate from the normal curve due to this added uncertainty in small samples. Thanks to our favorite Guinness brewer William Gossett (who published under “Student”), we know that in such cases, the sample mean divided by the standard error follows a t-distribution, a bell-shaped distribution whose tail thickness is determined by the degrees-of-freedom.

So far, we have focused on the sample mean, but we can imagine a sampling distribution for any parameter we wish to estimate, whether it be a regression coefficient, variance, factor loading, or any other value of interest. The challenge, however, is that in real research we often find ourselves making uncertain parametric assumptions in order to obtain a known sampling distribution. For example, when using a t-distribution to make inferences about a regression coefficient, we assume a sufficiently large sample size and that the errors are normally distributed, independent, and homoscedastic, conditions that may or may not hold in practice. When assumptions are met, such parametric procedures work exceedingly well; when not met, the resulting inferences can be both biased and misleading, sometimes markedly so.

What Is the Bootstrap?

The non-parametric bootstrap, first formally proposed by Bradley Efron in 1979, is a computational technique for empirically approximating the sampling distribution without the requirement of strong parametric assumptions. Instead of relying on mathematical formulas, the bootstrap uses the observed data itself as a stand-in for the population. (Thus, the term “bootstrap” which is drawn from the phrase draw yourself up by your own bootstraps meaning you take personal responsibility and use the resources you have available to you). The basic procedure is conceptually simple. First, you draw your sample of size N from the population in the usual way. Next, you draw a “bootstrap sample” also of size N from the original data with replacement. Then you compute and retain your statistic of interest on the bootstrap sample in whatever way you please (e.g., mean, regression coefficient, mediated effect). Finally, you repeat this process many times (often with 1000 bootstrap samples or more) to create an empirical distribution for the statistic from which to make inferences back to the population.

The critical step to understand is that we are randomly drawing a bootstrap sample from our original sample data with replacement. Say we have a sample of N=100 observations; we would draw say 1000 bootstrap samples of size 100 where in each one a given observation may appear repeatedly or not at all (thus the sampling “with replacement”). This empirical distribution of the estimate (whatever that might be) under very general conditions approximates the sampling distribution. As such, we can use this to compute standard errors, confidence intervals, and bias estimates in similar ways to that of the parametric sampling distribution but with far fewer assumptions about the population. For instance, the bootstrapped standard error is simply the standard deviation of the bootstrapped sample estimates. And a bootstrapped confidence interval can be computed by simply locating the 2.5^th and 97.5^th percentiles of the bootstrapped sample estimates, values that may or may not be symmetric around the estimate (in contrast to the symmetry assumed by parametric z– or t-type confidence intervals). The beauty of the bootstrap lies in how it transforms a theoretical problem (deriving the sampling distribution) into a computational one.

Typical Applications

The bootstrap has found applications across nearly every domain of research. A classic example in the social sciences relates to the testing of indirect effects in mediation models, path analysis, and structural equation modeling. Such an effect arises within a causal chain within which one variable affects another which in turn affects a third (with more complex chains also being possible). Each effect within the causal chain is represented by its own regression coefficient. Under standard assumptions, and in large samples, each regression coefficient estimate will have a normal sampling distribution, allowing for the usual inferences. However, we don’t want to test each link in the chain individually; we want to test the chain as a whole. That is, we want to test the indirect effect of the initial predictor on the final outcome as transmitted through the intervening variables (or mediators). The sample estimate of an indirect effect is obtained by computing the product of the regression coefficients involved in the chain. Easy enough. To test the indirect effect, however, we need to know its sampling distribution, and that’s where things get tricky. Each link in the chain has a normal sampling distribution, but a product of normal variates is generally not itself normally distributed. Using a normal sampling distribution as an approximation (the delta-method or “Sobel method” for testing indirect effects) is convenient but often leads to biased inferences. This is a well-known problem that has sparked a variety of solutions, one of which is to derive the correct parametric distribution for the indirect effect (known as the “distribution of the product” method). More commonly, however, investigators have turned to the non-parametric bootstrap to obtain empirically-based inferential tests that do not rely on parametric assumptions at all. Indeed, bootstrapping is now the gold standard for testing mediated effects in practice.

Regardless of whether one is evaluating an indirect effect, a variance estimate, or any sample estimate of interest, there are many potential uses of the bootstrap results. For example, we can estimate standard errors, particularly when no simple formula exists (e.g., in complex nonlinear models). Similarly, we can compute confidence intervals using several different bootstrap methods (percentile, bias-corrected, accelerated) that provide intervals with better coverage properties than parametric ones in certain settings. Further, in machine learning and predictive modeling, bootstrap samples can be used for model validation and cross-validation and to estimate prediction error. Finally, in fields where data collection is difficult or expensive, such as clinical trials, educational experiments, or niche social science surveys, the bootstrap offers a way to make inference with small samples and limited data. These are just a few examples of how the bootstrap can be used in practice, and many additional options are available.

Advantages of the Bootstrap

There are many advantages to the bootstrap. Unlike traditional parametric methods, the non-parametric bootstrap does not require specifying a functional form for the population distribution. This makes it attractive when normality or homoscedasticity is questionable. The method also works for a wide range of statistics including means, medians, regression coefficients, correlation coefficients, or even more complex estimands like Gini coefficients. At its core, the bootstrap is easy to explain and implement. With modern software (R, Stata, SAS, Python), the procedure often requires just a few lines of code. The bootstrap can be extended for clustered data, time series, or hierarchical designs (e.g., students nested within classrooms), making it useful in applied social science and education research. As a general method, the bootstrap is remarkably flexible and can be applied in many interesting and challenging research scenarios.

Disadvantages and Limitations

As with any procedure, there are also disadvantages that must be considered. Because the bootstrap treats the sample as a proxy for the population, any biases in the sample will propagate through the bootstrap distribution. In small or unrepresentative samples, the bootstrap may give misleading results. Although less of an issue today, the bootstrap can be computationally intensive, especially for large datasets or complex models. Thousands of resamples are often needed for stable estimates. The bootstrap may also perform poorly for statistics that depend heavily on the tails of the distribution (e.g., extreme quantiles, maximum values), because the resampled datasets cannot create values outside the observed range. In clustered or dependent data structures, naïve bootstrapping can underestimate variability unless modified (e.g., block bootstrap, cluster bootstrap). This is particularly relevant in education research, where students are not independent observations, or repeated measures applications, where observations are correlated over time within persons. Care must be taken when evaluating the potential use of the bootstrap procedure in practice given the associated limitations.

Conclusion

The bootstrap represents one of the great innovations in modern statistics: a method that converts inference from an algebraic to a computational problem. By resampling from the observed data, researchers can approximate the sampling distribution of almost any statistic, gaining access to standard errors, confidence intervals, and bias estimates without heavy reliance on parametric formulas. Its strengths (flexibility, fewer assumptions, and ease of implementation) make it a powerful tool, especially in social sciences and education where data are often messy, distributions non-normal, and sample sizes modest. Yet, it is not a panacea: bootstrap inference depends on sample representativeness, can be computationally costly, and struggles with extreme statistics or dependent data if applied naïvely. For applied researchers, the bootstrap is best viewed as one tool in the inferential toolbox. When combined with sound research design and thoughtful modeling, it provides a robust way to grapple with uncertainty and to extract credible insights from limited data.

Suggested Readings

Alfons, A., Ateş, N. Y., & Groenen, P. J. (2022). A robust bootstrap test for mediation analysis. Organizational Research Methods, 25, 591-617.

Efron, B. (1979). Bootstrap Methods: Another look at the jackknife. The Annals of Statistics, 7, 1-26.

Efron, B. (2000). The bootstrap and modern statistics. Journal of the American Statistical Association, 95, 1293-1296.

Efron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science, 54-75.

McLachlan, G.J. (1987). On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. Journal of the Royal Statistical Society, Series C, 36, 318-324.

Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879-891.

Stine, R. (1989). An introduction to bootstrap methods: Examples and ideas. Sociological Methods & Research, 18, 243-291.

Tibshirani, R. J., & Efron, B. (1993). An introduction to the bootstrap. Monographs on Statistics and Applied Probability, 57, 1-436.

The post Understanding the Bootstrap appeared first on CenterStat.

In a world overflowing with data, it’s easy to assume that numbers tell the whole story. But while quantitative data can show what is happening, it rarely explains why. That’s where qualitative research comes in—offering the rich, nuanced insights needed to truly understand human behavior, decision-making, and the social and cultural contexts that shape them.

Qualitative research goes beyond surface-level observations to uncover motivations, beliefs, and lived experiences. Through interviews, focus groups, observations, and textual analysis, it gives researchers the tools to see the world from the perspective of the people living in it. These methods have the power to transform the way we design programs, implement policies, and improve services across fields like healthcare, education, business, and community development.

And now, CenterStat is making it easier than ever to master these skills with the launch of the Applied Qualitative Research (AQR) workshop series—a comprehensive, four-part training program designed for both new and experienced researchers.

Why Qualitative Research Matters More Than Ever

In applied research and the social sciences, understanding the “human side” of the data is essential. Qualitative research provides:

• Deeper motivations – Find out the reasons behind actions, attitudes, and behaviors.
• Context-rich insights – Understand how cultural norms, social structures, and personal experiences influence outcomes.
• Flexibility – Adapt your research as new themes and discoveries emerge.
• Participant-centered perspectives – Amplify the voices of underrepresented or marginalized communities.
• Stronger quantitative studies – Use qualitative methods to clarify constructs, refine survey questions, and build better measurement tools.

For example, in healthcare, interviews with patients can reveal emotional responses to treatment or cultural barriers to care—details that can lead to more effective and equitable interventions. In organizational research, focus groups might uncover hidden resistance to change or shed light on team dynamics that numbers alone could never explain.

A Flexible, Practical Approach to Learning

CenterStat’s Applied Qualitative Research series is designed with real-world application in mind. Whether you work in academia, policy, healthcare, nonprofit organizations, or private industry, you’ll walk away with skills you can put to work immediately.

Each workshop combines clear, research-based instruction with practical exercises, templates, and tools. You can take the full series for a complete learning experience or choose the individual sessions that best fit your needs.

The Four Workshops in the AQR Series

1. Applied Qualitative Research: Foundations (Free Introductory Session)

Your starting point for mastering qualitative research. This session introduces you to the core principles of qualitative inquiry, including how it fits within a pragmatist framework. You’ll learn to identify the right research questions, select appropriate study populations, and define units of observation. The highlight is The Nine Elements of a Good Research Question, a practical tool that ensures your research starts on the right track.

2. In-Depth Interviews (IDIs)

Learn to conduct powerful, one-on-one conversations that uncover authentic insights. This workshop covers:

• When and why to use IDIs
• Sampling and recruitment strategies
• Interview guide development
• The art of inductive probing to elicit deeper responses
• Practical tips for in-person and remote interviews

Participants will also observe and critique a demonstration interview, and receive ready-to-use tools such as transcription protocols and data management templates.

3. Focus Groups (FGs)

Harness the dynamic energy of group conversation to explore shared and differing perspectives. You’ll learn:

• How to recruit participants and determine sample sizes
• Best practices for developing an FG discussion guide
• Strategies for moderating discussions and managing group dynamics
• How to work effectively with co-facilitators and assistants
• Special considerations for running digital focus groups

The session includes a demonstration FG and practical templates to simplify documentation and transcription.

4. Thematic Analysis

Transform your raw qualitative data into clear, actionable findings. This hands-on workshop takes you step-by-step through:

• Coding and identifying themes
• Building a codebook and linking themes into a conceptual model
• Using qualitative data analysis (QDA) software
• Writing results for academic and applied audiences

The focus is on producing research that is both rigorous and ready to make an impact.

Why Choose CenterStat for Qualitative Training?

• Expert Instructors – Learn from leaders in statistical and qualitative research education with years of hands-on experience.
• Practical Tools – Every session includes ready-to-use templates, protocols, and guides.
• Flexible Learning – Take the workshops in sequence or choose the ones that meet your current needs.
• Designed for All Levels – From beginners to seasoned researchers, every participant gains actionable skills.
• Real-World Relevance – Training is tailored for applied research, not just theory.

Make Your Research Count

In an era where data is everywhere, the ability to truly understand people—beyond the numbers—is a competitive advantage. The skills you develop through the Applied Qualitative Research series can help you design more effective studies, communicate your findings with greater impact, and ultimately create change that matters.

Whether you are launching your first qualitative study or seeking to expand your analytical toolkit, this series will give you the confidence, clarity, and capability to succeed.

Ready to get started? See upcoming sessions and register today at CenterStat.org.

The post Discover the Power of Qualitative Research appeared first on CenterStat.

At CenterStat, we believe you shouldn’t have to choose between quality and affordability. That’s why we’ve built an online education platform that delivers unmatched excellence, clarity, and value. When compared with other providers, CenterStat doesn’t just compete: we lead.

Here’s why CenterStat is the gold standard for online statistics training, and why thousands of learners around the world choose us as their trusted resource.

1. The Most Affordable Cost for the Highest Quality Training

Let’s face it: many online courses in statistics and data science come with a hefty price tag, often far exceeding their value. At CenterStat, our mission is different. We believe that world-class training should be accessible to everyone, not just those with large research budgets or institutional funding.

That’s why we’ve made our tuition structure among the most affordable in the industry—without compromising on quality. We deliver the depth, rigor, and expertise of a university-level course at a fraction of the cost. It’s not just a better price—it’s better value.

2. Instructors Who Are Leaders in Their Fields

Our instructors aren’t just skilled educators—they’re internationally recognized experts who have made novel contributions to quantitative methods in their respective fields. Every member of our instructional team is deeply experienced, actively publishing, and helping shape the landscape of modern statistical research.

Moreover, many of our faculty have won university-wide and national awards for both teaching and research. They bring that same passion for clarity, depth, and practical application to every workshop we offer.

3. High-Quality Materials with a Consistent Structure

One common complaint about online courses is inconsistency—each course is built differently, leaving learners confused and frustrated. At CenterStat, we take a different approach. Every course is developed within a carefully designed, consistent framework.

Each class includes comprehensive PDF lecture notes, step-by-step computer demonstration notes, fully executable computer code and real-world sample datasets. This level of organization allows you to focus on learning—not on figuring out how the materials fit together. Participants are confident that they will get the same level of quality and comprehensiveness no matter which classes in which they choose to enroll.

4. Self-Paced, Individually Pre-Recorded Video Instruction

Unlike many competitors who simply upload recordings of live classes, CenterStat creates custom, pre-recorded videos specifically designed for self-paced learning. This means there are no audio glitches or disorganized transitions, there are focused explanations tailored to asynchronous learners, and there are clear visual demonstrations, edited for maximum clarity. The result is a far more effective and engaging learning experience than when a live offering is simply recorded and posted for later viewing.

5. Real-Data Demonstrations Across All Major Software

Theory is important, but application is critical. At CenterStat, all of our courses include hands-on demonstrations using real data, not contrived classroom examples. Even better, we support instruction across all major statistical platforms including R, SAS, Stata, SPSS, Mplus, and Python. No matter what tools you use (or new tools you want to learn), our courses meet you where you are and guide you in developing skills that are immediately transferable to your own research or professional projects.

6. Lifetime Access, No Expirations: Ever. You heard us right: Ever. 

When you register for a CenterStat course, you own it for life. All videos, PDFs, code, and datasets remain available indefinitely—so you can revisit the material anytime, at your convenience.

Whether you want to refresh a method in a year or re-watch a demonstration before applying it to new data, CenterStat is always there to support you.

7. Evergreen Content with Free Updates

Most online training platforms treat courses as fixed products: once you buy them, any improvements or expansions are locked behind new fees. Not at CenterStat.

We treat all our major content as evergreen. That means if a course is updated, you get the new content automatically, at no additional cost—regardless of when you registered. You’ll always have access to the most current, relevant training in the field.

8. A Commitment to Free, Public Educational Resources

At CenterStat, we’re committed not just to our students, but to the broader public good. That’s why we’ve created and maintain a rich library of free instructional materials including extensive tutorial videos, lecture notes, coding tutorials, sample code and data, reference guides, and more.

We believe deeply in the broadest possible dissemination of quantitative education, and we make meaningful, high-quality resources available to everyone, regardless of financial status.

9. A Fully Integrated Curriculum

Most online training sites offer a scattered menu of unrelated courses created by different instructors with no overarching plan or coordination. CenterStat offers something better: a carefully integrated curriculum.

Every course is designed to fit within a coherent learning path, allowing you to build from foundational principles to advanced techniques with logical, structured progression. There’s executive oversight of content, which means no redundancy, no gaps, and no guesswork about which course to take next. You can be confident that you receive the same high quality CenterStat training no matter which course or instructor you choose. 

10. A Balanced Focus on Theory and Practice

At CenterStat, we never sacrifice one side of learning for the other. Our courses are carefully designed to strike a perfect balance between statistical theory and hands-on application.

You’ll learn the mathematical and conceptual foundations of each model alongside detailed walkthroughs of how to apply them to real-world data using modern software tools. This dual focus ensures that your understanding is both deep and immediately actionable. This means you’ll understand the source and solution to esoteric error messages (e.g., the dreaded psi is non-positive definite) while at the same time being able to tell a meaningful and compelling story about your findings to the reader. 

11. Rigorous Yet Engaging Instructional Style

Too often, statistical education swings between extremes—either overly dry and abstract, or superficial and oversimplified. Our approach is different. Our instructors present complex concepts in a rigorous yet colloquial style that makes advanced material feel approachable and—even enjoyable and at times downright humorous (just wait for Dan to start showing pictures of Patrick in awkward stages of childhood and adolescence).

We explain not just what to do, but why it works, using plain language, relevant examples, and a touch of humor to keep you engaged.

12. Recognized Excellence in Teaching and Research

Many of our instructors are award-winning educators and researchers, recognized nationally and internationally for their ability to communicate complex ideas clearly and meaningfully. When you learn from CenterStat, you’re learning from the best—not just in statistics, but in the art of teaching statistics. Some platforms allow any instructor to post any material they choose, but at CenterStat we hand pick our instructors from the best in the world and then guide them in developing their materials in the CenterStat Way. 

13. Full Transparency: Detailed Syllabi, Sample Content, and Clear Course Overviews

We understand that enrolling in a course is a commitment. That’s why we provide clear, detailed information about every workshop before you register. This includes full syllabi, sample materials, overview videos, clear learning objectives, software covered, and time expectations. 

We remove the guesswork so you can make confident, informed decisions about your training.

14. Tuition Support for Degree-Seeking Students

We’re proud to support the academic journey. If you’re an undergraduate or graduate student currently enrolled in a degree-granting program, you may qualify for reduced tuition rates.

We believe students shouldn’t have to wait for graduation—or major grant funding—to receive the training they need today. Let CenterStat meet your training goals no matter where you are in your professional development. 

15. Certificates of Completion

Upon completing any CenterStat workshop, you’ll receive a certificate of completion—a valuable credential that demonstrates your commitment to professional development. Whether for your CV, academic portfolio, or professional advancement, your certificate signals real achievement in a rigorous and respected program.

Experience the CenterStat Difference

At a time when online education is more accessible than ever, quality still matters. CenterStat stands apart because we combine deep academic rigor with practical usability, affordable pricing with unparalleled value, and a learner-first mindset that informs everything we do.

From our award-winning instructors to our lifetime access, from real-data software demonstrations to our commitment to educational equity, CenterStat doesn’t just deliver courses: we deliver clarity, confidence, and competence in statistical methods all at an affordable price.

Ready to elevate your statistical skills with the best in the field?

Explore our workshops today—and experience for yourself why CenterStat is trusted by students, scholars, and professionals around the world.

The post Why CenterStat is the Best Choice for Online Statistics Training appeared first on CenterStat.

Built for All Stages of the Research Journey

One of the greatest strengths of our workshops is their accessibility across experience levels. We’ve seen participants ranging from doctoral students and postdocs to established professors and NIH-funded investigators. Each of our programs—whether focused on multilevel modeling, structural equation modeling, longitudinal data analysis, or machine learning—is structured to accommodate varying levels of statistical background while still diving deep into theory and application.

“The key strength was the clarity of communication and the balance between technicalities and intuition. Dr. Curran & Dr. Bauer did not shy away from technical aspects of the content but also communicated about them as intuitively as possible.”
— Workshop participant, Introduction to Structural Equation Modeling

Practical Skills You Can Use Immediately

CenterStat workshops are not passive lectures—they’re dynamic, hands-on experiences. Each session blends foundational theory with real data exercises, ensuring that what you learn translates directly to your work. Whether you’re writing a grant, preparing a manuscript, or designing a study, our goal is to give you tools that make an impact now.

“Craig is easily one of the best statistics professors I have ever had. He has taken an incredibly complex topic that I have previously struggled to understand and apply to my own data and made it possible for me to feel confident in using and interpreting the analyses on my own
— Workshop participant, Modern Missing Data Analysis

Expert Instructors Who Are Also Practitioners

Our instructors are internationally recognized scholars who don’t just teach statistics—they use them every day in their own research. They know the challenges of applying quantitative methods in real-life scenarios, and they bring that experience into every lecture and lab session.

“This is the most interactive online workshop I have ever attended. It kept me very engaged, motivated to learn, and excited to apply techniques and discuss with the group. Greg and Emily are very knowledgeable, passionate, and eager to share their knowledge with attendees. They are fantastic at answering questions promptly, and they made the entire experience conversational, interactive, and exciting.”
— Workshop participant, Applied Qualitative Research

A Supportive Learning Community

When you join a CenterStat workshop, you’re not just gaining new knowledge—you’re joining a network of fellow researchers who are equally passionate about improving their craft. Our sessions foster collaboration, discussion, and long-term academic connections.

We also offer ongoing support after the workshop ends, including access to course materials, helpful guides, and recommendations for continuing education.

Results That Speak for Themselves

CenterStat alumni have gone on to publish in top-tier journals, secure major grants, and teach advanced methods at leading institutions. We take pride in helping researchers unlock their full potential by providing training that’s relevant, rigorous, and rooted in real-world impact.

“This is my third C&B workshop and I’ve really enjoyed all of them. I especially appreciate the attention to the algebra behind models–they’ve help me understand other/new-to-me approaches in method papers.”
— Workshop participant, Latent Curve Modeling

Ready to Level Up Your Quantitative Skills?

Explore upcoming workshop opportunities and see how CenterStat can support your research journey at www.centerstat.org. Whether you’re looking to master new techniques or refresh your knowledge, our expert-led sessions will help you gain the confidence and skills to tackle your most important research questions.

The post How CenterStat Workshops Empower Researchers at Every Level appeared first on CenterStat.

Credits are denominated in US dollars and are redeemable for up to five years from purchase, so purchasing credits is a great way to convert time-limited funds into an extended opportunity for professional development. Credits are fully transferable to multiple individuals without restrictions, making it easy to provide professional development opportunities to yourself or members of your organization.

As our way of trying to help out in tough times, we will also add an extra $100 in credits to your account for every $1000 of training credits you purchase.

Please visit our Training Credits page for complete details.

The post CenterStat Announces the Availability of Training Credits appeared first on CenterStat.

To understand suppression, we first need to remind ourselves of a simple two-predictor multiple regression model. Although throughout this note we focus on two predictors and one outcome, all of the concepts easily generalize to multiple predictors, sets of predictors, and even multiple outcomes (e.g., as might be found in a path analysis or structural equation model). This simple two predictor model can be expressed as

y=b₀+b₁x₁+b₂x₂+r

where y is the outcome, b₀ is the intercept, b₁ and b₂ are the regression coefficients relating x₁ and x₂ to y, respectively, and r is the residual. As always, each regression coefficient is the unique relation between that predictor and the outcome when controlling for (or above and beyond) the effects of the other predictor. In many situations, if two predictors are correlated with one another then the unique relation of one predictor and the outcome controlling for the other predictor is smaller than that same predictor when considered alone. (Spoiler alert: in suppression the opposite occurs, which is what makes it so incredibly weird).

The typical situation is best seen using a Venn diagram. Consider the bivariate relation between x₁ and y where, for the moment, we ignore x₂. In the Venn diagram the bivariate relation between these two variables is denoted as area a. This represents the relation between x₁ and y ignoring the effect of x₂.

Often, however, we are interested in the unique effects of two (or more) predictors (as well as the joint effect, but we ignore this for now). To obtain these, we bring both predictors into the model at the same time. Typically, x₂ is correlated with both x₁ and y. As seen in the diagram, in the presence of the second predictor, the effect of the first predictor, a, is usually smaller than it was before. This is quite natural in that the part of x₁ that is shared with x₂ is removed when assessing the unique relation between x₁ and y, making a smaller. This is business as usual.

However, let’s make things a bit stranger. More than 80 years ago, a brilliant quantitative psychologist named Paul Horst found a situation in which the relation between x₁ and y was actually larger in the presence of x₂ than when assessed in the absence of x₂ (Horst, 1941). Considering the above Venn diagrams, this makes absolutely no sense at all; it actually seems downright impossible. Yet it most definitely exists, and Horst somewhat unfortunately termed this situation suppression. It is unfortunate because x₂ is not suppressing the relation between x₁ and y (which many researchers assume). In actuality, x₂ is suppressing irrelevant variance in x₁ and, by doing so, enhances the relation between x₁ and y. It might have been better to refer to x₂ as an “enhancer” rather than a “suppressor”, but that historical ship has sailed so we are stuck with this terminology.

To understand suppression better, let’s first think about the substantive application in which Horst observed this phenomenon. He was part of a research team evaluating pilots during World War II. The pilots completed paper-and-pencil assessments measuring three types of cognitive reasoning: mechanical, numerical, and spatial. These three measures were then used to predict a score representing piloting ability. However, these measures were not as strongly predictive of piloting ability as had been anticipated. A fourth predictor was then included that was a measure of general verbal ability. Verbal ability was correlated with each of the three reasoning measures (as would be expected), but was not correlated with piloting ability (as would also be expected). Consistent with expectations, when verbal ability was added to the regression model it did not uniquely predict piloting ability. However, quite unexpectedly, the effects of all three reasoning measures were markedly larger compared to the model in which verbal ability was omitted. Horst determined that the reason was that verbal ability was removing irrelevant information from the three reasoning measures (that is, suppressing the part of the variance in reasoning that was related to verbal ability but unrelated to piloting ability) and this in turn enhanced the relations between what was left over in the reasoning measures and pilot ability, i.e., the unique relations.

It is helpful to consider a simple hypothetical example. We will define x₁ to be the predictor variable of interest (say mechanical reasoning) and x₂ the suppressor variable (say verbal ability). The simplest pattern of correlations consistent with traditional suppression is when x₁ is correlated with y, x₁ is correlated with x₂, and x₂ is not correlated with y. Of course, in any sample data there will rarely be a zero correlation between the suppressor and the outcome, but it might still obtain some negligible value. Let’s further say that the correlation between x₁ and y is .25, between x₁ and x₂ is .70, and between x₂ and y is zero. If we consider a model in which only x₁ predicts y, the standardized regression coefficient for x₁ is equal to .25 with a squared semi-partial correlation of .06 (that is, x₁ uniquely accounts for 6% of the variance in y). However, if we add x₂ as a second predictor, the standardized regression coefficient for x₁ increases to .49 and the squared semi-partial correlation doubles to .12 (that is, x₁ now uniquely accounts for 12% of the variance in y). The unique effect of x₁ on y when controlling for x₂ is markedly stronger than the bivariate relation of x₁ with y, the hallmark of suppression.

What on Earth is going on? Of course, the presence or absence of the suppressor does not change the bivariate relation between the predictor and the outcome (the correlation between x₁ and y is always .25). However, what the suppressor does change is the unique variability in the predictor that is available to be related to the outcome.

We can see this in a simple re-arrangement of the Venn diagram (the Venn diagram is an imperfect representation of the underlying mathematics, but visually gives a sense what is happening here). This shows that the suppressor, x₂, correlates with the predictor, x₁, as indicated by area b, but does not correlate with the outcome (reflected in the lack of overlap of the circles for x₂ and y ). Further, we can see that controlling for the suppressor reduces (or suppresses) part of the variance in x₁ that is unrelated to y (area b). This, in turns, enhances the proportional relation between x₁ and y (represented by area a). This is the core of suppression.

There have been dozens of papers written on suppression following Horst’s initial discovery, and we note several of these below. Many of these propose specific subtypes of suppression and describe under what unique conditions these might be encountered in practice. However, a concise general definition was given by Conger (1974) who wrote “A suppressor variable is defined to be a variable which increases the predictive validity of another variable (or set of variables) by its inclusion in a regression equation. This variable is a suppressor only for those variables whose regression weights are increased.” Importantly, this means that a variable is not inherently a suppressor in and of itself. Instead, a suppressor is defined by the impact it has on other variables in the model. That is, a variable might be a suppressor in one model but not in another.

This brings us to two initial points. First, there is nothing about suppression that is magical, mysterious, or misunderstood. The papers noted below explain in gory detail exactly what suppression is and under what conditions it exists, so be suspect of a paper that says “Suppression is a long-misunderstood issue…”. It is not. Initially confusing? Yes. Misunderstood? No.

Second, suppression is not some unavoidable artifact of measurement or estimation but instead can be fully accounted by substantive theory. Horst’s example is just one of many in the literature, nearly all of which make perfect sense within a given theoretical framework. As such, it is often beneficial to think about potential suppressors during the design phase of a study so that all relevant variables can be included in the analysis.

However, our third and final point is a bit of a punch in the face: we must consider two additional competing explanations for the role of a third variable in our models: confounding and mediation.

By far the clearest treatment of this was given by MacKinnon, Krull and Lockwood (2000) in a title that could not be more on point: Equivalence of the Mediation, Confounding, and Suppression Effect. You nearly don’t have to read the paper given the title. The paper opens with, “Once a relationship between two variables has been established, it is common for researchers to consider the role of a third variable in this relationship.” This is precisely what we have considered thus far. But to better see this point, we move from Venn diagrams to path diagrams. Let’s first consider the two-predictor regression that we have discussed up to this point:

This shows the usual expression of two correlated predictors and one outcome. Note that there are three measured variables, and these are all related to one another (the curved arrow reflects the correlation between the two predictors, and the two one-headed arrows reflect the partial regression coefficients). In a suppression situation, x₁ enhances the relation between x₁ and y.

However, with a simple re-arrangement of the diagram we get what is called confounding:

Note that all we have done is changed the correlation between the two predictors to a regression coefficient and now x₂ is a confounder in that it predicts both x₁ and y. This is the situation that is so fun to teach because we can give examples such as the number of fire trucks sent to a fire is positively correlated to the amount of damage done at the fire; but when the confounder of severity of fire is included, there is no relation between number of trucks and damage. Importantly, whereas a suppressor enhances the relation between x₁ and y, including a confounder as a second regressor reduces this same relation.

Finally, re-directing one arrow in the above path diagram results in mediation:

Now x₂ explains the relation between x₁ and y. For example, the predictor might be parent’s alcohol use, the outcome is the child’s alcohol use, and the mediator is impaired parenting. The inference is that the parent’s alcohol use impairs their own parenting behavior, and this in turn increases the probability that the child will drink alcohol themselves. In sum, suppression enhances the relation between a predictor and the outcome, confounding reduces the relation, and mediation explains the relation. How the heck do we differentiate among the three? MacKinnon et al. (2000) argue that you do not, and they demonstrate that the statistical tests of these three effects are all identical: each model is a simple re-expression of the others. They conclude the paper saying, “The statistical procedures provide no indication of which type of effect is being tested. That information must come from other sources.” The “other sources” to which they refer are prior knowledge and theory. All three effects are statistically isomorphic, and only theory can discern which most likely holds in the population. Further differentiation might also be possible by moving to experimental or longitudinal designs that allow for better testing of hypotheses about causal pathways.

Suggested Readings

                Conger A.J. (1974). A revised definition for suppressor variables: A guide to their identification and interpretation. Educational Psychological Measurement, 34, 35–46.

                Horst P. (1941). The role of predictor variables which are independent of the criterion. Social Science Research Council Bulletin, 48, 431–436

                MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1, 173-181.

                Tzelgov, J., & Henik, A. (1991). Suppression situations in psychological research: Definitions, implications, and applications. Psychological Bulletin, 109, 524-536.

                Velicer W.F. (1978). Suppressor variables and the semipartial correlation coefficient. Educational and Psychological Measurement, 38, 953–958

The post What is a suppressor variable, and how does this differ from confounding and mediation? appeared first on CenterStat.