Why LatentGOLD? First, it is a versatile latent variable modeling software program that can be used to estimate all of the models considered in the class, so why not? Second, not long ago we added an entire chapter to the workshop on approaches for robustly examining relationships between latent classes and external variables (e.g., predictors of class membership or distal outcomes predicted by class membership), and these approaches are exceptionally well implemented in LatentGOLD. This comes as no suprise given the primary developer of LatentGOLD, Jeroen Vermunt, also played a key role in the innovation of these methods. Third, Statistical Innovations Europe now offers a free academic license to LatentGOLD and they have partnered with us to provide a 90-day free license to non-academic workshop participants as well.

The LatentGOLD demonstrations complement prior demonstrations in Mplus and R, providing users with a variety of choices when fitting finite mixture models, including applications of latent profile analysis and latent class analysis.

If you previously enrolled in the class, then you have instant access to these materials as a benefit of our evergreen content model. Otherwise, enroll now to take advantage of this added software flexibility!

The post LatentGOLD demonstrations now included in Mixture Modeling and Latent Class Analysis workshop appeared first on CenterStat.

For decades, we have been able to expand growth models to include time-specific (and even lagged) effects for predictors that themselves change over time, often called time-varying covariates (or TVCs). TVC models allow us to transition to testing within-person hypotheses because the model does not just ask whether people change but whether within-person fluctuations in predictors are related to within-person deviations in the outcome. This is our focus here.

A Brief Review of Growth Modeling

Growth modeling is a massive topic that cannot be fully covered here. However, we have prior blog posts, a free online lecture series on different approaches to growth modeling, and we offer several full-length workshops on growth modeling using multilevel models (MLMs), structural equations models (SEMs), and models for intensive longitudinal data (ILD).

Briefly, the core motivating goal of growth modeling, whether estimated using the MLM or the SEM, is to summarize a set of repeated measures with a smoothed underlying trajectory that parsimoniously represents how the outcome changes over time. Individual trajectories can obtain a variety of functional forms (e.g., linear, quadratic, piecewise, exponential, etc.) or even display no systematic change over time (and thus necessitate only an intercept term).

A key outcome of the model is the estimation of the means (sometimes called fixed effects) and the variances (sometimes call random effects) of each defined growth component. For example, say we fit a linear trajectory to six repeated measures of aggressive behavior in children. We could obtain a mean starting point and rate of change reflecting the average trajectory for the full sample, and a variance in starting point and rate of change (and covariance between the two) reflecting child-to-child variability in the trajectory parameters. This model may be of theoretical interest on its own, but we often expand this model to include one or more covariates of theoretical interest. There are two types of covariates that are commonly considered.

Time-invariant vs. Time-varying Covariates

Time-invariant covariates (TICs) are assumed to have a constant value across measurement occasions. Examples include sex assigned at birth, baseline SES, treatment condition, or childhood adversity at the start of kindergarten. In growth models, TICs typically predict differences in trajectory parameters and assess questions such as does baseline SES predict the intercept (i.e., starting point) or does treatment condition predict the slope (i.e., rate of change)? An example finding might indicate that there are no treatment group differences in depression at baseline but that, on average, those who received treatment reported a significant decrement in depressive symptomatology over time compared to those who did not. An exemplar path diagram of a latent curve model with two TICs (x₁ and x₂) predicting trajectory intercepts and slopes (η₁ and η₂) for five repeated measures (y₁–y₅) is:

In contrast, time-varying covariates (TVCs) have the potential to differ in value at each measurement occasion. This is a critical distinction from TICs. Whereas baseline SES is by definition invariant to the passage of time, other constructs, such as daily stress, nightly sleep, current caregiver status, or weekly anxiety, change from one time point to the next. Whereas TICs predict the growth factors themselves (e.g., we regress the intercept and slope trajectory components on treatment group membership), TVCs directly predict the time-specific repeated assessments above-and-beyond the underlying growth process. Although a seemingly modest change to the structure of the model, this actually has profound implications for the types of questions we can ask, particularly those focused on within-person process. An exemplar path diagram of a latent curve model with a TVC (z₁–z₅ measured at time 1 to 5) is:

Note that TVCs can have both within- and between-person effects.  For example, suppose we wish to evaluate the relation between substance use and anxiety over a set of repeated measures.  The between-person effect would reflect average differences between people: Do people who report higher average levels of anxiety over time also report higher average levels of substance use over time? This contrasts from the within-person effect: When a person is more anxious than usual at a given time point do they tend to engage in more substance use than they typically do? Often with TVCs we are particularly interested in modeling these within-person processes, which can be extended to examine lead-lag relations, and dynamic systems in ways not possible with a standard TIC model.

In the spirit that you don’t get something for nothing, inclusion of TVCs introduces natural complexities that must be addressed, particularly in the management of the data, specification of the model, and substantive interpretation of the findings. However, these issues are all well understood and one needs to simply turn to existing resources that illustrate best practices.

Incorporating TVCs into Growth Models

One point of common confusion is how within- and between-person effects of TVCs are estimated and interpreted within the SEM versus MLM frameworks. Within the SEM growth model, the TVCs are incorporated in their raw metric as exogenous predictors of the repeated measures. Because the TVCs freely covary with the latent growth factors (or at least should; be sure your software package is estimating these relations), the resulting regression parameters represent pure estimates of within-person effects. The standard SEM specification does not provide pure estimates of between-person effects of TVCs; however, these can be obtained by expanding the model in particular ways (see Curran et al., 2012, for details).

In contrast, if the TVCs are incorporated into the MLM in their raw metric, the resulting regression coefficients inextricably combine the within-person and between-person effects (see Raudenbush & Bryk, 2002, for details). A well-known solution to this problem is to person-mean center, that is subtract the person mean of the TVC from each observed value of the TVC, and then use the person-mean-centered TVC as the predictor at Level 1; this captures the same pure within-person effect estimate as the SEM. However, the person mean of the TVC itself can also be incorporated as an additional predictor at Level 2 of the model to capture the between-effect.

Thus, under a broad set of conditions, the MLM and SEM will provide precisely the same (or nearly the same) estimates of the within and between effects of TVCs, even though this is accomplished in quite different ways.

Limitations

Both the SEM and MLM approaches to modeling TVCs include a number of assumptions, but two are of most interest here. First, using either of the methods described above, we make a fundamental assumption that although the distal outcome may be growing systematically over time, the TVC most decidedly is not. Think about this logically: we deviate the person mean from each measured TVC and the person mean is by definition constant over time. In other words, there is no growth trajectory for the TVC. Significant estimation and interpretation problems can be encountered if the TVC itself is growing over time and this is not adequately represented in the model (see Curran & Bauer, 2011, for detailed examples, and Wang & Maxwell, 2015, for exceptions). To be clear, methods exist for handling such conditions, but additional work is necessary to represent systematic co-occurring individual change in the model (Curran et al., 2014).

The second assumption relates to the strength of relation between the TVC and the outcome over time. Consider three possible scenarios: (1) the magnitude of the relation is equal at all time points; (2) the magnitude of the relation systematically increases or decreases with the passage of time; or (3) the magnitude of the relation obtains a unique value at every single time point. All three of these can be incorporated in both the SEM and the MLM, and formal tests are available to determine which condition best represents the sample data at hand. However, although these methods offer powerful tests of the nature of the relation between the TVC and the outcome, there is a well-developed yet little used method that provides more flexible insight into these over-time relations, and this is called the time-varying effects model, or TVEM.

Time-Varying Effects Models

Time-varying effects models offer a semi-parametric approach to the more traditional TVC model within the MLM framework. Whereas the MLM provides formal tests of structured relations between the TVC and the outcome (e.g., a bilinear “product” interaction between the TVC and time), the TVEM uses a spline method of estimation to approximate complex nonlinear relations that might wax and wane in magnitude over time. Confidence regions can then be plotted to show “sensitive periods” during which a TVC is significantly related to the outcome versus periods when it is not. These relations are primarily graphical in nature and lovely plots can be created that demonstrate the potentially complex nature of the relation between the TVC and the outcome as a function of time. The TVEM offers many advantages when trying to understand complex relations over time, yet these methods remain quite underutilized in practice. See Lanza and Linden-Carmichael (2021) for a thorough introduction to the estimation and interpretation of TVEMs within the social sciences.

Conclusion

In sum, time-varying covariate growth models allow for the introduction of powerful tests of within-person dynamics over time that are not present in a more standard TIC. These methods are well developed and widely used and can easily be incorporated in your own data applications. A variety of resources exist, and we cite several of these below. CenterStat also offers full workshops that provide detailed instruction on TVC models, including Multilevel Models for Longitudinal Data, Longitudinal Structural Equation Modeling, and Analysis of Intensive Longitudinal Data. Including TVCs in your models requires care but can also yield important new insights.

Suggested Readings

Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583-619.

Curran, P. J., Howard, A. L., Bainter, S. A., Lane, S. T., & McGinley, J. S. (2014). The separation of between-person and within-person components of individual change over time: a latent curve model with structured residuals. Journal of consulting and clinical psychology, 82(5), 879.

Curran, P.J., Lee, T.H., Howard, A.H., Lane, S.T., & MacCallum, R.C. (2012). Disaggregating within-person and between-person effects in multilevel and structural equation growth models. In: Hancock, G., editor. Advances in longitudinal methods in the social and behavioral sciences. Charlotte, NC: Information Age; p. 217-253.

Hamaker, E. L., Kuiper, R. M., & Grasman, R. P. (2015). A critique of the cross-lagged panel model. Psychological Methods, 20(1), 102.

Lanza, S. T., & Linden-Carmichael, A. N. (2021). Time-varying effect modeling for the behavioral, social, and health sciences. Springer.

Raudenbush, S.W. & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods. Advanced Quantitative Techniques in the Social Sciences Series/SAGE.

Shiyko, M.P., Burkhalter, J., Li, R., & Park, B. J. (2014). Modeling nonlinear time-dependent treatment effects: an application of the generalized time-varying effect model (TVEM). Journal of Consulting and Clinical Psychology, 82, 760.

Wang L.P. & Maxwell S.E. (2015). On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychological Methods, 20, 63-83.

The post The Strengths and Limitations of Time-Varying Covariate Growth Models appeared first on CenterStat.

Parametric Inference and the Sampling Distribution

In classical statistics, inference depends on specifying a model for the data-generating process and the idea of repeated sampling. For instance, suppose we take a sample of size N and calculate the sample mean. Ultimately, we’d like to make a conclusion about the mean in the population from which we drew our sample. However, we need to account for sampling error. That is, had we drawn a different sample of the same size, we would have obtained a different sample mean. In fact, there are infinitely many different samples of size N that we might have drawn from our population, each of which would yield a somewhat different mean, simply depending on who happened to get into the sample. The collection of means across all of these hypothetical samples has its own distribution, known as a sampling distribution. The sampling distribution reflects our uncertainty in the estimation of the population mean (i.e., variation due to sampling error) and it underpins confidence intervals, hypothesis tests, and other inferential procedures.

For instance, under the assumption that the we draw independent and identically distributed observations from a normal distribution (or by appealing to the Central Limit Theorem at large sample sizes), the sample mean follows a normal distribution with mean equal to the population mean and standard error equal to the standard deviation divided by the square root of N. If the standard deviation for the population is known, then we can simply divide our sample mean by the standard error and reference this to the standard normal distribution (z-distribution) to make inferences. More typically, however, we aren’t privy to this knowledge and must plug in our sample standard deviation to get the standard error. The sampling distribution of the mean will then deviate from the normal curve due to this added uncertainty in small samples. Thanks to our favorite Guinness brewer William Gossett (who published under “Student”), we know that in such cases, the sample mean divided by the standard error follows a t-distribution, a bell-shaped distribution whose tail thickness is determined by the degrees-of-freedom.

So far, we have focused on the sample mean, but we can imagine a sampling distribution for any parameter we wish to estimate, whether it be a regression coefficient, variance, factor loading, or any other value of interest. The challenge, however, is that in real research we often find ourselves making uncertain parametric assumptions in order to obtain a known sampling distribution. For example, when using a t-distribution to make inferences about a regression coefficient, we assume a sufficiently large sample size and that the errors are normally distributed, independent, and homoscedastic, conditions that may or may not hold in practice. When assumptions are met, such parametric procedures work exceedingly well; when not met, the resulting inferences can be both biased and misleading, sometimes markedly so.

What Is the Bootstrap?

The non-parametric bootstrap, first formally proposed by Bradley Efron in 1979, is a computational technique for empirically approximating the sampling distribution without the requirement of strong parametric assumptions. Instead of relying on mathematical formulas, the bootstrap uses the observed data itself as a stand-in for the population. (Thus, the term “bootstrap” which is drawn from the phrase draw yourself up by your own bootstraps meaning you take personal responsibility and use the resources you have available to you). The basic procedure is conceptually simple. First, you draw your sample of size N from the population in the usual way. Next, you draw a “bootstrap sample” also of size N from the original data with replacement. Then you compute and retain your statistic of interest on the bootstrap sample in whatever way you please (e.g., mean, regression coefficient, mediated effect). Finally, you repeat this process many times (often with 1000 bootstrap samples or more) to create an empirical distribution for the statistic from which to make inferences back to the population.

The critical step to understand is that we are randomly drawing a bootstrap sample from our original sample data with replacement. Say we have a sample of N=100 observations; we would draw say 1000 bootstrap samples of size 100 where in each one a given observation may appear repeatedly or not at all (thus the sampling “with replacement”). This empirical distribution of the estimate (whatever that might be) under very general conditions approximates the sampling distribution. As such, we can use this to compute standard errors, confidence intervals, and bias estimates in similar ways to that of the parametric sampling distribution but with far fewer assumptions about the population. For instance, the bootstrapped standard error is simply the standard deviation of the bootstrapped sample estimates. And a bootstrapped confidence interval can be computed by simply locating the 2.5^th and 97.5^th percentiles of the bootstrapped sample estimates, values that may or may not be symmetric around the estimate (in contrast to the symmetry assumed by parametric z– or t-type confidence intervals). The beauty of the bootstrap lies in how it transforms a theoretical problem (deriving the sampling distribution) into a computational one.

Typical Applications

The bootstrap has found applications across nearly every domain of research. A classic example in the social sciences relates to the testing of indirect effects in mediation models, path analysis, and structural equation modeling. Such an effect arises within a causal chain within which one variable affects another which in turn affects a third (with more complex chains also being possible). Each effect within the causal chain is represented by its own regression coefficient. Under standard assumptions, and in large samples, each regression coefficient estimate will have a normal sampling distribution, allowing for the usual inferences. However, we don’t want to test each link in the chain individually; we want to test the chain as a whole. That is, we want to test the indirect effect of the initial predictor on the final outcome as transmitted through the intervening variables (or mediators). The sample estimate of an indirect effect is obtained by computing the product of the regression coefficients involved in the chain. Easy enough. To test the indirect effect, however, we need to know its sampling distribution, and that’s where things get tricky. Each link in the chain has a normal sampling distribution, but a product of normal variates is generally not itself normally distributed. Using a normal sampling distribution as an approximation (the delta-method or “Sobel method” for testing indirect effects) is convenient but often leads to biased inferences. This is a well-known problem that has sparked a variety of solutions, one of which is to derive the correct parametric distribution for the indirect effect (known as the “distribution of the product” method). More commonly, however, investigators have turned to the non-parametric bootstrap to obtain empirically-based inferential tests that do not rely on parametric assumptions at all. Indeed, bootstrapping is now the gold standard for testing mediated effects in practice.

Regardless of whether one is evaluating an indirect effect, a variance estimate, or any sample estimate of interest, there are many potential uses of the bootstrap results. For example, we can estimate standard errors, particularly when no simple formula exists (e.g., in complex nonlinear models). Similarly, we can compute confidence intervals using several different bootstrap methods (percentile, bias-corrected, accelerated) that provide intervals with better coverage properties than parametric ones in certain settings. Further, in machine learning and predictive modeling, bootstrap samples can be used for model validation and cross-validation and to estimate prediction error. Finally, in fields where data collection is difficult or expensive, such as clinical trials, educational experiments, or niche social science surveys, the bootstrap offers a way to make inference with small samples and limited data. These are just a few examples of how the bootstrap can be used in practice, and many additional options are available.

Advantages of the Bootstrap

There are many advantages to the bootstrap. Unlike traditional parametric methods, the non-parametric bootstrap does not require specifying a functional form for the population distribution. This makes it attractive when normality or homoscedasticity is questionable. The method also works for a wide range of statistics including means, medians, regression coefficients, correlation coefficients, or even more complex estimands like Gini coefficients. At its core, the bootstrap is easy to explain and implement. With modern software (R, Stata, SAS, Python), the procedure often requires just a few lines of code. The bootstrap can be extended for clustered data, time series, or hierarchical designs (e.g., students nested within classrooms), making it useful in applied social science and education research. As a general method, the bootstrap is remarkably flexible and can be applied in many interesting and challenging research scenarios.

Disadvantages and Limitations

As with any procedure, there are also disadvantages that must be considered. Because the bootstrap treats the sample as a proxy for the population, any biases in the sample will propagate through the bootstrap distribution. In small or unrepresentative samples, the bootstrap may give misleading results. Although less of an issue today, the bootstrap can be computationally intensive, especially for large datasets or complex models. Thousands of resamples are often needed for stable estimates. The bootstrap may also perform poorly for statistics that depend heavily on the tails of the distribution (e.g., extreme quantiles, maximum values), because the resampled datasets cannot create values outside the observed range. In clustered or dependent data structures, naïve bootstrapping can underestimate variability unless modified (e.g., block bootstrap, cluster bootstrap). This is particularly relevant in education research, where students are not independent observations, or repeated measures applications, where observations are correlated over time within persons. Care must be taken when evaluating the potential use of the bootstrap procedure in practice given the associated limitations.

Conclusion

The bootstrap represents one of the great innovations in modern statistics: a method that converts inference from an algebraic to a computational problem. By resampling from the observed data, researchers can approximate the sampling distribution of almost any statistic, gaining access to standard errors, confidence intervals, and bias estimates without heavy reliance on parametric formulas. Its strengths (flexibility, fewer assumptions, and ease of implementation) make it a powerful tool, especially in social sciences and education where data are often messy, distributions non-normal, and sample sizes modest. Yet, it is not a panacea: bootstrap inference depends on sample representativeness, can be computationally costly, and struggles with extreme statistics or dependent data if applied naïvely. For applied researchers, the bootstrap is best viewed as one tool in the inferential toolbox. When combined with sound research design and thoughtful modeling, it provides a robust way to grapple with uncertainty and to extract credible insights from limited data.

Suggested Readings

Alfons, A., Ateş, N. Y., & Groenen, P. J. (2022). A robust bootstrap test for mediation analysis. Organizational Research Methods, 25, 591-617.

Efron, B. (1979). Bootstrap Methods: Another look at the jackknife. The Annals of Statistics, 7, 1-26.

Efron, B. (2000). The bootstrap and modern statistics. Journal of the American Statistical Association, 95, 1293-1296.

Efron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science, 54-75.

McLachlan, G.J. (1987). On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture. Journal of the Royal Statistical Society, Series C, 36, 318-324.

Preacher, K. J., & Hayes, A. F. (2008). Asymptotic and resampling strategies for assessing and comparing indirect effects in multiple mediator models. Behavior Research Methods, 40, 879-891.

Stine, R. (1989). An introduction to bootstrap methods: Examples and ideas. Sociological Methods & Research, 18, 243-291.

Tibshirani, R. J., & Efron, B. (1993). An introduction to the bootstrap. Monographs on Statistics and Applied Probability, 57, 1-436.

The post Understanding the Bootstrap appeared first on CenterStat.

In a world overflowing with data, it’s easy to assume that numbers tell the whole story. But while quantitative data can show what is happening, it rarely explains why. That’s where qualitative research comes in—offering the rich, nuanced insights needed to truly understand human behavior, decision-making, and the social and cultural contexts that shape them.

Qualitative research goes beyond surface-level observations to uncover motivations, beliefs, and lived experiences. Through interviews, focus groups, observations, and textual analysis, it gives researchers the tools to see the world from the perspective of the people living in it. These methods have the power to transform the way we design programs, implement policies, and improve services across fields like healthcare, education, business, and community development.

And now, CenterStat is making it easier than ever to master these skills with the launch of the Applied Qualitative Research (AQR) workshop series—a comprehensive, four-part training program designed for both new and experienced researchers.

Why Qualitative Research Matters More Than Ever

In applied research and the social sciences, understanding the “human side” of the data is essential. Qualitative research provides:

• Deeper motivations – Find out the reasons behind actions, attitudes, and behaviors.
• Context-rich insights – Understand how cultural norms, social structures, and personal experiences influence outcomes.
• Flexibility – Adapt your research as new themes and discoveries emerge.
• Participant-centered perspectives – Amplify the voices of underrepresented or marginalized communities.
• Stronger quantitative studies – Use qualitative methods to clarify constructs, refine survey questions, and build better measurement tools.

For example, in healthcare, interviews with patients can reveal emotional responses to treatment or cultural barriers to care—details that can lead to more effective and equitable interventions. In organizational research, focus groups might uncover hidden resistance to change or shed light on team dynamics that numbers alone could never explain.

A Flexible, Practical Approach to Learning

CenterStat’s Applied Qualitative Research series is designed with real-world application in mind. Whether you work in academia, policy, healthcare, nonprofit organizations, or private industry, you’ll walk away with skills you can put to work immediately.

Each workshop combines clear, research-based instruction with practical exercises, templates, and tools. You can take the full series for a complete learning experience or choose the individual sessions that best fit your needs.

The Four Workshops in the AQR Series

1. Applied Qualitative Research: Foundations (Free Introductory Session)

Your starting point for mastering qualitative research. This session introduces you to the core principles of qualitative inquiry, including how it fits within a pragmatist framework. You’ll learn to identify the right research questions, select appropriate study populations, and define units of observation. The highlight is The Nine Elements of a Good Research Question, a practical tool that ensures your research starts on the right track.

2. In-Depth Interviews (IDIs)

Learn to conduct powerful, one-on-one conversations that uncover authentic insights. This workshop covers:

• When and why to use IDIs
• Sampling and recruitment strategies
• Interview guide development
• The art of inductive probing to elicit deeper responses
• Practical tips for in-person and remote interviews

Participants will also observe and critique a demonstration interview, and receive ready-to-use tools such as transcription protocols and data management templates.

3. Focus Groups (FGs)

Harness the dynamic energy of group conversation to explore shared and differing perspectives. You’ll learn:

• How to recruit participants and determine sample sizes
• Best practices for developing an FG discussion guide
• Strategies for moderating discussions and managing group dynamics
• How to work effectively with co-facilitators and assistants
• Special considerations for running digital focus groups

The session includes a demonstration FG and practical templates to simplify documentation and transcription.

4. Thematic Analysis

Transform your raw qualitative data into clear, actionable findings. This hands-on workshop takes you step-by-step through:

• Coding and identifying themes
• Building a codebook and linking themes into a conceptual model
• Using qualitative data analysis (QDA) software
• Writing results for academic and applied audiences

The focus is on producing research that is both rigorous and ready to make an impact.

Why Choose CenterStat for Qualitative Training?

• Expert Instructors – Learn from leaders in statistical and qualitative research education with years of hands-on experience.
• Practical Tools – Every session includes ready-to-use templates, protocols, and guides.
• Flexible Learning – Take the workshops in sequence or choose the ones that meet your current needs.
• Designed for All Levels – From beginners to seasoned researchers, every participant gains actionable skills.
• Real-World Relevance – Training is tailored for applied research, not just theory.

Make Your Research Count

In an era where data is everywhere, the ability to truly understand people—beyond the numbers—is a competitive advantage. The skills you develop through the Applied Qualitative Research series can help you design more effective studies, communicate your findings with greater impact, and ultimately create change that matters.

Whether you are launching your first qualitative study or seeking to expand your analytical toolkit, this series will give you the confidence, clarity, and capability to succeed.

Ready to get started? See upcoming sessions and register today at CenterStat.org.

The post Discover the Power of Qualitative Research appeared first on CenterStat.

At CenterStat, we believe you shouldn’t have to choose between quality and affordability. That’s why we’ve built an online education platform that delivers unmatched excellence, clarity, and value. When compared with other providers, CenterStat doesn’t just compete: we lead.

Here’s why CenterStat is the gold standard for online statistics training, and why thousands of learners around the world choose us as their trusted resource.

1. The Most Affordable Cost for the Highest Quality Training

Let’s face it: many online courses in statistics and data science come with a hefty price tag, often far exceeding their value. At CenterStat, our mission is different. We believe that world-class training should be accessible to everyone, not just those with large research budgets or institutional funding.

That’s why we’ve made our tuition structure among the most affordable in the industry—without compromising on quality. We deliver the depth, rigor, and expertise of a university-level course at a fraction of the cost. It’s not just a better price—it’s better value.

2. Instructors Who Are Leaders in Their Fields

Our instructors aren’t just skilled educators—they’re internationally recognized experts who have made novel contributions to quantitative methods in their respective fields. Every member of our instructional team is deeply experienced, actively publishing, and helping shape the landscape of modern statistical research.

Moreover, many of our faculty have won university-wide and national awards for both teaching and research. They bring that same passion for clarity, depth, and practical application to every workshop we offer.

3. High-Quality Materials with a Consistent Structure

One common complaint about online courses is inconsistency—each course is built differently, leaving learners confused and frustrated. At CenterStat, we take a different approach. Every course is developed within a carefully designed, consistent framework.

Each class includes comprehensive PDF lecture notes, step-by-step computer demonstration notes, fully executable computer code and real-world sample datasets. This level of organization allows you to focus on learning—not on figuring out how the materials fit together. Participants are confident that they will get the same level of quality and comprehensiveness no matter which classes in which they choose to enroll.

4. Self-Paced, Individually Pre-Recorded Video Instruction

Unlike many competitors who simply upload recordings of live classes, CenterStat creates custom, pre-recorded videos specifically designed for self-paced learning. This means there are no audio glitches or disorganized transitions, there are focused explanations tailored to asynchronous learners, and there are clear visual demonstrations, edited for maximum clarity. The result is a far more effective and engaging learning experience than when a live offering is simply recorded and posted for later viewing.

5. Real-Data Demonstrations Across All Major Software

Theory is important, but application is critical. At CenterStat, all of our courses include hands-on demonstrations using real data, not contrived classroom examples. Even better, we support instruction across all major statistical platforms including R, SAS, Stata, SPSS, Mplus, and Python. No matter what tools you use (or new tools you want to learn), our courses meet you where you are and guide you in developing skills that are immediately transferable to your own research or professional projects.

6. Lifetime Access, No Expirations: Ever. You heard us right: Ever. 

When you register for a CenterStat course, you own it for life. All videos, PDFs, code, and datasets remain available indefinitely—so you can revisit the material anytime, at your convenience.

Whether you want to refresh a method in a year or re-watch a demonstration before applying it to new data, CenterStat is always there to support you.

7. Evergreen Content with Free Updates

Most online training platforms treat courses as fixed products: once you buy them, any improvements or expansions are locked behind new fees. Not at CenterStat.

We treat all our major content as evergreen. That means if a course is updated, you get the new content automatically, at no additional cost—regardless of when you registered. You’ll always have access to the most current, relevant training in the field.

8. A Commitment to Free, Public Educational Resources

At CenterStat, we’re committed not just to our students, but to the broader public good. That’s why we’ve created and maintain a rich library of free instructional materials including extensive tutorial videos, lecture notes, coding tutorials, sample code and data, reference guides, and more.

We believe deeply in the broadest possible dissemination of quantitative education, and we make meaningful, high-quality resources available to everyone, regardless of financial status.

9. A Fully Integrated Curriculum

Most online training sites offer a scattered menu of unrelated courses created by different instructors with no overarching plan or coordination. CenterStat offers something better: a carefully integrated curriculum.

Every course is designed to fit within a coherent learning path, allowing you to build from foundational principles to advanced techniques with logical, structured progression. There’s executive oversight of content, which means no redundancy, no gaps, and no guesswork about which course to take next. You can be confident that you receive the same high quality CenterStat training no matter which course or instructor you choose. 

10. A Balanced Focus on Theory and Practice

At CenterStat, we never sacrifice one side of learning for the other. Our courses are carefully designed to strike a perfect balance between statistical theory and hands-on application.

You’ll learn the mathematical and conceptual foundations of each model alongside detailed walkthroughs of how to apply them to real-world data using modern software tools. This dual focus ensures that your understanding is both deep and immediately actionable. This means you’ll understand the source and solution to esoteric error messages (e.g., the dreaded psi is non-positive definite) while at the same time being able to tell a meaningful and compelling story about your findings to the reader. 

11. Rigorous Yet Engaging Instructional Style

Too often, statistical education swings between extremes—either overly dry and abstract, or superficial and oversimplified. Our approach is different. Our instructors present complex concepts in a rigorous yet colloquial style that makes advanced material feel approachable and—even enjoyable and at times downright humorous (just wait for Dan to start showing pictures of Patrick in awkward stages of childhood and adolescence).

We explain not just what to do, but why it works, using plain language, relevant examples, and a touch of humor to keep you engaged.

12. Recognized Excellence in Teaching and Research

Many of our instructors are award-winning educators and researchers, recognized nationally and internationally for their ability to communicate complex ideas clearly and meaningfully. When you learn from CenterStat, you’re learning from the best—not just in statistics, but in the art of teaching statistics. Some platforms allow any instructor to post any material they choose, but at CenterStat we hand pick our instructors from the best in the world and then guide them in developing their materials in the CenterStat Way. 

13. Full Transparency: Detailed Syllabi, Sample Content, and Clear Course Overviews

We understand that enrolling in a course is a commitment. That’s why we provide clear, detailed information about every workshop before you register. This includes full syllabi, sample materials, overview videos, clear learning objectives, software covered, and time expectations. 

We remove the guesswork so you can make confident, informed decisions about your training.

14. Tuition Support for Degree-Seeking Students

We’re proud to support the academic journey. If you’re an undergraduate or graduate student currently enrolled in a degree-granting program, you may qualify for reduced tuition rates.

We believe students shouldn’t have to wait for graduation—or major grant funding—to receive the training they need today. Let CenterStat meet your training goals no matter where you are in your professional development. 

15. Certificates of Completion

Upon completing any CenterStat workshop, you’ll receive a certificate of completion—a valuable credential that demonstrates your commitment to professional development. Whether for your CV, academic portfolio, or professional advancement, your certificate signals real achievement in a rigorous and respected program.

Experience the CenterStat Difference

At a time when online education is more accessible than ever, quality still matters. CenterStat stands apart because we combine deep academic rigor with practical usability, affordable pricing with unparalleled value, and a learner-first mindset that informs everything we do.

From our award-winning instructors to our lifetime access, from real-data software demonstrations to our commitment to educational equity, CenterStat doesn’t just deliver courses: we deliver clarity, confidence, and competence in statistical methods all at an affordable price.

Ready to elevate your statistical skills with the best in the field?

Explore our workshops today—and experience for yourself why CenterStat is trusted by students, scholars, and professionals around the world.

The post Why CenterStat is the Best Choice for Online Statistics Training appeared first on CenterStat.

Built for All Stages of the Research Journey

One of the greatest strengths of our workshops is their accessibility across experience levels. We’ve seen participants ranging from doctoral students and postdocs to established professors and NIH-funded investigators. Each of our programs—whether focused on multilevel modeling, structural equation modeling, longitudinal data analysis, or machine learning—is structured to accommodate varying levels of statistical background while still diving deep into theory and application.

“The key strength was the clarity of communication and the balance between technicalities and intuition. Dr. Curran & Dr. Bauer did not shy away from technical aspects of the content but also communicated about them as intuitively as possible.”
— Workshop participant, Introduction to Structural Equation Modeling

Practical Skills You Can Use Immediately

CenterStat workshops are not passive lectures—they’re dynamic, hands-on experiences. Each session blends foundational theory with real data exercises, ensuring that what you learn translates directly to your work. Whether you’re writing a grant, preparing a manuscript, or designing a study, our goal is to give you tools that make an impact now.

“Craig is easily one of the best statistics professors I have ever had. He has taken an incredibly complex topic that I have previously struggled to understand and apply to my own data and made it possible for me to feel confident in using and interpreting the analyses on my own
— Workshop participant, Modern Missing Data Analysis

Expert Instructors Who Are Also Practitioners

Our instructors are internationally recognized scholars who don’t just teach statistics—they use them every day in their own research. They know the challenges of applying quantitative methods in real-life scenarios, and they bring that experience into every lecture and lab session.

“This is the most interactive online workshop I have ever attended. It kept me very engaged, motivated to learn, and excited to apply techniques and discuss with the group. Greg and Emily are very knowledgeable, passionate, and eager to share their knowledge with attendees. They are fantastic at answering questions promptly, and they made the entire experience conversational, interactive, and exciting.”
— Workshop participant, Applied Qualitative Research

A Supportive Learning Community

When you join a CenterStat workshop, you’re not just gaining new knowledge—you’re joining a network of fellow researchers who are equally passionate about improving their craft. Our sessions foster collaboration, discussion, and long-term academic connections.

We also offer ongoing support after the workshop ends, including access to course materials, helpful guides, and recommendations for continuing education.

Results That Speak for Themselves

CenterStat alumni have gone on to publish in top-tier journals, secure major grants, and teach advanced methods at leading institutions. We take pride in helping researchers unlock their full potential by providing training that’s relevant, rigorous, and rooted in real-world impact.

“This is my third C&B workshop and I’ve really enjoyed all of them. I especially appreciate the attention to the algebra behind models–they’ve help me understand other/new-to-me approaches in method papers.”
— Workshop participant, Latent Curve Modeling

Ready to Level Up Your Quantitative Skills?

Explore upcoming workshop opportunities and see how CenterStat can support your research journey at www.centerstat.org. Whether you’re looking to master new techniques or refresh your knowledge, our expert-led sessions will help you gain the confidence and skills to tackle your most important research questions.

The post How CenterStat Workshops Empower Researchers at Every Level appeared first on CenterStat.

Credits are denominated in US dollars and are redeemable for up to five years from purchase, so purchasing credits is a great way to convert time-limited funds into an extended opportunity for professional development. Credits are fully transferable to multiple individuals without restrictions, making it easy to provide professional development opportunities to yourself or members of your organization.

As our way of trying to help out in tough times, we will also add an extra $100 in credits to your account for every $1000 of training credits you purchase.

Please visit our Training Credits page for complete details.

The post CenterStat Announces the Availability of Training Credits appeared first on CenterStat.

To understand suppression, we first need to remind ourselves of a simple two-predictor multiple regression model. Although throughout this note we focus on two predictors and one outcome, all of the concepts easily generalize to multiple predictors, sets of predictors, and even multiple outcomes (e.g., as might be found in a path analysis or structural equation model). This simple two predictor model can be expressed as

y=b₀+b₁x₁+b₂x₂+r

where y is the outcome, b₀ is the intercept, b₁ and b₂ are the regression coefficients relating x₁ and x₂ to y, respectively, and r is the residual. As always, each regression coefficient is the unique relation between that predictor and the outcome when controlling for (or above and beyond) the effects of the other predictor. In many situations, if two predictors are correlated with one another then the unique relation of one predictor and the outcome controlling for the other predictor is smaller than that same predictor when considered alone. (Spoiler alert: in suppression the opposite occurs, which is what makes it so incredibly weird).

The typical situation is best seen using a Venn diagram. Consider the bivariate relation between x₁ and y where, for the moment, we ignore x₂. In the Venn diagram the bivariate relation between these two variables is denoted as area a. This represents the relation between x₁ and y ignoring the effect of x₂.

Often, however, we are interested in the unique effects of two (or more) predictors (as well as the joint effect, but we ignore this for now). To obtain these, we bring both predictors into the model at the same time. Typically, x₂ is correlated with both x₁ and y. As seen in the diagram, in the presence of the second predictor, the effect of the first predictor, a, is usually smaller than it was before. This is quite natural in that the part of x₁ that is shared with x₂ is removed when assessing the unique relation between x₁ and y, making a smaller. This is business as usual.

However, let’s make things a bit stranger. More than 80 years ago, a brilliant quantitative psychologist named Paul Horst found a situation in which the relation between x₁ and y was actually larger in the presence of x₂ than when assessed in the absence of x₂ (Horst, 1941). Considering the above Venn diagrams, this makes absolutely no sense at all; it actually seems downright impossible. Yet it most definitely exists, and Horst somewhat unfortunately termed this situation suppression. It is unfortunate because x₂ is not suppressing the relation between x₁ and y (which many researchers assume). In actuality, x₂ is suppressing irrelevant variance in x₁ and, by doing so, enhances the relation between x₁ and y. It might have been better to refer to x₂ as an “enhancer” rather than a “suppressor”, but that historical ship has sailed so we are stuck with this terminology.

To understand suppression better, let’s first think about the substantive application in which Horst observed this phenomenon. He was part of a research team evaluating pilots during World War II. The pilots completed paper-and-pencil assessments measuring three types of cognitive reasoning: mechanical, numerical, and spatial. These three measures were then used to predict a score representing piloting ability. However, these measures were not as strongly predictive of piloting ability as had been anticipated. A fourth predictor was then included that was a measure of general verbal ability. Verbal ability was correlated with each of the three reasoning measures (as would be expected), but was not correlated with piloting ability (as would also be expected). Consistent with expectations, when verbal ability was added to the regression model it did not uniquely predict piloting ability. However, quite unexpectedly, the effects of all three reasoning measures were markedly larger compared to the model in which verbal ability was omitted. Horst determined that the reason was that verbal ability was removing irrelevant information from the three reasoning measures (that is, suppressing the part of the variance in reasoning that was related to verbal ability but unrelated to piloting ability) and this in turn enhanced the relations between what was left over in the reasoning measures and pilot ability, i.e., the unique relations.

It is helpful to consider a simple hypothetical example. We will define x₁ to be the predictor variable of interest (say mechanical reasoning) and x₂ the suppressor variable (say verbal ability). The simplest pattern of correlations consistent with traditional suppression is when x₁ is correlated with y, x₁ is correlated with x₂, and x₂ is not correlated with y. Of course, in any sample data there will rarely be a zero correlation between the suppressor and the outcome, but it might still obtain some negligible value. Let’s further say that the correlation between x₁ and y is .25, between x₁ and x₂ is .70, and between x₂ and y is zero. If we consider a model in which only x₁ predicts y, the standardized regression coefficient for x₁ is equal to .25 with a squared semi-partial correlation of .06 (that is, x₁ uniquely accounts for 6% of the variance in y). However, if we add x₂ as a second predictor, the standardized regression coefficient for x₁ increases to .49 and the squared semi-partial correlation doubles to .12 (that is, x₁ now uniquely accounts for 12% of the variance in y). The unique effect of x₁ on y when controlling for x₂ is markedly stronger than the bivariate relation of x₁ with y, the hallmark of suppression.

What on Earth is going on? Of course, the presence or absence of the suppressor does not change the bivariate relation between the predictor and the outcome (the correlation between x₁ and y is always .25). However, what the suppressor does change is the unique variability in the predictor that is available to be related to the outcome.

We can see this in a simple re-arrangement of the Venn diagram (the Venn diagram is an imperfect representation of the underlying mathematics, but visually gives a sense what is happening here). This shows that the suppressor, x₂, correlates with the predictor, x₁, as indicated by area b, but does not correlate with the outcome (reflected in the lack of overlap of the circles for x₂ and y ). Further, we can see that controlling for the suppressor reduces (or suppresses) part of the variance in x₁ that is unrelated to y (area b). This, in turns, enhances the proportional relation between x₁ and y (represented by area a). This is the core of suppression.

There have been dozens of papers written on suppression following Horst’s initial discovery, and we note several of these below. Many of these propose specific subtypes of suppression and describe under what unique conditions these might be encountered in practice. However, a concise general definition was given by Conger (1974) who wrote “A suppressor variable is defined to be a variable which increases the predictive validity of another variable (or set of variables) by its inclusion in a regression equation. This variable is a suppressor only for those variables whose regression weights are increased.” Importantly, this means that a variable is not inherently a suppressor in and of itself. Instead, a suppressor is defined by the impact it has on other variables in the model. That is, a variable might be a suppressor in one model but not in another.

This brings us to two initial points. First, there is nothing about suppression that is magical, mysterious, or misunderstood. The papers noted below explain in gory detail exactly what suppression is and under what conditions it exists, so be suspect of a paper that says “Suppression is a long-misunderstood issue…”. It is not. Initially confusing? Yes. Misunderstood? No.

Second, suppression is not some unavoidable artifact of measurement or estimation but instead can be fully accounted by substantive theory. Horst’s example is just one of many in the literature, nearly all of which make perfect sense within a given theoretical framework. As such, it is often beneficial to think about potential suppressors during the design phase of a study so that all relevant variables can be included in the analysis.

However, our third and final point is a bit of a punch in the face: we must consider two additional competing explanations for the role of a third variable in our models: confounding and mediation.

By far the clearest treatment of this was given by MacKinnon, Krull and Lockwood (2000) in a title that could not be more on point: Equivalence of the Mediation, Confounding, and Suppression Effect. You nearly don’t have to read the paper given the title. The paper opens with, “Once a relationship between two variables has been established, it is common for researchers to consider the role of a third variable in this relationship.” This is precisely what we have considered thus far. But to better see this point, we move from Venn diagrams to path diagrams. Let’s first consider the two-predictor regression that we have discussed up to this point:

This shows the usual expression of two correlated predictors and one outcome. Note that there are three measured variables, and these are all related to one another (the curved arrow reflects the correlation between the two predictors, and the two one-headed arrows reflect the partial regression coefficients). In a suppression situation, x₁ enhances the relation between x₁ and y.

However, with a simple re-arrangement of the diagram we get what is called confounding:

Note that all we have done is changed the correlation between the two predictors to a regression coefficient and now x₂ is a confounder in that it predicts both x₁ and y. This is the situation that is so fun to teach because we can give examples such as the number of fire trucks sent to a fire is positively correlated to the amount of damage done at the fire; but when the confounder of severity of fire is included, there is no relation between number of trucks and damage. Importantly, whereas a suppressor enhances the relation between x₁ and y, including a confounder as a second regressor reduces this same relation.

Finally, re-directing one arrow in the above path diagram results in mediation:

Now x₂ explains the relation between x₁ and y. For example, the predictor might be parent’s alcohol use, the outcome is the child’s alcohol use, and the mediator is impaired parenting. The inference is that the parent’s alcohol use impairs their own parenting behavior, and this in turn increases the probability that the child will drink alcohol themselves. In sum, suppression enhances the relation between a predictor and the outcome, confounding reduces the relation, and mediation explains the relation. How the heck do we differentiate among the three? MacKinnon et al. (2000) argue that you do not, and they demonstrate that the statistical tests of these three effects are all identical: each model is a simple re-expression of the others. They conclude the paper saying, “The statistical procedures provide no indication of which type of effect is being tested. That information must come from other sources.” The “other sources” to which they refer are prior knowledge and theory. All three effects are statistically isomorphic, and only theory can discern which most likely holds in the population. Further differentiation might also be possible by moving to experimental or longitudinal designs that allow for better testing of hypotheses about causal pathways.

Suggested Readings

                Conger A.J. (1974). A revised definition for suppressor variables: A guide to their identification and interpretation. Educational Psychological Measurement, 34, 35–46.

                Horst P. (1941). The role of predictor variables which are independent of the criterion. Social Science Research Council Bulletin, 48, 431–436

                MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding and suppression effect. Prevention Science, 1, 173-181.

                Tzelgov, J., & Henik, A. (1991). Suppression situations in psychological research: Definitions, implications, and applications. Psychological Bulletin, 109, 524-536.

                Velicer W.F. (1978). Suppressor variables and the semipartial correlation coefficient. Educational and Psychological Measurement, 38, 953–958

The post What is a suppressor variable, and how does this differ from confounding and mediation? appeared first on CenterStat.

This same approach to decision making appears routinely in daily life. For example, we might want to know if a person referred to a clinic is likely to be diagnosed with major depression or not: they either are truly depressed or not (which is unknown to us) and we must decide based on some brief screening test if we believe it is likely that they suffer from depression. Or we might want to know if it is likely someone has a medical diagnosis that requires a more invasive biopsy procedure. Or, one that is near and dear to us all, we may want to know if we do or do not have COVID. There is a “true” condition (you really do or really do not have COVID) and we obtain a positive or negative result on a rapid test we bought for ten dollars from Walgreens. We have precisely the same four possible outcomes as shown above, two that are correct (it says you have COVID if you really do, or you do not have COVID if you really do not) and two that are incorrect (it says you have COVID when you really don’t, or you do not have COVID if you really do).

This is such an important concept that there are specific terms that capture these possible outcomes. Sensitivity is the probability that you will receive a positive rapid test result if you truly have COVID (the probability of a true positive for those with the disease). Specificity is the probability that you will receive a negative rapid test result if you truly do not have COVID (the probability of a true negative for those without the disease). One minus sensitivity thus represents the error of obtaining a false negative result, and one minus specificity represents the error of obtaining a false positive result. However, all of the above assumes that the rapid test has one of two outcomes: the little window on the rapid test either indicates a negative or a positive result. But this begs the very important question, how does the test know? That is, the test is based on a continuous reading of antigen levels yet some scientist in some lab decided that an exact point on this continuous antigen scale would change the test result from negative to positive. Determining this ideal point on a continuum is one of the many uses for ROC analysis.

ROC stands for receiver operating characteristic, the history of which can be traced back to the development of radar during World War II. Radar was in its infancy and engineers were struggling to determine how it could best be calibrated to maximize the probability of identifying a real threat (an enemy bomber, or a true positive) while minimizing the probability of a false alarm (a bird or a rain squall, or a false positive). The challenge was where to best set the continuous sensitivity of the receiver (called gain) to optimally balance these two outcomes. In other words, there was an infinite continuum of possible gain settings and they needed to determine a specific value that would balance true versus false readings. This is precisely the situation in which we find ourselves when using a brief screening instrument to identify depression or blood antigen levels to identify COVID.

To be more concrete, say that we had a 20-item screening instrument for major depression designed to assess whether an individual should be referred for treatment or not, but we don’t know at what specific score a referral should be made. We thus want to examine the ability of the continuous measure to optimally discriminate between true and false positive decisions across all possible cut-offs on the continuum. To accomplish this, we gather a sample of individuals with whom we conduct a comprehensive diagnostic workup to determine “true” depression, and we give the same individuals our brief 20-item screener and obtain a person-specific scale score that is continuously distributed. We can now construct what is commonly called a ROC curve that plots the true positive rate (or sensitivity) against the false positive rate (or one minus specificity) across all possible cut-points on a continuous measure. That is, we can determine how every possible cut-point on the screener discriminates between those who did or did not receive a comprehensive depression diagnosis.

To construct a ROC curve, we begin by creating a bivariate plot in which the y-axis represents sensitivity (or true positives) and the x-axis represents one minus specificity (or false positives). Because we are working in the metric of probabilities each axis is scaled between zero and one. We are thus plotting the true positive rate against the false positive rate across the continuum of possible cut points on the screener. Next, a 45-degree line is fixed from the origin (or 0,0 point) to the upper right quadrant (or 1,1 point) to indicate random discrimination; that is, for a given cut-point on the continuous measure, you are as likely to make a true positive as you are a false positive. However, the key information in the ROC curve is superimposing the sample-based curve that is associated with your continuous screener; this reflects the actual true-vs-false positive rate across all possible cut-offs of your screener. An idealized ROC curve (drawn from Wikipedia) is presented below.

If the screener has no ability to discriminate between the two groups, the sample-based ROC curve will fall on the 45-degree line. However, that rarely happens in practice; instead, the curve capturing the true-to-false positive rates across all possible cut-points will lie above the 45-degree line indicating that the test is performing better than chance alone. The further the ROC curve deviates from the 45-degree line, the better able the screener is to correctly assign individuals to groups. At the extreme, a perfect screener will fall in the upper left corner (the 0,1 point) indicating all decisions are true positives and none are false positives. This too rarely if ever occurs in practice, and a screener will nearly always fall somewhere in the upper-left area of the plot.

But how do we know if our sample-based curve is meaningfully higher than the 45-degree line? There are many ways that have been proposed to evaluate this, but the most common is computing the area under the curve, or AUC. Because the plot defines a unit square (that is, it is one unit wide and one unit tall), 50% of the area of the square falls below the 45-degree line. Because we are working with probabilities, we can literally interpret this to mean that a there is a 50-50 chance a randomly drawn person from the depressed group has a higher score on the screener than a randomly drawn person from the non-depressed group. This of course reflects that the screener has no better than random chance of correctly specifying an individual. But what if the AUC for the screener was say .80? This would reflect that there is a probability of .8 that a randomly drawn person from the depressed group will have a higher score on the screener than a randomly drawn person from the non-depressed group. In other words, the screener is able to discriminate between the two groups at a higher rate than chance alone. But how high is high enough? There is not really a “right” answer, but conventional benchmarks are that AUCs over .90 are “excellent”, values between .70 and .90 are “acceptable” and values below .70 are “poor”.  Like most general benchmarks in statistics, these are subjective, and much will ultimately depend on the specific theoretical question, measures, and sample at hand. (We could also plot multiple curves to compare two or more screeners, but we don’t detail this here.)

Note, however, that the AUC is a characteristic of the screener itself and we have not yet determined the optimal cut-point to use to classify individual cases. For example, say we wanted to determine the optimal value on our 20-item depression screener that would maximize the true positives and minimize the false positives in our referral for individuals to obtain a comprehensive diagnostic evaluation. Imagine that individual scores could range in value from zero to 50 and we could in principle set the cut-off value at any point on the scale. The ROC curve allows us to compare the true positive to false positive rate across the entire range of the screener and estimate what the true vs. false positive classification at each and every value of the screener. We then can select the optimal value that best balances true positives from false positives, and that value becomes are cut-off point at which we demarcate who is referred for a comprehensive diagnostic evaluation and those who are not. There are a variety of methods for accomplishing this goal, including computing the closest point at which the curve approaches the upper-left corner, the point at which a certain ratio of true-to-false positives is reached, and using more recent methods drawn from Bayesian estimation and machine learning. Some of these methods become quite complex, and we do not detail these here.

Regardless of method used, it is important to realize that the optimal cut-point may not be universal but varies by one or more moderators (e.g., biological sex or age) such that one cut-point is ideal for children and another for adolescents. Further, the ideal cut-point might be informed by the relative cost of making a true vs. false positive. For example, a more innocuous example might be determining if a child might benefit from additional tutoring in mathematics compared to a much more severe determination of whether an individual might suffer from severe depression and be at risk for self-harm. Different criteria might be used in determining the optimal cut-point for the former vs. the latter. Importantly, this statistical architecture is quite general and can be applied across a wide array of settings within the social sciences and offers a rigorous and principled method to help guide optimal decision making. We offer several suggested readings below.

Suggested Readings

Fan, J., Upadhye, S., & Worster, A. (2006). Understanding receiver operating characteristic (ROC) curves. Canadian Journal of Emergency Medicine, 8, 19-20.

Hart, P. D. (2016). Receiver operating characteristic (ROC) curve analysis: A tutorial using body mass index (BMI) as a measure of obesity. J Phys Act Res, 1, 5-8.

Janssens, A. C. J., & Martens, F. K. (2020). Reflection on modern methods: Revisiting the area under the ROC Curve. International journal of epidemiology, 49, 1397-1403.

Mandrekar, J. N. (2010). Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology, 5, 1315-1316.

Petscher, Y. M., Schatschneider, C., & Compton, D. L. (Eds.). (2013). Applied quantitative analysis in education and the social sciences. Routledge.

Youngstrom, E. A. (2014). A primer on receiver operating characteristic analysis and diagnostic efficiency statistics for pediatric psychology: we are ready to ROC. Journal of Pediatric Psychology, 39, 204-221.

The post What are ROC curves and how are these used to aid decision making? appeared first on CenterStat.

To start, there is often much confusion over what constitutes intensive longitudinal data (or ILD), in large part because there exists no formal definition that separates ILD from other types of longitudinal data. That said, ILD tends to fall between two traditional data structures obtained from alternative designs: panel data and time series data. It’s useful to first consider these traditional structures to see how several of their features will combine within ILD.

Historically, the most common method for gathering longitudinal data in psychology and the social and health sciences has been the panel design. Typically, a panel design involves assessing a large sample of subjects (say 200 or more) at a much smaller number of time points (say three to six) that tend to be widely spaced in time (say six or 12 months or more). Panel data are often used to empirically examine long-term trajectories of change that might span multiple years, and common analytic methods include the standard latent curve model or a multilevel growth model. (See our prior Help Desk entry on the relation between the LCM and MLM).

A second type of longitudinal design, commonly used in economics among other areas, is the time series design, which resides at the opposite end of the continuum from the panel design. More specifically, a time series design is often based on just a single unit that is repeatedly assessed a very large number of times (say 100 to 200 or more) at intervals that tend to be close together in time (say daily or even hourly). Time series data are often used to empirically examine short-term dynamic processes that might unfold hour-by-hour or day-by-day (e.g., the daily closing cost of the S&P500) and many specialized analytic methods exist to fit models to these highly dense data.

ILD tends to fall between the two extremes of panel data on one end and time series on the other. More specifically, ILD tends to have fewer subjects than panel data but more than time series (say 50 or 100 subjects) and more time points than panel data but possibly fewer than time series (say 30 or 40 assessments). Data might be captured using wearable technology (e.g., heart rate or blood pressure monitors) or by sending random prompts throughout the day via smart phones or other electronic devices (e.g., a tone sounds on a smart phone three times throughout the day and an individual is prompted to respond to a brief feelings survey). As a hypothetical example, a study might be designed to randomly measure nicotine cravings and cigarette use in a sample of 50 individuals four times per day for a two week period resulting in 56 assessments on each individual, thus falling between traditional panel and time series designs in structure.

In the spirit of be careful what you ask for, once you obtain intensive longitudinal data you must then select an optimal modeling strategy to test your motivating hypotheses, and this is not always an easy task. To begin, some longitudinal models that we are familiar with from panel data simply will not work with ILD. Consider the latent curve model (LCM): because the LCM is embedded within the structural equation model, each observed time point is represented by a manifest variable in the model. This works well if the model is fit to annual assessments of some outcome (say antisocial behavior at age 6, 7, 8, 9 and 10) where each age-specific measure serves as an indicator on the underlying latent curve factor. However, the LCM rapidly breaks down with higher numbers of repeated measures in which only one observation may have been obtained at any given assessment (e.g., 9:15am, 9:52am, and so on). For our prior example with 56 repeated assessments taken on 50 subjects, the LCM is simply not an option.

We can next consider the multilevel model (MLM) and it turns out that this option works quite well for many ILD research applications. (See our Office Hours channel on YouTube for a lecture on the MLM with repeated measures data). The MLM approaches the complex ILD structure as nested data in which repeated assessments are nested within individual. Interestingly, unlike the standard LCM, the MLM can be applied to both more traditional panel data and to ILD. The reason is that, whereas the LCM incorporates the passage of time into the factor loading matrix and requires an observed variable at each assessment, the MLM incorporates the passage of time as a numerical predictor in the regression model. As such, the MLM can easily allow for highly dense (meaning many time points) and highly sparse (meaning few or even one assessment is shared by any individual at any given time point) data without problem. (The LCM can under certain circumstances be contorted to accommodate some of these features as well, but the MLM does this seamlessly). However, there are several complications that must be addressed when fitting an MLM to intensive longitudinal data that do not commonly arise in panel data.

The first issue is what is called serial correlation of the residuals for the repeatedly measured outcome.  With apologies for the technical terminology, this means is that for a given person, when there is a “bump” at one timepoint, that tends to carry over to the next time point too.  For instance, say a person’s average heart rate is 72 BPM. I measured this person at 9:10am and 9:26am. What I don’t know is that this person was late for their 9:00am job, which lead them to move faster and increased their stress, and they had only just arrived at 9:10am.  This manifested in a heart rate of 91 BPM at 9:10 and 83 BPM at 9:26. The initial bump has thus not entirely dissipated by the second assessment.

Serial correlation is often not of importance in panel data because these perturbations have long since washed out (the residual correlation goes to zero over the long lags). A person’s heart rate might be higher than usual when I assess them at age 26 because they had a second shot of espresso or got in an argument with a colleague at work, but the effect of the espresso or argument has long since worn off by the time I reassess them at age 27.  Of course, even with panel data the repeated measures are correlated, but not because of serial correlation of within-person residuals but because of individual differences in level and change over time.  For instance, some people have consistently higher heart rates and others have consistently lower heart rates and this stability will lead to across-person positive correlations in repeated measures. We typically model these individual differences in level and change via latent growth factors / random effects when fitting LCMs / MLMs. Such individual differences may be an important source of correlation in ILD too, but we also have to contend with the serially correlated residuals. Although an added complexity, the MLM is quite well suited at incorporating serial correlations such as these. Complex error terms can be defined among the time-specific residuals such as auto-regressive, Gaussian decay, spatial power, or Toeplitz structures. It is very important these serial correlations be represented in the model if needed both to gain insights into the phenomenon under study and to ensure that other parameter estimates of interest are not biased.

A second issue that often arises in ILD is the presence of cycles or transition points that might occur during the assessment period. For example, daily measures taken over several weeks may vary as a function of weekday vs. weekend (e.g., if studying college drinking) or might cycle regularly throughout a day (e.g., hourly heart rate data varying as a function of waking to sleeping and back to waking). Although such cycles and transition points might be present in panel data as well, these are less likely to occur because there are typically fewer time-linked assessments and these tend to aggregate over longer durations (e.g., if we ask “over the past 30 days” to obtain monthly alcohol use levels, these ratings will implicitly smooth over weekday-weekend differences in daily alcohol use). In contrast, multiple cycles might be observed in ILD spanning a 50 or 60 time point series.

Finally, a third issue is the distinction between within- versus between-person effects. Often ILD is collected with the idea of assessing processes as they unfold in real time for individual participants (“life as lived”). For instance, we might be interested in using ILD to test a negative reinforcement hypothesis for alcohol use. That is, we wish to test the proposition that people drink more than they typically do when they are experiencing increased negative affect under the expectation that this will reduce their negative affect. Using a daily diary study, we measure negative affect each day and alcohol use each night and we build a model to predict alcohol use from negative affect. To fully assess the negative reinforcement hypothesis, we must differentiate the within-person effect (e.g., when my negative affect is higher than usual I drink more than is typical for me)  from any between-person correlation that may also exist (e.g.,  that people who have higher negative affect in general tend to drink more in general).  Fortunately, with the MLM we have well developed methods for separating within- and between-person effects, although there are some complications to consider (see our prior help desk post specifically on this issue)

The MLM is thus well suited to address all of these complexities that commonly arise in intensive longitudinal data. Once incorporated, the MLM offers many of the very same advantages as when applied to panel data: time-varying predictors can be incorporated at level-1 with either fixed or random effects, time invariant predictors can be incorporated at level-2, and interactions can be estimated within or across levels of analysis. However, there are two key limitations of the MLM that may or may not arise in a given application. The first is that, similar to the traditional general linear model, the MLM assumes all measures are error-free and all observed variance is “true” variance. This is often (if not always) an unrealistic assumption and violation of this assumption can lead to significant biases in the estimated results. The second is that the MLM only allows for one dependent variable at a time and is thus limited to the estimation of unidirectional effects. Say that you are interested in testing the reciprocal relations between depression during the day and substance use that evening, and you obtain multiple daily measures spanning a week of time. The MLM allows for the estimation of the prediction of substance use from depression, but not the simultaneous estimation of the reciprocal prediction of depression from substance use. As such, the MLM is only evaluating one part of the research hypotheses at hand.

However, recent developments have introduced a new analytic procedure that combines elements of the MLM, the SEM, and time series models called the dynamic structural equation model (or DSEM). The DSEM functionally picks up where the MLM leaves off, but expands the model to potentially include latent factors (to estimate and remove measurement error) and multiple dependent variables (to estimate reciprocal effects between two or more variables over time). DSEM is a recent development and much has yet to be learned about best practices in applied research settings, but it represents a significant development in our ability to fit complex models to ILD.

Want to learn more?  We recently had the honor of being invited to provide a series of three lectures on intensive longitudinal data analysis for the American Psychological Association and we have posted our lecture materials in the resources section of the CenterStat home page (https://centerstat.org/apa-ild/). The first session discusses the challenges and opportunities of ILD; the second focuses on the analysis of ILD using the multilevel model; and the third focuses on the analysis of ILD using the dynamic structural equation model. In addition to those resources, below are several suggested readings on the design, collection, and analysis of intensive longitudinal data. Asynchronous access to CenterStat workshops on Multilevel Modeling and Analyzing Intensive Longitudinal Data is also available to those who might wish to register for additional training. You can also check our workshop schedule for upcoming live offerings.

Good luck with your work!

                Asparouhov, T., Hamaker, E. L., & Muthén, B. (2018). Dynamic structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 25, 359-388.

                Asparouhov, T., & Muthén, B. (2020). Comparison of models for the analysis of intensive longitudinal data. Structural Equation Modeling: A Multidisciplinary Journal, 27, 275-297.

                Bolger, N., & Laurenceau, J. P. (2013). Intensive longitudinal methods: An introduction to diary and experience sampling research. Guilford Press.

                Hamaker, E. L., Asparouhov, T., Brose, A., Schmiedek, F., & Muthén, B. (2018). At the frontiers of modeling intensive longitudinal data: Dynamic structural equation models for the affective measurements from the COGITO study. Multivariate Behavioral Research, 53, 820-841.

                Hoffman, L. (2015). Longitudinal analysis: Modeling within-person fluctuation and change. Routledge.

                McNeish, D., & Hamaker, E. L. (2020). A primer on two-level dynamic structural equation models for intensive longitudinal data in Mplus. Psychological Methods, 25, 610-635.

                McNeish, D., Mackinnon, D. P., Marsch, L. A., & Poldrack, R. A. (2021). Measurement in intensive longitudinal data. Structural Equation Modeling: A Multidisciplinary Journal, 28, 807-822.

                Walls, T. A., & Schafer, J. L. (Eds.). (2006). Models for intensive longitudinal data. Oxford University Press.

The post What exactly qualifies as intensive longitudinal data and why am I not able to use more traditional growth models to study stability and change over time? appeared first on CenterStat.