Simpson’s paradox and unexpected results

The algorithms used in version 3.0 and later versions of WarpPLS have been revised so as to pick up instances of what is known as “Simpson’s paradox”. As a result, there may be changes in some coefficients and P values, when compared with previous versions.

Often the P value of the ARS fit index will go up, if instances of Simpson’s paradox are present in the model.

Simpson’s paradox is characterized by the path coefficient and correlation for a pair of variables having different signs. In this situation, the contribution of a predictor variable to the explained variance of the criterion variable in a latent variable block is negative.

In other words, if the predictor latent variable were to be removed from the block, the R-squared for the criterion latent variable would go up. A similar effect would be observed if the direction of the causality was reversed.

One widely held interpretation is that Simpson’s paradox could be an indication that the direction of a hypothesized relationship is reversed, or that the hypothesized relationship is nonsensical/improbable.

In the context of WarpPLS analyses, this is more likely to occur when nonlinear algorithms are used and/or full collineary VIFs are high, but may also occur under other conditions.

How are the model fit indices calculated by WarpPLS?

WarpPLS is unique among software that implement PLS-SEM algorithms in that it provides users with a number of model-wide fit indices; arguably more than any other SEM software. Three of the main model fit indices calculated by WarpPLS are the following: average path coefficient (APC), average R-squared (ARS), and average variance inflation factor (AFVIF).

They are discussed in the WarpPLS User Manual, which is available separately from the software, as a standalone document, on the WarpPLS web site.

These fit indices (there are several others) are calculated as their name implies, that is, as averages of: the (absolute values of the ) path coefficients in the model, the R-squared values in the model, and the variance inflation factors in the model. All of these are also provided individually by the software.

The P values for APC and ARS are calculated through re-sampling. A correction is made to account for the fact that these indices are calculated based on other parameters, which leads to a biasing effect – a variance reduction effect associated with the central limit theorem.

Typically the addition of new latent variables into a model will increase the ARS, even if those latent variables are weakly associated with the existing latent variables in the model. However, that will generally lead to a decrease in APC, since the path coefficients associated with the new latent variables will be low. Thus, the APC and ARS will counterbalance each other, and will only increase together if the latent variables that are added to the model enhance the overall predictive and explanatory quality of the model.

The AFVIF index will increase if new latent variables are added to the model in such a way as to add multicolinearity to the model, which may result from the inclusion of new latent variables that overlap in meaning with existing latent variables. It is generally undesirable to have different latent variables in the same model that measure the same thing; those should be combined into one single latent variable. Thus, the AFVIF brings in a new dimension that adds to a comprehensive assessment of a model’s overall predictive and explanatory quality.

Starting in version 6.0 of WarpPLS, new indices are available that allow investigators to assess the fit between the model-implied and empirical indicator correlation matrices. These new indices are available from the "Explore" menu option, after Step 5 is completed. They are discussed on page 26 of the WarpPLS User Manual for version 6.0, and on a video clip (links below).

http://www.scriptwarp.com/warppls/UserManual_v_6_0.pdf#page=26

https://youtu.be/YutkhEPW-CE

Also, these new indices are discussed in the following article (see link below for PDF): Kock, N. (2020). Using indicator correlation fit indices in PLS-SEM: Selecting the algorithm with the best fit. Data Analysis Perspectives Journal, 1(4), 1-4.

https://scriptwarp.com/dapj

As a final note, I would like to point out that the interpretation of the model fit indices depends on the goal of the SEM analysis. If the goal is to test hypotheses, where each arrow represents a hypothesis, then the model fit indices are of some importance, but in a limited way. However, if the goal is to find out whether one model has a better fit with the original data than another, then the model fit indices become more important, and are a useful set of measures related to model quality.

Warped paths become significant in WarpPLS: YouTube video

Yet another new YouTube video is available for WarpPLS:

http://www.youtube.com/watch?v=_NQnVckeBb8

This video shows how path coefficients sometimes go up, and P values become significant, when warping takes place in a structural equation modeling (SEM) analysis using the software WarpPLS.

Since path coefficients are typically associated with hypotheses, which are supported if the paths are found to be significant, this will likely be music to many researchers' ears.

However, it is important to make two important points regarding this effect:

1. Path coefficients are not artificially inflated. They increase simply because the software is taking the nonlinear associations between latent variables into account when estimating path coefficients. Much like a researcher would apply a log(X) transformation to a predictor, before an ordinary regression analysis, if he or she knew that the predictor's relationship with a criterion variable Y was of the type Y=log(X).

2. Path coefficients do not always increase. Due to the nature of standardized partial regression coefficient calculation (the path coefficients, or betas, are standardized partial regression coefficients), when several predictor latent variables (LVs) point a one criterion LV, if one of the predictor LVs increases, some of the others predictor LVs may decrease as a result. In a sense, the predictor LVs "compete" for pieces of the space of variance explained for the criterion LV; if the predictor LVs are correlated, they tend "steal" variance space from each other (so to speak).