Links to specific topics

(See also under "Labels" at the bottom-left area of this blog)
[ Welcome post ] [ Installation issues ] [ WarpPLS.com ] [ Posts with YouTube links ] [ PLS-SEM email list ]

Saturday, January 23, 2010

How are the model fit indices calculated by WarpPLS?


Three of the main model fit indices calculated by WarpPLS are the following: average path coefficient (APC), average R-squared (ARS), and average variance inflation factor (AFVIF).

They are discussed in the WarpPLS User Manual, which is available separately from the software, as a standalone document, on the WarpPLS web site.

The fit indices are calculated as their name implies, that is, as averages of: the (absolute values of the ) path coefficients in the model, the R-squared values in the model, and the variance inflation factors in the model. All of these are also provided individually by the software.

The P values for APC and ARS are calculated through re-sampling. A correction is made to account for the fact that these indices are calculated based on other parameters, which leads to a biasing effect – a variance reduction effect associated with the central limit theorem.

Typically the addition of new latent variables into a model will increase the ARS, even if those latent variables are weakly associated with the existing latent variables in the model. However, that will generally lead to a decrease in APC, since the path coefficients associated with the new latent variables will be low. Thus, the APC and ARS will counterbalance each other, and will only increase together if the latent variables that are added to the model enhance the overall predictive and explanatory quality of the model.

The AFVIF index will increase if new latent variables are added to the model in such a way as to add multicolinearity to the model, which may result from the inclusion of new latent variables that overlap in meaning with existing latent variables. It is generally undesirable to have different latent variables in the same model that measure the same thing; those should be combined into one single latent variable. Thus, the AFVIF brings in a new dimension that adds to a comprehensive assessment of a model’s overall predictive and explanatory quality.

As a final note, I would like to point out that the interpretation of the model fit indices depends on the goal of the SEM analysis. If the goal is to test hypotheses, where each arrow represents a hypothesis, then the model fit indices are of little importance. However, if the goal is to find out whether one model has a better fit with the original data than another, then the model fit indices are a useful set of measures related to model quality.

9 comments:

Ned Kock said...

Update:

From version 2.0 of WarpPLS on the APC is calculated based on the absolute values of the path coefficients.

This post was revised to reflect this change.

NASRADEEN MILAD said...

what is the best R2 in warppls, on my model i got it as 0.16 , by the way i use non-parametric data ?

thanks

Ned Kock said...

They can be as high as .7, but keep in mind that an R-squared of .697 or greater is a sign of lateral collinearity:

Kock, N., & Lynn, G.S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13(7), 546-580.

http://www.scriptwarp.com/warppls/pubs/Kock_Lynn_2012.pdf

Marloes said...

Dear Ned Kock,

Thank you very much for this amazing program; I used it extensively in my research.

I have one question regarding the model fit indices that I cannot solve by myself, even not with the help of the manual, blogs and youtube podcasts.

My model indices for Average block Variance Inflation Factor (AVIF) and Average full collinearity (AFVIF) are ‘INF’. I expected here to be a number, but instead there are these three letters. Could you help me with the interpretation of this, because I have no idea what it means and what I should do about it.

Thank you very very much in advance for your help. If you need any further information from me, such as the other quality indices, please let me know.

Best wishes,
Marloes

Ned Kock said...

Hi Marloes.

These are signs that collinearity is trending toward infinity. This may happen due to the presence of redundant LVs, or use of factor-based PLS algorithms when measurement error is very high (Cronbach's alpha < .5). You may find the following pubs. useful:

Kock, N., & Lynn, G.S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13(7), 546-580.

http://www.scriptwarp.com/warppls/pubs/Kock_Lynn_2012.pdf

Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. Laredo, TX: ScriptWarp Systems.

https://drive.google.com/file/d/0B76EXfrQqs3hYlZhTWdWcXRockU/view

Ned Kock said...

By the way, the 3D figure in Appendix B of Kock & Lynn (2012) shows how collinearity can trend toward infinity under certain combinations of correlations among LVs.

Nurulhuda Ibrahim said...

Dear Dr Kock,
Q1) I use PLS Regression algorithm in the model exploratory phase, and use the Factor based CFM1 for confirmatory analysis phase. However, with factor based, R-squared for the criterion variable is very high (1.28), but reduced to 0.6 if I change to PLS regression. Is this also the sign of lateral collinearity? Other assessment model measurements are well meeting the cut off values.

Q2) My study compares 3 models. The result shows that the control models has better ARS compared to the treatment model (which has more LVs than the control models). Yet, the R-squared for the criterion variable is better for the treatment model. Is it okay to compare R-squared among models rather than using ARS?

Thank you.

Sincerely,
Nurul

Ned Kock said...
This comment has been removed by the author.
Ned Kock said...

Hi Nurulhuda.

Measurement error and composite weights are estimated before the SEM analysis is run, whenever Factor-Based PLS algorithms are used. Measurement error and composite weights play a key role in these algorithms. If at least one measurement error weight is greater than the corresponding composite weight, the user is warned about possible unreliability of results. This happens usually when at least one of the Cronbach’s alpha coefficients associated with the latent variables is lower than 0.5.

Regarding model fit, I tend to favor the use of at least the APC and ARS indices in combination. Typically the addition of new latent variables into a model will increase the ARS, even if those latent variables are weakly associated with the existing latent variables in the model. However, that will generally lead to a decrease in the APC, since the path coefficients associated with the new latent variables will be low. Thus, the APC and ARS will counterbalance each other, and will only increase together if the latent variables that are added to the model enhance the overall predictive and explanatory quality of the model.

In addition to the clarifications above, which may or may not apply to your factor-based analyses (hopefully they do), I hope that the materials linked below can be of use to clarify the issues you raised.

User Manual (links to specific pages):

http://www.scriptwarp.com/warppls/UserManual_v_5_0.pdf#page=46

http://www.scriptwarp.com/warppls/UserManual_v_5_0.pdf#page=50

Kock, N. (2014). Advanced mediating effects tests, multi-group analyses, and measurement model assessments in PLS-based SEM. International Journal of e-Collaboration, 10(3), 1-13.

http://www.scriptwarp.com/warppls/pubs/Kock_2014_UseSEsESsLoadsWeightsSEM.pdf

Kock, N. (2011). Using WarpPLS in e-collaboration studies: Descriptive statistics, settings, and key analysis results. International Journal of e-Collaboration, 7(2), 1-18.

http://www.scriptwarp.com/warppls/pubs/Kock_2011_IJeC_WarpPLSEcollab2.pdf