Links to specific topics

(See also under "Labels" at the bottom-left area of this blog)
[ Welcome post ] [ Installation issues ] [ WarpPLS.com ] [ Posts with YouTube links ] [ Model-driven data analytics ] [ PLS-SEM email list ]

Saturday, January 23, 2010

How are the model fit indices calculated by WarpPLS?


WarpPLS is unique among software that implement PLS-SEM algorithms in that it provides users with a number of model-wide fit indices; arguably more than any other SEM software. Three of the main model fit indices calculated by WarpPLS are the following: average path coefficient (APC), average R-squared (ARS), and average variance inflation factor (AFVIF).

They are discussed in the WarpPLS User Manual, which is available separately from the software, as a standalone document, on the WarpPLS web site.

These fit indices (there are several others) are calculated as their name implies, that is, as averages of: the (absolute values of the ) path coefficients in the model, the R-squared values in the model, and the variance inflation factors in the model. All of these are also provided individually by the software.

The P values for APC and ARS are calculated through re-sampling. A correction is made to account for the fact that these indices are calculated based on other parameters, which leads to a biasing effect – a variance reduction effect associated with the central limit theorem.

Typically the addition of new latent variables into a model will increase the ARS, even if those latent variables are weakly associated with the existing latent variables in the model. However, that will generally lead to a decrease in APC, since the path coefficients associated with the new latent variables will be low. Thus, the APC and ARS will counterbalance each other, and will only increase together if the latent variables that are added to the model enhance the overall predictive and explanatory quality of the model.

The AFVIF index will increase if new latent variables are added to the model in such a way as to add multicolinearity to the model, which may result from the inclusion of new latent variables that overlap in meaning with existing latent variables. It is generally undesirable to have different latent variables in the same model that measure the same thing; those should be combined into one single latent variable. Thus, the AFVIF brings in a new dimension that adds to a comprehensive assessment of a model’s overall predictive and explanatory quality.

Starting in version 6.0 of WarpPLS, new indices are available that allow investigators to assess the fit between the model-implied and empirical indicator correlation matrices. These new indices are available from the "Explore" menu option, after Step 5 is completed. They are discussed on page 26 of the WarpPLS User Manual for version 6.0, and on a video clip (links below).

http://www.scriptwarp.com/warppls/UserManual_v_6_0.pdf#page=26

https://youtu.be/YutkhEPW-CE

Also, these new indices are discussed in the following article (see link below for PDF): Kock, N. (2020). Using indicator correlation fit indices in PLS-SEM: Selecting the algorithm with the best fit. Data Analysis Perspectives Journal, 1(4), 1-4.

https://scriptwarp.com/dapj

As a final note, I would like to point out that the interpretation of the model fit indices depends on the goal of the SEM analysis. If the goal is to test hypotheses, where each arrow represents a hypothesis, then the model fit indices are of some importance, but in a limited way. However, if the goal is to find out whether one model has a better fit with the original data than another, then the model fit indices become more important, and are a useful set of measures related to model quality.

24 comments:

Ned Kock said...

Update:

From version 2.0 of WarpPLS on the APC is calculated based on the absolute values of the path coefficients.

This post was revised to reflect this change.

Unknown said...

what is the best R2 in warppls, on my model i got it as 0.16 , by the way i use non-parametric data ?

thanks

Ned Kock said...

They can be as high as .7, but keep in mind that an R-squared of .697 or greater is a sign of lateral collinearity:

Kock, N., & Lynn, G.S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13(7), 546-580.

http://www.scriptwarp.com/warppls/pubs/Kock_Lynn_2012.pdf

Marloes said...

Dear Ned Kock,

Thank you very much for this amazing program; I used it extensively in my research.

I have one question regarding the model fit indices that I cannot solve by myself, even not with the help of the manual, blogs and youtube podcasts.

My model indices for Average block Variance Inflation Factor (AVIF) and Average full collinearity (AFVIF) are ‘INF’. I expected here to be a number, but instead there are these three letters. Could you help me with the interpretation of this, because I have no idea what it means and what I should do about it.

Thank you very very much in advance for your help. If you need any further information from me, such as the other quality indices, please let me know.

Best wishes,
Marloes

Ned Kock said...

Hi Marloes.

These are signs that collinearity is trending toward infinity. This may happen due to the presence of redundant LVs, or use of factor-based PLS algorithms when measurement error is very high (Cronbach's alpha < .5). You may find the following pubs. useful:

Kock, N., & Lynn, G.S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13(7), 546-580.

http://www.scriptwarp.com/warppls/pubs/Kock_Lynn_2012.pdf

Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. Laredo, TX: ScriptWarp Systems.

https://drive.google.com/file/d/0B76EXfrQqs3hYlZhTWdWcXRockU/view

Ned Kock said...

By the way, the 3D figure in Appendix B of Kock & Lynn (2012) shows how collinearity can trend toward infinity under certain combinations of correlations among LVs.

Nurulhuda Ibrahim said...

Dear Dr Kock,
Q1) I use PLS Regression algorithm in the model exploratory phase, and use the Factor based CFM1 for confirmatory analysis phase. However, with factor based, R-squared for the criterion variable is very high (1.28), but reduced to 0.6 if I change to PLS regression. Is this also the sign of lateral collinearity? Other assessment model measurements are well meeting the cut off values.

Q2) My study compares 3 models. The result shows that the control models has better ARS compared to the treatment model (which has more LVs than the control models). Yet, the R-squared for the criterion variable is better for the treatment model. Is it okay to compare R-squared among models rather than using ARS?

Thank you.

Sincerely,
Nurul

Ned Kock said...
This comment has been removed by the author.
Ned Kock said...

Hi Nurulhuda.

Measurement error and composite weights are estimated before the SEM analysis is run, whenever Factor-Based PLS algorithms are used. Measurement error and composite weights play a key role in these algorithms. If at least one measurement error weight is greater than the corresponding composite weight, the user is warned about possible unreliability of results. This happens usually when at least one of the Cronbach’s alpha coefficients associated with the latent variables is lower than 0.5.

Regarding model fit, I tend to favor the use of at least the APC and ARS indices in combination. Typically the addition of new latent variables into a model will increase the ARS, even if those latent variables are weakly associated with the existing latent variables in the model. However, that will generally lead to a decrease in the APC, since the path coefficients associated with the new latent variables will be low. Thus, the APC and ARS will counterbalance each other, and will only increase together if the latent variables that are added to the model enhance the overall predictive and explanatory quality of the model.

In addition to the clarifications above, which may or may not apply to your factor-based analyses (hopefully they do), I hope that the materials linked below can be of use to clarify the issues you raised.

User Manual (links to specific pages):

http://www.scriptwarp.com/warppls/UserManual_v_5_0.pdf#page=46

http://www.scriptwarp.com/warppls/UserManual_v_5_0.pdf#page=50

Kock, N. (2014). Advanced mediating effects tests, multi-group analyses, and measurement model assessments in PLS-based SEM. International Journal of e-Collaboration, 10(3), 1-13.

http://www.scriptwarp.com/warppls/pubs/Kock_2014_UseSEsESsLoadsWeightsSEM.pdf

Kock, N. (2011). Using WarpPLS in e-collaboration studies: Descriptive statistics, settings, and key analysis results. International Journal of e-Collaboration, 7(2), 1-18.

http://www.scriptwarp.com/warppls/pubs/Kock_2011_IJeC_WarpPLSEcollab2.pdf

http://puisiku-puisikita.blogspot.com/ said...

It's a great thing to chat you Mr. kock. as you said that three main indicators of goodness of fit are APC, ARS and AFVIF. My question is that i am little bit confused about AFVIF you have mentioned which stands for average variance inflation factor. Is that AFVIF or AVIF which is the part of three main GoF?
Thank you

Ned Kock said...

Hi Revan.

The AVIF index will increase if new latent variables are added to the model in such a way as to add vertical collinearity in the model’s latent variable blocks. The AFVIF index will increase if new latent variables are added to the model in such a way as to add full collinearity into the model (i.e., either vertical or lateral collinearity). Full collinearity is often referred to as “muticollinearity”. High AVIF and AFVIF values may result from the inclusion of new latent variables that overlap in meaning with existing latent variables. It is generally undesirable to have different latent variables in the same model that measure the same underlying construct; those should be combined into one single latent variable. Thus, the AVIF and AFVIF indices bring in new dimensions that add to a comprehensive assessment of a model’s overall predictive and explanatory quality. Because of the way in which these indices are calculated, the AFVIF is not sensitive to variations in collinearity due to the use of nonlinear algorithms. The AVIF, on the other hand, is sensitive to the use of nonlinear algorithms. Therefore it is recommended that both indices, AVIF and AFVIF, be reported in studies, as they are not redundant indices.

There is more on this on page 62 of: Kock, N. (2017). WarpPLS User Manual: Version 6.0. Laredo, TX: ScriptWarp Systems.

The above is available from warppls.com.

http://puisiku-puisikita.blogspot.com/ said...

Hi Mr. Kock,

Thank you very much for your answer. It is really help me but i have conveyed to my colleagues that we should just focus on three items of GoF which are APC, ARS and AVIF for the linear model. Is that right what i've conveyed to them ?

Thank You.

Best Regards,

Rifan Jefri Sunarsono

http://puisiku-puisikita.blogspot.com/ said...

Hi Mr. Kock,

so sorry, what i mean is AFVIF for the linear model.

Best Regards,

Rifan Jefri Sunarsono

Ned Kock said...

These three indices tend to be very telling, but there more indices available, including indices to assess the fit between the model-implied and empirical indicator correlation matrices. For these, see page 25 of Manual (below) available from warppls.com:

Kock, N. (2017). WarpPLS User Manual: Version 6.0. Laredo, TX: ScriptWarp Systems.

Nur Maulida Ahniah said...

Dear Prof. Kock,
I have 4 LVs, 3 of them are reflective model, and 1 is formative model. From UserManual Version 5.0, I knew that there are 10 model fit. Tenenhaus GoF is not recommended for formative model, and SPR, SSR, RSCR, and NLBDCR were an experimental index (my research is not an experimental).
Should I just use 5 model fit (APC, ARS, AARS, AVIF, and AFVIF) or should I use all 10 model fit?

Thank you.

Sincerely,

Ahniah

Ned Kock said...

Hi Ahniah. I suggest reporting all fit indices, as some readers like to see them all. If you want to avoid "crowding" a paper, perhaps the 5 you mentioned should be in the main body of the paper, and the others in an appendix.

Rifki Safrreza (JackerSX) said...

Hi Mr. Kock

I want to ask something about my result.
my model looks good until I found the Average Path Coefficient was p=0.003
and to make that fit it should be <0.001
any reason why can be like that or why to make it fit should be p<0.001
I cannot find the reason on the internet.

Thank you

Ned Kock said...

I am not sure I understand the concern. A P value of 0.003 is statistically significant at the 0.05 level, and thus meets the expectation of good fit with respect to the APC.

Unknown said...

Hi! Mr. Kock

I do have following results of model fit

Average path coefficient (APC) 0.303
Average R-squared (ARS) 0.336
Average adjusted R-squared (AARS) 0.334

Please guide me that how should i report (describe results) in my thesis pertaining to model fit.

Ned Kock said...

Hi Anon. See Table B.2 for an example of reporting format:

http://cits.tamiu.edu/kock/pubs/journals/2015JournalCliodynamics/Kock_2015_Cliodynamics_WheatRiceChina.pdf

Gaby said...

Hi Mr. Kock,

I want to ask about my model fir result. What does "average block VIF not available" mean? my research consisted of 1 independent variable (8 indicators) and 2 dependent variables (each 3 indicators and 2 indicators). Is "not available" okay? and why can it appear "not available" and not numbers? Please help me. Thank you very much, have a nice day!

Ned Kock said...

Hi Gaby. See page 98 of the User Manual:

http://cits.tamiu.edu/WarpPLS/UserManual_v_7_0.pdf

Alda said...

Hi Mr. Kock

I want to ask about modet fit and quality indices, just want to be sure, if my ARS and AARS have P=0.103 and P=0.248 then these two are insignificant, right ?

Thank you.

Ned Kock said...

Generally you would expect P<.05 for significance with the ARS and AARS. So, those Ps you got would be non-significant (this is the preferred term, not "insignificant"). However, in very simple models, such as comparison of means models (see paper below), the use of those indices would be meaningless.

Kock, N. (2013). Using WarpPLS in e-collaboration studies: What if I have only one group and one condition? International Journal of e-Collaboration, 9(3), 1-12.