Statistical significance and effect size tests in SEM: Common method bias and strong theorizing

The article below provides evidence in support of a few very important methodological propositions: (a) we should not do away with classic statistical significance tests, but should combine them with effect size tests, and tests of common method bias; (b) high quality theorizing is very important if we are to profitably use a combination of classic statistical significance, effect size, and common method bias tests; and (c) the full collinearity VIF threshold in common method bias assessment for factor-based PLSF-SEM should be 10, as opposed to the 3.3 number used with classic composite-based PLS algorithms.

Kock, N., & Dow, K. E. (2025). Statistical significance and effect size tests in SEM: Common method bias and strong theorizing. Advances in Management Accounting, 37(1), 95–105.

Link to full-text file for this article:

Statistical significance and effect size tests in SEM: Common method bias and strong theorizing.

Abstract:

We generally acknowledge the problematic nature of classic statistical significance tests based on P-values or confidence intervals. In fact, we demonstrate based on an illustrative model for which we created simulated data, that with low and high statistical power, path coefficients in structural equation modeling whose true values are zero, routinely end up being found to be significantly different from zero at the P < .05 level. However, we argue that we should not do away with classic statistical significance tests, and that these tests can be useful but should be complemented by other methodological tools, including effect size tests, and tests of common method bias. We also argue that high quality theorizing is very important if we are to profitably use a combination of classic statistical significance, effect size, and common method bias tests.

Important note for PLSF-SEM users (repeated below for emphasis):

The full collinearity VIF threshold in common method bias assessment for factor-based PLSF-SEM should be 10, as opposed to the 3.3 number used with classic composite-based PLS algorithms.

Best regards to all!

Conducting a difference-in-differences analysis with PLS-SEM: The classic 2x2 approach

The article below shows how one can conduct a difference-in-differences analysis employing the classic 2x2 approach for this type of analysis, using structural equation modeling via partial least squares (PLS-SEM).

Kock, N. (2024). Conducting a difference-in-differences analysis with PLS-SEM: The classic 2x2 approach. Data Analysis Perspectives Journal, 5(5), 1-8.

Link to full-text file for this and other DAPJ articles:

https://scriptwarp.com/dapj/#Published_Articles

Abstract:

Difference-in-differences analyses often employ a classic 2x2 scenario, which involves two conditions, control and treatment; and two points in time, before and after an intervention that may be tied to one of the conditions. In our analysis, we assess the impact on labor productivity of being in a more technology-intensive US state, instead of a more manufacturing-intensive one. Consistently with the difference-in-differences analysis scenario, we also assess the full latent growth effect of a government-driven age discrimination crackdown, in the technology-intensive state, on the previous effect – of being in a technology-intensive state on labor productivity. We do this by employing a model analyzed in the context of structural equation modeling via partial least squares (PLS-SEM). We also discuss advantages of using PLS-SEM in this scenario; which include assessments of causality, common method bias, and endogeneity.

Best regards to all!

Contributing to the success of PLS in SEM: An action research perspective

The article below discusses how I employed an action research approach, by working closely with WarpPLS users, to contribute to the success of PLS in SEM. A big thank you to WarpPLS users!

Kock, N. (2023). Contributing to the success of PLS in SEM: An action research perspective. Communications of the Association for Information Systems, 52(1), 730-734.

Link to full-text file for this article:

Contributing to the success of PLS in SEM: An action research perspective

Abstract:

I share with Evermann and Rönkkö (2022) the belief that classic composite-based partial least squares path modeling (PLS-PM) presents shortcomings when used to conduct structural equation modeling (SEM) analyses. The shortcomings can be traced back to one fundamental problem, which is that latent variables (LVs) are approximated in PLS-PM as exact linear combinations of their corresponding indicators. In SEM, each LV is in fact a factor; i.e., a linear combination of the indicators and a measurement residual. My approach to addressing the shortcomings of PLS-PM is rather unique among researchers concerned with quantitative methods. I have employed an action research approach, helping investigators employ SEM in their empirical studies. This has led to my development of a widely used software tool for SEM analyses. I illustrate my action research orientation by discussing three recent methodological developments with which I have been closely involved.

Best regards to all!

Robustness tests in PLS-SEM: Video for the Qatar Chapter of the Decision Sciences Institute

The so-called "robustness" tests that some refer to, in connection with PLS-SEM, are generally tests of: nonlinearity, common method bias, and endogeneity. In WarpPLS, nonlinearity can be tested via full latent growth, commom method bias via FCVIFs, and endogeneity via instrumental variables. The video linked below covers most of these issues.

https://youtu.be/I3YYdpdXhII

This video is of a presentation to faculty and students at the Qatar Chapter of the Decision Sciences Institute on the topics of factor-based PLS-SEM (PLSF-SEM), endogeneity, and common method bias. (SEM = structural equation modeling.) Addressing these issues helps with publishing in top-tier information systems and decision sciences journals, among others.

Best regards to all!

Common structural variation reduction in PLS-SEM: Replacement analytic composites and the one fourth rule

The article below explains how one can accomplish a common structural variation reduction, via replacement analytic composites and the one fourth rule, in the context of structural equation modeling via partial least squares (PLS-SEM).

Kock, N. (2021). Common structural variation reduction in PLS-SEM: Replacement analytic composites and the one fourth rule. Data Analysis Perspectives Journal, 2(5), 1-6.

Link to full-text file for this and other DAPJ articles:

https://scriptwarp.com/dapj/#Published_Articles

Abstract:

Path coefficients may be distorted, in the context of structural equation modeling via partial least squares (PLS-SEM), due to excess common structural variation shared in a model. This may be caused by methodological issues; e.g., the use of highly correlated but conceptually distinct latent variables, or common method bias. We discuss a common structural variation reduction procedure using WarpPLS, a leading PLS-SEM software tool. This procedure relies on the creation of analytic composites as replacements for latent variables, where the weights are one fourth of the original path coefficients among the latent variables and their predictors in the structural model, and with signs that are the opposites of the signs of the original path coefficients.

Best regards to all!

Common method bias in PLS-SEM: Video, three articles, and sample dataset

The video linked below provides an overview on how to test for common method bias, in the context of structural equation modeling via partial least squares (PLS-SEM).

https://youtu.be/r5p0zHBqfBs

The articles below explain how one can conduct tests for common method bias in PLS-SEM. The first two articles (particularly the second) discuss the highest full collinearity variance inflation factor (FCVIF) test. The third article discusses Harman’s single factor test.

Kock, N., & Lynn, G.S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13(7), 546-580.

Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal of e-Collaboration, 11(4), 1-10.

Kock, N. (2021). Harman’s single factor test in PLS-SEM: Checking for common method bias. Data Analysis Perspectives Journal, 2(2), 1-6.

Links to PDF files are available (from the "Publications" area of WarpPLS.com):

https://scriptwarp.com/warppls/#Publications

Finally, the site area below (the "Resources" area of WarpPLS.com) provides a sample dataset available to users interested in trying the tests discussed above: "Dataset with and without common method bias".

https://scriptwarp.com/warppls/#Resources

Enjoy!

Harman’s single factor test in PLS-SEM: Checking for common method bias

The article below explains how one can check for common method bias using Harman’s single factor test in the context of structural equation modeling via partial least squares (PLS-SEM).

Kock, N. (2021). Harman’s single factor test in PLS-SEM: Checking for common method bias. Data Analysis Perspectives Journal, 2(2), 1-6.

A link to a PDF file is available ().

Abstract:

Common method bias can be defined, in the context of structural equation modeling via partial least squares (PLS-SEM), as a phenomenon that is caused by the measurement method used in a study, and not by the network of causes and effects connecting the latent variables in the study. We illustrate how Harman’s single factor test of common method bias can be conducted with WarpPLS, a leading PLS-SEM software tool.

Testing for common method bias in PLS-SEM using full collinearity VIFs

Full collinearity variance inflation factors (VIFs) can be used for common method bias tests that are more conservative than, and arguably superior to, the traditionally used tests relying on exploratory factor analyses. Full collinearity VIFs and their use for common method bias tests, as well as other tests, are addressed in the following publications (also available from WarpPLS.com):

Kock, N. (2015). Common method bias in PLS-SEM: A full collinearity assessment approach. International Journal of e-Collaboration, 11(4), 1-10.

PDF file:

https://drive.google.com/file/d/0B76EXfrQqs3hYlZhTWdWcXRockU/view

Kock, N., & Lynn, G.S. (2012). Lateral collinearity and misleading results in variance-based SEM: An illustration and recommendations. Journal of the Association for Information Systems, 13(7), 546-580.

PDF file:

http://www.scriptwarp.com/warppls/pubs/Kock_Lynn_2012.pdf

Essentially, testing for the existence of common method bias through this method entails comparing the full collinearity VIFs calculated by WarpPLS for all latent variables to the threshold of 3.3 (or 5.0, if factor-based algorithms are used). If all full collinearity VIFs are equal to or lower than the threshold, this can be seen as an indication that the model is free from common method bias.