Links to specific topics

(See also under "Labels" at the bottom-left area of this blog)
[ Welcome post ] [ Installation issues ] [ WarpPLS.com ] [ Posts with YouTube links ] [ Model-driven data analytics ] [ PLS-SEM email list ]

Thursday, September 1, 2016

Hypothesis testing with confidence intervals and P values


While P values are widely used in PLS-based SEM, as well as in SEM in general, the statistical significances of path coefficients, weights and loadings can also be assessed employing T ratios and/or confidence intervals. These can be obtained in WarpPLS through the menu option “Explore T ratios and confidence intervals”, which also allows you to set the confidence level to be used. This menu option becomes available after Step 5 is completed.

Related YouTube video: Explore T Ratios and Confidence Intervals in WarpPLS

https://youtu.be/Xao0T2MxJZM

An article is also available explaining how WarpPLS users can test hypotheses based on confidence intervals, contrasting that approach with the one employing P values. A variation of the latter approach, employing T ratios, is also briefly discussed. Below are the reference, link to PDF file, and abstract for the article.

Kock, N. (2016). Hypothesis testing with confidence intervals and P values in PLS-SEM. International Journal of e-Collaboration, 12(3), 1-6.

PDF file

Abstract:
E-collaboration researchers usually employ P values for hypothesis testing, a common practice in a variety of other fields. This is also customary in many methodological contexts, such as analyses of path models with or without latent variables, as well as simpler tests that can be seen as special cases of these (e.g., comparisons of means). We discuss here how a researcher can use another major approach for hypothesis testing, the one building on confidence intervals. We contrast this approach with the one employing P values through the analysis of a simulated dataset, created based on a model grounded on past theory and empirical research. The model refers to social networking site use at work and its impact on job performance. The results of our analyses suggest that tests employing confidence intervals and P values are likely to lead to very similar outcomes in terms of acceptance or rejection of hypotheses.

Note 1:
On Table 1 in the article, each T ratio and confidence interval limits (lower and upper) are calculated through the formulas included below. Normally a hypothesis will not be supported if the confidence interval includes the number 0 (zero).

T ratio = (path coefficient) / (standard error).

Lower confidence interval = (path coefficient) - 1.96 * (standard error).

Upper confidence interval = (path coefficient) + 1.96 * (standard error).

Note 2:
Here is a quick note to technical readers. The P values reported in Table 1 in the article are calculated based on the T ratios using the incomplete beta function, which does not assume that the T distribution is exactly normal. In reality, T distributions have heavier tails than normal distributions, with the difference becoming less noticeable as sample sizes increase.


9 comments:

Kim said...

Hi Ned,

Thanks for this. Will a future version of WarpPLS generate CIs for model coefficients, perhaps alongside the coefficients or in their own right (e.g., 90, 95, and 99%)?

Cheers,
Kim

Ned Kock said...

Thanks for the suggestion Kim.

Sven said...

Most research papers in my filed report the t-values and their confidence interval. Is there any way I can get that information from WarpPLS. The reviewers might not like the idea of reporting only path coefficients and P-values.
Thanks, Sven

Ned Kock said...

Hi Sven. The article linked to the post explains how to easily calculate T values and CIs based on the statistics reported by WarpPLS.

Ned Kock said...

I also added a note with the direct formulas for calculation of the Ts and CIs. I hope this helps!

Anonymous said...

Thank you for the very valuable information that you share in your blog. I have a question about p-values and confidence interval. With 95% confidence intervaly and 0.05 p-value (the WarpPLS defualt), I obtained mostly identical results, but still there were a few cases of disagremment in terms of rejecting or failing to reject the null hypothesis. For example, for a given relationship, (β=0.172, p=0.035<0.05, CI=-0.012 0.357, f2=0.061). All cases of disagreemnt are very similar to the one reported here.

Any insights you could kindly provide here? Is CI not robust enough to detect this signficance? or is it the other way around- that p-values is giving false positives. I am aware that the magnitude of the relationship is very small, but not sure which one to report (p-values or CI) and why?

Ned Kock said...

In this case the CI results would be more consistent with a two-tailed P test, which I would tend to think might lead to a false negative. The reason is conceptually non-trivial, and is explained in the article below (PDF on warppls.com).

Kock, N. (2015). One-tailed or two-tailed P values in PLS-SEM? International Journal of e-Collaboration, 11(2), 1-7.

Rohit Kaushik said...

Hello Kock Sir,
Sir I have encountered negative path coefficient but insignificant result, how can we interpret negative path coefficient???

Ned Kock said...

If it is nonsignificant, the interpretation is that the path is zero at the population level.