JohnXie的个人博客分享 http://blog.sciencenet.cn/u/JohnXie

博文

假如我们正确地解读p-值

已有 663 次阅读 2024-8-5 19:39 |个人分类:对统计推断及统计显著性问题的评述与讨论|系统分类:观点评述

假如我们正确地解读p-值,我们就不得不承认以下这些事实。

·        p 值是给定数据下原假设成立的概率吗?不是,因为 p 值是指在原假设成立的情况下,观察到比所观察到的统计量及其更极端情况的概率。因此,除非我们知道原假设的先验分布,否则 p 值无法告诉我们给定样本数据下原假设成立的概率。

·        p 值是给定数据下备择假设成立的概率吗?不是,请参见上面问题的答案。

·        如果原假设被拒绝,那么 p 值是你错误地拒绝了原假设的概率吗?不是,错误拒绝原假设的概率是由 α 给出的 I 型错误率。此外,α 定义的 I 型错误率的概念实际上并不适用于基于单一组样本数据分析结果就做出结论的研究模式(例如,它只适用于 Neyman-Pearson 假设检验范式,该范式是基于从同一目标总体中抽取一系列随机样本而得出分析结果的方法)。

·        p 值是给定原假设成立的情况下观察到我们数据样本的概率吗?不是,如果原假设的分布是连续的,那么观察到我们数据样本的概率等于零;如果是离散分布,那么观察到我们数据样本的概率等于 1/N,其中 N 是所有可能结果的数量。

·        如果重复同样的实验,是否会得到相同的 p 值?不是,实际上,在原假设下,p 值服从均匀分布。

·        小的 p 值是否表明原假设不太可能成立,或者在原假设下发生了罕见事件(即这种罕见事件是由于随机机会发生的)?不是,因为小的 p 值可能是许多原因造成的(不仅仅是上述的两个原因)。小的 p 值可能是由于(1)样本量非常大;(2)原假设是错误的;(3)在原假设成立的情况下发生了罕见事件;(4)所要求的假设条件不成立,例如随机抽样条件不成立等等。

·        两个研究中的相同 p 值是否提供了相同程度的拒绝原假设的证据?一般来说,答案是否定的。只有在非常罕见的情况下,如果两个研究中的所有内容和所制定的假设都完全相同,包括样本量,相同的 p 值才可能提供相同程度的证据来拒绝原假设。在任何其他情况下,p 值是难以相互比较的,也无法得出任何结论。

一句话总结:p 值只能为科学推断分析提供非常有限的信息,统计显著性的概念在大多数情况下是不必要的,并且常常误导我们的科学研究发现的结论。

以下是以上中文信息所对应的英文版本:

- Is the p-values the probability that the null hypothesis is true given the data? No, because a p-value is the probability of observing more extreme values than the test statistic, if the null hypothesis is true. therefore, a p-value cannot tell us the probability of the null hypothesis is true given the sample data unless we know the prior distribution of the null hypothesis.

- Is the p-values the probability that the alternative hypothesis is true given the data? No, see the answers to the question above.

- If the null hypothesis is rejected, is the p-values the probability of your rejection error? No, the rejection error is the type I error given by alpha. Furthermore, the concept of type I error defined by alpha does not really apply in a single study setting (e.g., it is only applicable in the Neyman-Pearson approach of hypothesis testing which refer to a sequence of random samples from the same target population).

- Is the p-values the probability to observe our data sample given the null hypothesis is true? No, the probability to observe our data sample given the null hypothesis equals zero if the null distribution is continuous or 1/N where N is the number of all possible outcomes if it is a discrete distribution.

- If one repeats an experiments does one obtain the same p-values? No, actually, under the null hypothesis, p-values follow a uniform distribution.

- Does a small p-value indicate either the null hypothesis is unlikely to be true or a rare event has happened under the null hypothesis (namely such rare event happened due to random chance)? No, because a small p-value can be the result of many causes (other than the above mentioned binary causes). A small p-value could be due to (1) a very large sample size; (2) null hypothesis is false; (3) a rare event happened while the null hypothesis is true; (4) assumption conditions were violated, e.g., the random sampling condition does not hold.

- Does the same p-values from two studies provide the same evidence against the null hypothesis? In general the answer is No. The same p-values from different studies may provide the same evidence against the null hypothesis only in the very rare case if everything in the two studies and the formulated hypotheses is identical. This includes also the sample sizes. In any other case, p-values are difficult to compare with each other and no conclusion can be drawn.

In summary, p-values can only provide very limited information for scientific inference purpose and the concept of statistical significance is mostly unnecessary and often misleading.



https://blog.sciencenet.cn/blog-3503579-1445209.html

上一篇:一份针对统计假设检验问题所收集的的综合参考资料清单(2024年更新版)
收藏 IP: 49.187.150.*| 热度|

17 宁利中 杨正瓴 黄河宁 王安良 郑永军 王从彦 孙颉 高宏 王涛 周忠浩 崔锦华 檀成龙 朱晓刚 王成玉 刘进平 尤明庆 钟炳

该博文允许注册用户评论 请点击登录 评论 (3 个评论)

数据加载中...

Archiver|手机版|科学网 ( 京ICP备07017567号-12 )

GMT+8, 2024-8-7 08:23

Powered by ScienceNet.cn

Copyright © 2007- 中国科学报社

返回顶部