小小建議：

• 得0分或看不懂的（6題都沒有答對），建議再唸一次大二統計學

• 16-50分或猜的1-3題答對的，建議再唸一次碩士多變量分析 或進階統計學

• 51-83 分（1-2 題沒有答對），多讀幾次 Haller and Krauss (2002); Gigerenzer (2004); Cohen (1988,1992)

• 100分的，統計Master太厲害了.

Rigor & Relevance(嚴謹和相關)是研究的兩大支柱，在研究相關性上，研究人員大多都會研究自己領域相關的問題，相關性(Relevance)較好，因此，我們聚焦在研究的嚴謹性(Rigor)，關於顯著性p value的解釋，常常有些誤解，例如，可能對P值的過度解讀或統計概念的誤解。多年前的1個統計研究，測試了研究人員正確理解P值能力（Gigerenzer 2004） ，英文原文如下：

Suppose you have a treatment that you suspect may alter performance on a certain task. You compare the means of your control and experimental groups (say 20 subjects in each sample). Further, suppose you use a simple independent means t-test and your result is significant (t = 2.7, d.f. = 18, p = 0.01). Please mark each of the statements below as “true” or “false.” “False” means that the statement does not follow logically from the above premises. Also note that several or none of the statements may be correct.

1.You have absolutely disproved the null hypothesis (that is, there is no difference between the population means).

[] true/false []

2. You have found the probability of the null hypothesis being true.

[] true/false []

3. You have absolutely proved your experimental hypothesis (that there is a difference between the population means).

[] true/false []

4. You can deduce the probability of the experimental hypothesis being true.

[] true/false []

5.You know, if you decide to reject the null hypothesis, the probability that you are

making the wrong decision.

[] true/false []

6. You have a reliable experimental finding in the sense that if, hypothetically, the experiment were repeated a great number of times, you would obtain a significant

result on 99% of occasions.

[] true/false []

英文解答如下：

Which statements are in fact true? Recall that a p-value is the probability of the observed data (or of more extreme data points), given that the null hypothesis H0 is true, defined in symbols as p(D|H0).This definition can be rephrased in a more technical form by introducing the statistical model underlying the analysis (Gigerenzer et al., 1989, chapter 3).

Statements 1 and 3 are easily detected as being false, because a significance test can never disprove the null hypothesis or the (undefined) experimental hypothesis. They are instances of the illusion of certainty (Gigerenzer, 2002).

Statements 2 and 4 are also false. The probability p(D|H0) is not the same as p(H0|D), and more generally, a significance test does not provide a probability for a hypothesis.

The statistical toolbox, of course, contains tools that would allow estimating probabilities of hypotheses, such as Bayesian statistics. Statement 5 also refers to a probability of a hypothesis. This is because if one rejects the null hypothesis, the only possibility of making a wrong decision is if the null hypothesis is true. Thus, it makes essentially the same claim as Statement 2 does, and both are incorrect.

Statement 6 amounts to the replication fallacy (Gigerenzer, 1993, 2000). Here, p=1% is taken to imply that such significant datawould reappear in 99% of the repetitions. Statement 6 could be made only if one knew that the null hypothesiswas true. In formal terms, p(D|H0) is confused with 1−p(D).

To sum up, all six statements are incorrect. Note that all six err in the same direction of

wishful thinking: They make a p-value look more informative than it is.

中文說明：

P值的定義是：當虛無假設（零假設）為真，與樣本結果相同的概率，表示為P（D/H₀）

-----------------------

假設你推測一種治療方法或許科可以改善人們在某種測驗上的績效，你設計了一個實驗研究，比較了對照組和實驗組（假設每組20個樣本）的均數，使用了獨立樣本t檢驗，獲得了以下結果: t = 2.7, d.f. = 18, p = 0.01, 請你以“對”或“錯”回答以下六个題目，“錯”是表示該題內容與前面研究結果的邏輯不相符。

1.     你完全否定了虛無假設（零假設）（虛無假設（零假設）指的是兩個總體的均數没有差異。） 對/錯。

2.     你找到了虛無假設（零假設）為真的概率。對/錯。

3.     你完全證明了實驗假說（實驗假說指的是兩個總體的均數存在差異）對/錯。

4.     你可以推斷出實驗假說為真的概率。對/錯。

5.     假如你决定要拒絕零假設，你已經知道這個决定是錯的概率。對/錯。

6.     你得到了一個可靠的研究結果，原因為：假如這個實驗被重複了非常多次，你將有99%的可能獲得一個顯著的結果。對/錯。

簡要説明 Statements (abbreviated)

1)     H₀ is absolutely disproved

2)     Probability of H₀ is found

3)     H₁ is absolutely proved

4)     Probability of H₁ is found

5)     Probability of Type I error

6)     Probability of replication

正確的答案：這六題的說法都是“錯的”。

P值的定義：當零假設為真時，觀察值的概率用符號表示：P(D/H₀)

題1和題3是錯的，因為一個顯著的測試從未否定H₀或H₁ 。

題2和題4是錯的，P值表示P(D/H₀)，而題2和題4指的是P(H₀/D)，是不一樣的意思。

題5也是錯的，題5參考題3假設的幾率，結果和題2相同，概念都是錯的。

題6是錯的，題6的P = 1%是指顯著資料重複99%，也就是1-P(D)，而原始P值表示P(D/H₀)，不等於1-P(D)。

参考文献：

• Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

• Cohen, J. (1992). A Power Primer. Psychological Bulletin, 112(1), 155-159.

• Haller, H., Krauss, S., 2002. Misinterpretations of significance: a problem students share with their teachers? Methods of Psychological Reseach.

• Gerd Gigerenzer, Mindless statistics, The Journal of Socio-Economics 33 (2004) 587–606