Which statement about p-values and effect sizes is true?

Which statement about p-values and effect sizes is true?

• A. A small p-value guarantees a large effect size.
• B. A small p-value can occur with a small effect if the sample size is large.
• C. The p-value measures magnitude of the effect.
• D. The effect size determines the p-value directly.

Answer: (B)

P-values reflect evidence against the null hypothesis and depend on both how big the effect is and how precise the estimate is. The precision improves with larger sample sizes, which lowers the standard error of the effect estimate. When the sample size is large, even a small effect can produce a test statistic large enough to yield a very small p-value, because the data are precise enough to detect tiny deviations from no effect. That’s why a small p-value can occur with a small actual effect if the study is large. Conversely, a large effect can fail to give a small p-value if the study is small or highly variable, since precision is then low. The p-value tells you about the strength of evidence against the null given the data, not the magnitude of the effect itself; the effect size is a separate measure of how big the effect is.

P-values reflect evidence against the null hypothesis and depend on both how big the effect is and how precise the estimate is. The precision improves with larger sample sizes, which lowers the standard error of the effect estimate. When the sample size is large, even a small effect can produce a test statistic large enough to yield a very small p-value, because the data are precise enough to detect tiny deviations from no effect. That’s why a small p-value can occur with a small actual effect if the study is large. Conversely, a large effect can fail to give a small p-value if the study is small or highly variable, since precision is then low. The p-value tells you about the strength of evidence against the null given the data, not the magnitude of the effect itself; the effect size is a separate measure of how big the effect is.