Null Hypothesis Significance Testing: What you should know (Part 4)

Arbitrary alpha levels

One of the criticisms of NHST is the use of arbitrary alpha levels (level of significance). The choice of an alpha of 5% (which corresponds to a p value of 0.05 since 5% equates with 0.05 in decimals) is arbitrary.

Fisher suggested that researchers not adhere to a fixed value of alpha, but to determine alpha separately for each null hypothesis. The level of alpha chosen should correspond to the researcher’s threshold to say that chance has been eliminated. However, using different values for different sets of data is likely to cause confusion.

Perhaps using an alpha level of 0.05 as a screening device in small-scale studies that are part of an ongoing series of studies is acceptable. However, when one-time studies with high stakes are conducted, then much smaller alpha levels are recommended (although it is very unusual to take high-stakes decisions based on a single study).


1 thought on “Null Hypothesis Significance Testing: What you should know (Part 4)

  1. Pingback: Confidence Intervals: The basics | communitymedicine4all

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.