Silly Null Hypotheses

NHST is being misused when it tests a null hypothesis in which the effect can only go in one direction. In instances where p values do not add to the interpretation of results, they are best omitted. Similarly, although p values do not communicate effect size, many mistakenly imply that they do.

Many of the null hypotheses tested in literature are false only in the statistical sense, but could be treated as true in a practical sense with little chance of negative consequences.

Examples:

• There is no difference in basic knowledge of HIV/AIDS between nurses and doctors.
• There is no difference in treatment adherence between healthcare professionals and the general public.

NHST is unnecessary if the only purpose of a hypothesis test is to establish a small difference whose size and direction are of no interest.

Example:

A researcher investigates the effect of two diets on lowering post prandial blood glucose. She notices that there is a small difference between the two diets. To establish this difference, she performs a hypothesis test that shows there is a difference between the two diets- p value is less than 0.05. However, she does not mention the actual difference in mg/dl of serum glucose (effect size), or which diet lowered serum glucose more (direction of difference).

Even small differences can become statistically significant if the sample size is large enough, so merely knowing that there is a difference is not very useful. What is more useful is knowing if the effect size is large enough to warrant a change in care; or which diet would bring about a greater reduction in serum blood sugar. The p value does not shed light on these important questions. Therefore, performing a hypothesis test merely to establish a difference is unwarranted.