ELI5: Explain Like I'm 5

Misuse of p-values

P-values are numbers that scientists use to decide whether their findings are really important or just happened by chance. Imagine you are playing a game of chance, like flipping a coin. If you flip a coin four times and it comes up heads each time, you might think something special was going on. However, if you flip the same coin 40 times and it comes up heads 20 times and tails 20 times, you will probably understand that it was just chance.

When scientists do experiments, they compare their findings to random chance to see whether they have found something special or not. They use p-values to do this. If the p-value is small, it means that their findings are unlikely to be just chance and therefore might be meaningful.

However, some scientists misuse p-values by setting a very low threshold for their p-value (say, 0.05) and then not reporting the other p-values they found that were bigger than this threshold. This way, they make their findings seem more important than they really are.

It’s important to use p-values carefully and not misuse them, because wrong conclusions from statistical analyses can lead to wrong decisions being made in fields like medicine, economics, or engineering.