Experimentwise error rate
Imagine you have a big bag of candies and you want to test if they all taste the same or if some flavors are better than others. You can do this by taking a few candies from the bag and tasting them one by one.
But what if you want to be really sure that your conclusion is correct? You might decide to taste more than just a few candies. You might taste all the candies in the bag!
This is similar to what scientists do when they conduct experiments. They want to make sure that their conclusion is correct, so they do not just test one thing, but many different things.
For example, a scientist might want to test if a new drug can cure a disease. They might test the drug on different groups of people, use different dosages, and measure different outcomes. Each of these tests is called a "comparison".
Now, imagine you are tasting candies and you declare that one flavor is better than the others, but then you taste more candies and change your mind. This is like changing your conclusion after doing more tests.
The problem with doing many tests is that the more comparisons you make, the higher the chance that you will make a mistake. It's like flipping a coin many times - the more you flip it, the higher the chance it will land on tails at least once.
In science, this mistake is called a "false positive" - it means that you conclude something is true when it's actually not. This is why scientists use something called an "experimentwise error rate".
This means that they decide ahead of time how many comparisons they will make, and what level of risk they are willing to take of making a false positive. For example, they might say that they will do 20 comparisons and they are willing to accept a 5% chance of a false positive for each one. This means that overall, they are willing to accept a 5% chance of making a false positive somewhere in their 20 tests.
So, just like you might limit the number of candies you taste to avoid getting a stomach ache, scientists limit the number of tests they do to make sure their conclusions are accurate!
But what if you want to be really sure that your conclusion is correct? You might decide to taste more than just a few candies. You might taste all the candies in the bag!
This is similar to what scientists do when they conduct experiments. They want to make sure that their conclusion is correct, so they do not just test one thing, but many different things.
For example, a scientist might want to test if a new drug can cure a disease. They might test the drug on different groups of people, use different dosages, and measure different outcomes. Each of these tests is called a "comparison".
Now, imagine you are tasting candies and you declare that one flavor is better than the others, but then you taste more candies and change your mind. This is like changing your conclusion after doing more tests.
The problem with doing many tests is that the more comparisons you make, the higher the chance that you will make a mistake. It's like flipping a coin many times - the more you flip it, the higher the chance it will land on tails at least once.
In science, this mistake is called a "false positive" - it means that you conclude something is true when it's actually not. This is why scientists use something called an "experimentwise error rate".
This means that they decide ahead of time how many comparisons they will make, and what level of risk they are willing to take of making a false positive. For example, they might say that they will do 20 comparisons and they are willing to accept a 5% chance of a false positive for each one. This means that overall, they are willing to accept a 5% chance of making a false positive somewhere in their 20 tests.
So, just like you might limit the number of candies you taste to avoid getting a stomach ache, scientists limit the number of tests they do to make sure their conclusions are accurate!