Alpha level is the probability that a null hypothesis is rejected when it is actually true. In other words, it is the probability that a wrong decision will be made. Hypothesis testing are often based on a sample that is drawn from a population from interest. A sample is used because of the financial, physical and time resource limitation of conducting a census. Despite all the precautions taken to draw a representative sample, there is still a chance that the sample may not reflect the entire population. Consequently, an erroneous inference about the population is drawn based on the sample that was selected. Alpha level shows the probability of error that be tolerated by the researchers.
The key factor that will determine whether to lower or increase the alpha level is the criticalness of intended application of the findings. Experiments that are intended to prolong human life or threaten human life require a lower alpha level. This is because errors can be vary fatal. Therefore, they should be minimized as much as possible.
For example, experiments testing a new drug that promises to cure cancer requires a very low alpha. This is because cancer is life-threatening disease that needs to be treated critically. Another example would be an experiment testing the survival rate of a given surgery procedure. Complex surgeries are risky and may result in death. Therefore, the alpha should be as low as possible.
Experiments that test applications that are not life threatening can have a higher alpha. They may include experiments conducted out of curiosity with little applications in the real world. An example is a study of the impact of gender on career choices or comparing IQ of various races.
References
Bauer, J. (2009). Statistical Analysis for Decision Makers in Healthcare. New York: CRC Press.
