How we change what others think, feel, believe and do

Two error types

Explanations > Social Research > Drawing conclusions > Two error types

Type 1 error | Type 2 error | See also

In an experiment, we seek to demonstrate that a primary hypothesis is true or false. This leads to two classic types of error. Careful design and can significantly reduce the chance of these errors occurring.

Type 1 error

The Type 1 error (often written 'Type I error') occurs when it is concluded that something is true when it is actually false. In other words the experiment falsely appears to be 'successful'.

This generally means the primary hypothesis, H₁ is believed true, while it is actually false. This is usually proven by finding the null hypothesis, H₀ is probably false, within an acceptable tolerance. 

Type 1 errors often occur due to carelessness or bias on the behalf of the researcher. When their hypothesis is 'proven' they may well be loathe to challenge their findings. As such, type 1 errors can be more common than type 2 errors.

It can be very frustrating when you desperately believe something is true but you are unable to conclusively prove this to be so. It is sad that some researchers feel driven to fake data in order to draw such false conclusions, particularly when professional reputation and research grants may hang in the balance.

The probability of making a Type 1 error is often known as 'alpha' (a), or 'a' or 'p' (when it is difficult to produce a Greek letter). For statistical significance to be claimed, this often has to be less than 5%, or 0.05. For high significance it may be further required to be less than 0.01.

Type 1 errors are also known as 'errors of the first kind'.

Type 2 error

The Type 2 error (often written 'Type II error') occurs when it is concluded that something is false while it is actually true. In other words the experiment falsely appears to be 'unsuccessful'.

This generally means the primary hypothesis, H₁ is believed false, while it is actually true. This is usually proven by finding the null hypothesis, H₀ is probably true, within an acceptable tolerance. 

Type 2 errors can occur when there are mistakes in experimental design, sampling or analysis that cloak actual relationships, for example when the sample is too small or where variation in contextual variables hide the actual relationship.

Being found to have made a type 1 error can lead to accusations of cheating, which can be professionally very damaging. Because of this, type 2 error can be made by researchers who are paranoid about avoiding type 1 errors and are consequently over-cautious in their conclusions. 

The probability of making a Type 2 error is known as 'beta' (b, in contrast to the 'alpha' of Type 1). Cohen (1992) suggests that a maximum acceptable probability of a Type 2 error should be 0.2 (20%).

Type 2 errors are sometimes called 'errors of the second kind'.

Results Matrix

The table below shows four possibilities in the results of experiments.

Type 1 and Type 2 errors are as described above.

When a significant change is correctly found then the effect can be measured to identify how important this is.

When no change is correctly found, the power indicates how likely this is.

See also

Measurement error, Confidence, Experimental power, Experimental effect

Cohen, J. (1992). A power primer.  Psychological Bulletin, 112, 1, 155-159.