How we change what others think, feel, believe and do
Things vary, and few more so than people. Variation is the bane of the experimenter who seeks to identify clear correlation.
Random error is that which causes random and uncontrollable effects in measured results across a sample, for example where rainy weather may depress some people.
The effect of random error is to cause additional spread in the measurement distribution, causing an increase in the standard deviation of the measurement. The average should not be affected, which is good news if this is being quoted in results.
The stability of the average is due to the effect of regression to the mean, whereby random effects makes a high score as likely as a low score, so in a random sample they eventually cancel one another out.
The true score is that which is sought. It is not the same as the observed score as this includes the random error, as follows:
Observed score = True score + random error
When the random error is small, then the observed score will be close to the true score and thus be a fair representation. If, however, the random error is large, the observed score will be nothing like the true score and has no value.
The effect of random error is that repeated measurements will give a result across a range of measures, often with the true score in the middle. This is one reason why means are used (to cause regression to the mean).
Another effect is that if a test score is near a boundary it may incorrectly cross the boundary. For example a school exam result is close to the A/B grade level, then the grade given may not be a reflection of the actual ability of the student.
Assuming an observed score is that true score is a dangerous trap, particularly if you have no real idea of how big the random error may be.
In addition to natural error, additional variation from the true score may be introduced when there is some error caused by problems in the measurement system, such as when bad weather affects everyone in the study or when poor questions results in answers which do not reflect true opinions.
There are many ways of allowing or introducing systematic error and elimination of this is a critical part of experimental design, as well as assessment of the context environment at the time of the experiment.
The effect of systematic error is often to shift the mean of the measurement distribution, which can be particularly pernicious if this is to be quoted in results.
Measurement error is the real variation from the true score, and includes both random error and systematic error.
Observed score = True score + random error + systematic error
Measurement error can be reduced by such as:
When measuring variance in analysis of data, for example using the F-ratio, the model variance is the variance that can be explained by the experiment, and this thus 'good' variance. Residual variance is that which cannot be explained by the model being used and is hence undesirable.
A test statistic may thus, for example, be based on the ratio of the model variance to the residual variance. The F-ratio is calculated as MSM/MSR, where MS is the mean square.