How we change what others think, feel, believe and do
Solomon Four-Group design
Use this design when it is suspected that, in taking a test more than once, earlier tests have an effect on later tests, for example by learning or priming effects.
In addition to the basic pre-test/treat/post-test design, do three additional tests, one without the treatment, one without the pre-test and one without both pre-test and treatment.
In design notation this may be written as:
In a test where there is no priming or learning effect, the pre-test and scores without treatment will all be be similar.
Where there is a priming or learning effect, then repeated tests without the treatment will show a significant change, whilst posts-tests without a pre-test will give results dissimilar to the basic pre-test and post-test design.
In a teaching experiment the Solomon design shows that testing before and without treatment have similar results, whilst results after teaching are significantly improved. This indicates that the treatment is effective and not subject to priming or learning effects.
In another experiment, the initial test seems to indicate that teaching has an effect. However, (b) shows that without teaching the score significantly improves, (c) shows that without a pre-test the score is not as impressive as (a). The single post-test (d) gives a score similar to pre-tests, as might be expected.
Pre-test and post-test are common ways of determining change caused by a treatment, but they are subject to improvement effects.
The Solomon design applies different variations of the test, omitting various elements and thus allowing the effects of these omissions to be assessed.
Note that for a reliable result, several sets of four tests should be applied and the means used.