How we change what others think, feel, believe and do

Factor Analysis

Explanations > Social Research > Analysis > Factor Analysis

Description | Example | Discussion | See also

Description

When trying to explain something by measuring a range of independent variables, Factor Analysis helps reduce the number of reported variables by determining significant variables and 'combining' these into a a single variable (or 'factor'). It may be used in this way either to discover factors or to test a hypothesis that they exist.

The determination is statistical, but the output is a 'virtual' or unobserved variable which joins the measured variables in a linear formula that combines the observed measurements with 'factor loading' constant numbers.

Factor Analysis also can be used to help demonstrate how a complex measurement instrument is really measuring one or a few bigger things.

Example

Perhaps the most well-known result of factor analysis is 'IQ', which is a 'virtual' variable based on variable measurements of ability in mathematics, language and logic.

The variables in many personality assessments, such as 16PF, have been identified using factor analysis.

Discussion

Factor Analysis originated in psychometrics by Charles Spearman (as in Spearman correlation) and Raymond Cattell (as in 16PF) with the assessment of intelligence and personality, and has since spread to many other fields, from marketing to operations research.

Factor Analysis is different to much research, which focuses on the relationships between  independent and dependent variables. In contrast, Factor Analysis focuses on the relationship between multiple independent variables. 'Factor' basically means 'independent variable', although in this case the 'factors' are the new 'virtual' variables.

The use of Factor Analysis and its results is often an imprecise science and hence subject to debate (for example 'IQ' has been heavily criticized). Nevertheless it provides a simplification process that in practice can be very useful.

Principal component analysis

Principal Component Analysis is a variant of Factor Analysis and is equivalent when model 'errors' have the same variance. The difference lies in how Principal Component Analysis uses the total variance in the data and assume linear variable combinations, whilst Common Factor Analysis uses the common variance in the data and assumes latent variables.

Principal Component Analysis constructs as many components as there are original variables. The first component takes into account the greatest amount of variance between the variables, giving the weighting of these variables to form the single component. The second component is constructed to account for as much variance is left over that is not accounted for by the first component. The third component accounts for variance not accounted for by the first two components, and so on.

Normally, the first component is much larger than the rest, with a rapid drop off through the second and third components.  If there is no useful way of reducing the matrix of variable correlations into a smaller number of factors then all components will be approximately equal.

Eigenvalues

Eigenvalues represent the proportion of variance explained by a given variable. With five variables, the sum of the eigenvalues will be 5. Sorting the factors by eigenvalue thus results in the first factor having the greatest importance (explaining the greatest amount of variance).

Eigenvalues can be used to help identify the factors to select and carry forward for future use. Ways of doing this include:

• Greater than one: Select variables where the eigenvalue is >1.
• Fixed number: Select the top N variables only.
• Proportion explained: Select a proportion of variance that you want to explain, say 95%, then select those variables that fit within this.
• A Scree Plot is a simple sorted graph of Eigenvalues and can be used to select variables by taking only those on the steep part of the curve.

Orthogonal rotation

Factors selected through eigenvalue analysis may be refined further through orthogonal rotation. 'Orthogonal' factors are metaphorically at 'right angles' to one another, which means they do not correlate with one another and are so even more independent, making them useful measures.

For example if you were measuring types of intelligence, it would help if, say, creative intelligence was completely different from mathematical intelligence, so if you were working on one it would not 'leak' over into another area.

See also

Multiple regression