Factor Analysis: A Simplified Overview

Factor Analysis is a statistical technique that we use to identify underlying relationships between different variables. Imagine we have a lot of related variables, and we suspect that these could be influenced by a few underlying factors. Factor analysis helps us to unearth these underlying factors.

                                                        Diagram source: Statistics By Jim

Basic Concept:

Think of factors as underlying (and unobservable) variables that somehow influence the observable variables we measure directly. Factor analysis tries to find out how many of these hidden factors might be influencing the patterns of response we see in our data and what variables are related to which underlying factor(s).

Example:

Imagine we are researching why students get the grades they do. We have data on various variables, such as attendance rate, hours spent studying, sleeping hours, part-time job hours, and so forth. These variables can be many and somewhat overwhelming to analyse individually.

Let’s dive a bit into the example:

Identifying Factors: We hypothesize that these observable variables (e.g., study hours, sleep hours) might be influenced by a few unobservable factors like Work Ethic or Time Management.

The Relationship:

Maybe hours spent studying and attendance rate are both influenced by an underlying factor we might label as Diligence.

While sleeping hours and part-time job hours might be influenced by Time Management.

Why bother?

It helps us reduce our workload: Instead of dealing with 5, 10, or 50 variables, we can group them under a few factors and work with those, making our analysis more straightforward and interpretable.

It provides insight into the patterns or structures (the underlying factors) in our data: We can understand what hidden influences might be driving the observable patterns in our data.

Steps in Factor Analysis:

Extraction: Extract the minimum number of factors that can aptly represent the patterns in the relationship among variables.

Rotation: Rotate the factors to ensure that they make sense (both statistically and theoretically). Rotation can help simplify and interpret the data.

Interpretation: Assign labels to the factors (like 'Diligence' or 'Time Management' in our example) and interpret the data accordingly.

Application:

Once the factors are identified and interpreted, we can:

·       Use them to understand how different variables interact.

·       Develop strategies (like study plans or interventions) that target the underlying factor, affecting all associated variables simultaneously.

To sum-up, Factor Analysis simplifies data by finding the unobservable factors influencing the patterns of observed variables, aiding in data interpretation and strategy formulation, especially when dealing with numerous variables.