PRINCIPAL-AXIS FACTOR ANALYSIS

Introduction and Fundamental Definition

Principal-Axis Factor Analysis (PFA), often referred to interchangeably as Common Factor Analysis, stands as a fundamental multivariate statistical technique within the domain of psychometrics and data reduction. The primary objective of PFA is highly specific: to identify the smallest possible set of underlying, unobservable constructs, termed factors, that are responsible for explaining the shared or common variance observed among a larger battery of measured variables. Unlike related methods that focus on summarizing total variance, PFA deliberately targets only the variance that is common across variables, systematically excluding variance unique to each measurement. This methodology embodies the core philosophical premise that correlations among observed variables are generated by latent variables, and the statistical task is to reverse-engineer these underlying influences, attempting to find the least number of factors accounting for the common variance of a set of variables.

The core principle guiding PFA is simplification and parsimony. When researchers collect data involving numerous highly correlated measures—such as items on a personality questionnaire or various indicators of a single economic construct—it is inefficient and often misleading to treat each variable as statistically independent. PFA provides a rigorous mathematical solution by decomposing the variance structure. It seeks to model the interrelationships by postulating latent factors that drive the observed correlations. If the procedure is successful, the resulting factors capture the essence of the original data set with significantly fewer dimensions, allowing for a clearer, more interpretable understanding of the underlying psychological or sociological phenomena being studied. This focus on commonality ensures that the resulting factors are robust theoretical constructs rather than mere data compression artifacts, simplifying common variance amongst a set of different variables.

The historical development of PFA is deeply rooted in the origins of factor analysis pioneered by Charles Spearman, who first proposed a two-factor theory distinguishing between a general intelligence factor (g) and specific factors. PFA extends this logic by allowing for multiple common factors rather than just one. Mathematically, PFA initiates the analysis by estimating the portion of variance in each variable that is shared with others—a concept known as communality—and uses these estimates on the diagonal of the correlation matrix instead of the perfect variance (1.0) used in Principal Component Analysis. This critical initial step defines PFA as a true common factor model, setting it apart by explicitly acknowledging that measurement error and specific variance cannot be explained by the common factors and must therefore be excluded from the analysis aimed at latent structure discovery.

Theoretical Framework: The Common Factor Model

The theoretical foundation of Principal-Axis Factor Analysis rests squarely upon the Common Factor Model, a linear model asserting that each observed variable is a linear function of a set of common factors plus a unique factor specific to that variable. Formally, for any observed variable $X_i$, its variance is partitioned into two orthogonal components: variance explained by the underlying common factors shared by all variables in the set, and variance attributable to the unique factor, which encompasses both the specific reliable variance of $X_i$ not shared with other variables and random measurement error. This strict separation is crucial; the entire objective of PFA is to model and explain the common part, leaving the unique part unmodeled by the latent constructs, which are the primary focus of the analysis.

In mathematical terms, the model postulates that the correlation between any two observed variables, $X_i$ and $X_j$, is entirely explained by their shared loadings on the common factors. If the common factors are extracted and accounted for, the residual correlation between $X_i$ and $X_j$ should ideally approach zero. This property, known as the condition of local independence, signifies that the common factors successfully mediate all relationships among the observed variables. The factor loadings derived through PFA represent the correlation coefficients between the observed variables and the derived latent factors, indicating the degree to which each variable contributes to the definition of the factor. Higher loadings suggest a stronger relationship and greater relevance of that variable to the underlying construct represented by the factor, thus providing the statistical linkage between the observed data and the hypothesized theoretical structure.

Understanding the structure of variance partitioning is paramount to appreciating PFA. The total variance of any observed variable ($X_i$) equals its communality ($h_i^2$) plus its uniqueness ($u_i^2$). The communality, $h_i^2$, is the sum of the squared factor loadings for variable $i$ across all extracted common factors, representing the variance reliably explained by the common structure. Conversely, the uniqueness, $u_i^2$, is the variance left unexplained by the common factors. A key implication of this framework is that if a variable has a very low communality, it shares little variance with the other variables in the analysis and is therefore poorly represented by the resulting factor structure, suggesting it might be better removed or treated separately. PFA’s adherence to this model ensures that the resulting factors are true reflections of shared psychological constructs rather than statistical aggregates of unique variations or measurement artifacts.

The Critical Role of Communality Estimation

The most defining procedural characteristic distinguishing Principal-Axis Factor Analysis from Principal Component Analysis (PCA) lies in the estimation of communalities. Communality, defined as the proportion of a variable’s variance that is shared with all other variables in the set, must be estimated before the factor extraction process begins. Since the true communalities are unknown prior to factor extraction, PFA must employ iterative estimation techniques. The process begins with initial estimates inserted into the diagonal of the correlation matrix, replacing the perfect unity values (1.0) that represent total variance. This crucial substitution ensures that only the common variance matrix is analyzed, effectively factoring out the unique variances and measurement error from the outset, aligning the analysis directly with the Common Factor Model.

Various methods exist for providing initial communality estimates. Historically, the most common initial estimate is the squared multiple correlation (SMC) of each variable with all other variables in the set. The SMC represents a lower bound for the true communality, as it indicates the maximum variance that one variable can share linearly with the remaining variables. While other methods, such as utilizing the maximum absolute correlation in the column, are sometimes employed, the SMC approach is statistically preferred because it provides a more conservative and robust starting point for the subsequent iterative procedures. The choice of initial estimate, while important for computational stability, often becomes less critical as the iterative process proceeds and the estimates converge towards the maximum likelihood solution.

The core strength of PFA derives from its iterative refinement process concerning communality estimation. After the initial factors are extracted using the estimated communalities, the loadings are used to calculate new, improved estimates of the communalities. These new estimates are then re-inserted into the diagonal of the correlation matrix, and the factors are extracted again. This cycle of extraction and re-estimation continues until the communality estimates stabilize, meaning the difference between the communality estimates from one iteration to the next falls below a specified convergence criterion (e.g., 0.001). This iterative optimization ensures that the final factor solution maximally accounts for the common variance observed in the data, thereby fulfilling PFA’s goal of finding the most parsimonious model explaining shared relationships while accurately reflecting the latent structure.

Extraction Methodology and Iterative Refinement

The extraction methodology in Principal-Axis Factor Analysis involves a sequential process designed to isolate orthogonal factors that capture maximum variance from the modified correlation matrix (the matrix containing communality estimates on the diagonal). The mathematical procedure is closely related to eigenanalysis, similar to PCA, but applied strictly to the common variance structure defined by the estimated communalities. The first factor extracted is the linear combination of the observed variables that accounts for the largest proportion of the common variance remaining in the matrix. Subsequent factors are extracted to be orthogonal (uncorrelated) to all previously extracted factors, and each accounts for the maximal amount of remaining unexplained common variance, proceeding until the factor extraction criteria are met or convergence is achieved.

The iterative nature of PFA is what defines its methodological rigor and distinguishes it from non-iterative techniques. The cycle ensures that the factor solution is optimized for the common variance structure, achieving maximal explanatory power with minimal dimensions:

1. Initial Estimation: Insert initial communality estimates (e.g., SMCs) into the diagonal of the correlation matrix, preparing the data for common variance analysis.
2. Factor Extraction: Calculate eigenvalues and eigenvectors of the modified correlation matrix to extract the initial set of factors and their loadings, prioritizing the components that explain the most shared variance.
3. Communality Re-estimation: Use the squared factor loadings from the extracted factors to calculate new, improved communality estimates ($h^2$). This step ensures that the estimates reflect the variance explained by the current factor solution.
4. Convergence Check: Compare the new communality estimates with the estimates from the previous iteration. If the differences are significant (e.g., exceeding a tolerance level), the process has not converged, and the analysis returns to Step 1 with the new estimates. If the differences are negligible (convergence achieved), the factor extraction process is finalized.

This systematic refinement guarantees that the final set of factors is the most accurate representation possible of the underlying latent structure given the constraints of the common factor model.

Following the primary extraction, the raw factor loadings are typically mathematically determined to maximize variance but often result in a structure where most variables load moderately on multiple factors, making psychological interpretation challenging. To achieve psychological meaningfulness and statistical simplicity, the factor structure must undergo rotation. Rotation methods aim to maximize high loadings on some factors while minimizing loadings on others, achieving a state known as simple structure. Common rotation techniques fall into two categories: orthogonal (e.g., Varimax), which maintains the independence of factors, and oblique (e.g., Oblimin, Promax), which allows the factors themselves to be correlated. The choice of rotation depends entirely on the theoretical expectation of the constructs; if the underlying constructs are believed to be related (which is often the case in social science), oblique rotation is preferred, yielding a clearer and more realistic interpretation of the factor structure and providing a matrix of inter-factor correlations.

Criteria for Factor Retention and Interpretation

A crucial decision in Principal-Axis Factor Analysis is determining how many common factors to retain for interpretation. Retaining too many factors leads to overfitting, loss of parsimony, and the modeling of random error, while retaining too few results in an inadequate explanation of the common variance structure. Several widely accepted criteria guide this decision, and expert statistical practice dictates that multiple criteria should be considered synergistically, using statistical rules as guides and theoretical considerations as the ultimate determinant of the final model.

Key criteria for factor retention include:

• Kaiser Criterion (Eigenvalue Greater Than One): This is the most common default rule, suggesting that only factors associated with an eigenvalue greater than 1.0 should be retained. An eigenvalue represents the variance accounted for by a factor. If a factor’s eigenvalue is less than 1.0, it accounts for less variance than a single variable, making it generally unprofitable to retain. However, this rule is often criticized for sometimes overestimating the number of factors, particularly in datasets with many variables, and should not be used in isolation.
• Scree Plot Test: Developed by Cattell, the Scree Plot graphs the eigenvalues against the factor number. Researchers look for the point where the slope of the plot levels off (the “elbow”). Factors before the elbow are retained, as they represent substantial variance explanation, while factors after the elbow (the “scree” or rubble) are discarded as they account for only marginal, often specific, variance. This test is subjective but often provides a visually intuitive and strong guide, especially when combined with other methods.
• Parallel Analysis (PA): Considered the most accurate objective method, PA compares the observed eigenvalues against eigenvalues generated from random data of the same size and structure. Only factors whose observed eigenvalues exceed the corresponding random data eigenvalues are retained, ensuring that the retained factors are statistically meaningful and not due to chance correlations inherent in the data matrix.
• Theoretical Justification: Ultimately, the interpretability and theoretical relevance of the factor structure must supersede purely statistical rules. If a factor is statistically significant but cannot be meaningfully interpreted or aligned with existing theory, it may be dropped. Conversely, if retaining an additional factor significantly improves the explanatory model’s fit to a pre-existing theoretical framework, it may be retained even if it marginally fails a statistical rule.

Once the factors are retained and rotated, interpretation involves examining the factor loading matrix. Variables with high absolute loadings (typically greater than 0.3 or 0.4) on a specific factor are used to define the nature of that factor. The researcher assigns a conceptual name to the factor that best summarizes the content or meaning of the variables loading highly onto it. For instance, if a factor has high loadings from variables like ‘sociability,’ ‘assertiveness,’ and ‘talkativeness,’ the factor might be labeled ‘Extraversion.’ The success of PFA is judged not just by statistical fit metrics, but by the clarity, robustness, and theoretical relevance of these derived, interpretable latent constructs, ensuring the model is useful for scientific inquiry.

Distinction from Principal Component Analysis (PCA)

A frequent source of confusion in multivariate statistics lies in the distinction between Principal-Axis Factor Analysis (PFA) and Principal Component Analysis (PCA). While both are data reduction techniques that utilize eigenanalysis, their fundamental goals, underlying models, and mathematical approaches are distinct, leading to potentially different results and interpretations. Understanding this difference is critical for methodological rigor and selecting the appropriate technique for a given research question.

The core difference lies in the treatment of variance. PFA adheres strictly to the Common Factor Model, focusing exclusively on modeling the shared variance (communality) among variables. It views the observed variables as indicators of underlying latent constructs, and its primary goal is to uncover these causal, latent variables. Conversely, PCA is primarily a data summarization technique. It operates under the Component Model, aiming to account for the total variance in the data. PCA computes components as linear combinations of the observed variables, treating all variance—common, specific, and error—as relevant. In mathematical terms, PCA analyzes the correlation matrix with 1.0s (total variance) on the diagonal, whereas PFA analyzes the correlation matrix with estimated communalities ($h^2$) on the diagonal, explicitly removing the unique variance component.

The implications of this difference are profound for interpretation and subsequent theory building. Since PCA utilizes total variance, the resulting components are statistical composites that maximize variance extraction but do not explicitly distinguish between reliable shared information and measurement noise. They are descriptive summaries of the data, meaning a component is mathematically defined by the observed variables. PFA, by stripping away uniqueness and error variance through its iterative communality estimation, produces factors that are hypothesized to represent the true underlying theoretical constructs (latent variables) responsible for the observed correlations. Therefore, PFA is generally preferred in research contexts where the explicit aim is to test a measurement model, validate a construct, or identify the causes of intercorrelations, making it the technique of choice in psychometrics and latent structure discovery. While PCA often yields results numerically similar to PFA when communalities are high, researchers must select PFA if they are committed to the theoretical premise of the common factor model and the identification of unobserved causes.

Assumptions and Methodological Considerations

Like all statistical methods, Principal-Axis Factor Analysis relies on certain statistical assumptions about the data structure for the results to be valid and reliable. Though factor analysis is generally robust to minor violations, severe breaches can compromise the interpretation of the latent structure and lead to unstable or misleading factor solutions. Understanding and checking these assumptions is essential for responsible application of the technique.

Primary assumptions and data requirements include:

• Level of Measurement: Variables should ideally be measured at the interval or ratio level. While ordinal data is sometimes used in practice (e.g., Likert scales), the assumption of continuous underlying distributions is preferred for accurate variance decomposition.
• Linearity: The relationships between the observed variables and the underlying factors are assumed to be linear. If relationships are curvilinear or highly complex, PFA may fail to accurately model the common variance, requiring alternative non-linear modeling approaches.
• Sampling Adequacy: The dataset must contain sufficient correlations among variables to justify factor extraction. This is typically assessed using measures such as the Kaiser-Meyer-Olkin (KMO) Measure of Sampling Adequacy, which should ideally be above 0.6, and Bartlett’s Test of Sphericity, which tests the null hypothesis that the correlation matrix is an identity matrix (i.e., that no correlations exist). Rejection of the null hypothesis in Bartlett’s test supports the use of PFA.
• Absence of Outliers and Multicollinearity: Extreme multivariate outliers can heavily influence the correlation matrix and factor loadings. Similarly, extreme multicollinearity (variables that are nearly perfectly correlated) can cause computational instability and convergence issues during the iterative process, demanding careful data cleaning prior to analysis.

Beyond statistical assumptions, careful methodological considerations surrounding the design and implementation are critical. The quality of the factor solution is fundamentally dependent on the quality and content of the observed variables—the “Garbage In, Garbage Out” principle applies strongly. Researchers must ensure that the items or variables selected are theoretically relevant and adequately sample the domain of interest. If the observed variables do not adequately represent the construct, the resulting factors will be weak or uninterpretable. PFA’s iterative nature and the reliance on accurate communality estimation also mean that convergence issues can occasionally arise, particularly in small samples or when the factor structure is weak. In such cases, careful review of the initial communality estimates and the number of factors being sought is necessary to ensure a stable and interpretable solution, often necessitating adjustments to the model specification or data collection.

Applications and Conclusion

Principal-Axis Factor Analysis is an indispensable tool across a vast array of scientific disciplines, finding its most pronounced application in the behavioral and social sciences, as well as in business and medical research where latent constructs are frequently modeled. It is the preferred method for scale development, where researchers use PFA to confirm that a battery of survey items reliably measures the hypothesized underlying constructs (e.g., measuring dimensions of emotional intelligence, organizational climate, or health literacy). It is also extensively used in large-scale data analysis to reduce the complexity of extensive variable sets, transforming hundreds of correlated measures into a handful of robust, interpretable latent scores suitable for subsequent analysis, such as regression or structural equation modeling.

Specific applications demonstrate the power of PFA in uncovering latent structure that is not directly observable. In educational psychology, PFA might reveal distinct components of learning ability or motivation, separating academic self-efficacy from intrinsic drive. In marketing research, it can identify underlying consumer preference dimensions, reducing hundreds of product rating variables into core factors like ‘Value,’ ‘Luxury,’ and ‘Durability.’ In clinical settings, PFA is used to refine diagnostic instruments by identifying the underlying symptom clusters that define specific psychological disorders. The factors derived through PFA serve as powerful, theory-driven composite variables that possess superior reliability and construct validity compared to individual observed measures, significantly enhancing the researcher’s ability to model complex phenomena accurately.

In conclusion, Principal-Axis Factor Analysis stands as the statistical technique dedicated to the principle of common variance simplification. By iteratively estimating and analyzing only the variance shared among variables, PFA successfully achieves its aim: finding the least number of factors accounting for the common variance of a set of variables. This methodology allows researchers to transition from a complicated matrix of observed correlations to a parsimonious, interpretable model of underlying latent causes, thereby providing fundamental insight into the structure of human behavior, attitudes, and complex systems. Its rigorous adherence to the Common Factor Model ensures that the results obtained are maximally relevant to theoretical construct validation and the advancement of knowledge in fields reliant on the measurement of unobservable traits.