This book is Work in Progress. I appreciate your feedback to make the book better.

8.5 Confirmatory Factors

Specifying and evaluating measurement models

Exploratory factor analysis may suggest a structure. Confirmatory factor analysis turns that suggestion into an explicit measurement claim.

Suppose nine observed tests are intended to measure three abilities: visual processing, textual ability, and speed. A CFA states which indicators measure which factors, which factor correlations are allowed, and which cross-loadings or residual relations are fixed to zero.

CFA therefore asks:

Can a specified measurement model reproduce the observed relationships well enough to support its intended interpretation and use?

The word confirmatory does not mean that the theory will be confirmed automatically. It means that the restrictions are stated before the model is evaluated.

8.5.1 From a Pattern to a Measurement Claim

A one-factor measurement equation is

\[ x_{pi} = \nu_i + \lambda_i\eta_p + \varepsilon_{pi}. \]

The model says that person \(p\)'s response to indicator \(i\) contains:

  • an indicator intercept \(\nu_i\);
  • a factor contribution \(\lambda_i\eta_p\);
  • residual variation \(\varepsilon_{pi}\).

For several indicators and factors, the model-implied covariance matrix is

\[ \boldsymbol{\Sigma}(\boldsymbol{\vartheta}) = \boldsymbol{\Lambda} \boldsymbol{\Phi} \boldsymbol{\Lambda}^{\top} + \boldsymbol{\Theta}. \]

The loading matrix \(\boldsymbol{\Lambda}\) connects indicators to factors. The factor covariance matrix \(\boldsymbol{\Phi}\) describes variation and relationships among factors. The residual covariance matrix \(\boldsymbol{\Theta}\) describes indicator variation not represented by the factors.

Estimation chooses parameters \(\boldsymbol{\vartheta}\) so that the model-implied covariance matrix resembles the observed covariance matrix \(\mathbf{S}\).

8.5.2 Fixed and Free Parameters

A CFA is defined as much by what it excludes as by what it estimates.

A conventional three-factor model may state that:

  • each item loads on one intended factor;
  • omitted cross-loadings are fixed to zero;
  • the three factors may correlate;
  • residual variances are estimated;
  • residual correlations are initially fixed to zero.

In lavaan, the measurement model is compact:

The operator =~ means is measured by. With std.lv = TRUE, each latent factor variance is fixed to one to define its scale.

The syntax is short, but the claim is strong: the model assigns measurement roles, excludes many possible relations, and asks whether those restrictions are compatible with the data.

8.5.3 Identification and Scaling

Before asking whether a model fits, we must ask whether its parameters can be estimated uniquely.

Identification asks whether the observed moments determine one unique parameter solution. With \(q\) observed variables, a covariance matrix contains

\[ \frac{q(q+1)}{2} \]

distinct variances and covariances. If the model estimates \(k\) free parameters from \(m\) observed moments, the degrees of freedom are

\[ df=m-k. \]

  • \(df<0\): the model is underidentified.
  • \(df=0\): the model is just identified and reproduces the observed moments by construction.
  • \(df>0\): the model is overidentified and its restrictions can be contradicted by the data.

Counting moments is necessary but not always sufficient; the equations must also contain independent information.

A latent variable has no natural unit. Its scale is commonly defined by:

  1. fixing one loading to one; or
  2. fixing the factor variance to one.

This scaling choice defines a unit. It does not mean that the fixed value is known substantively.

A one-factor model with three indicators is often just identified when the factor scale is fixed and residuals are independent. It can estimate the parameters, but global fit cannot test the one-factor restrictions. Four or more indicators usually create degrees of freedom, although identification still depends on the complete specification.

A unique estimate is not the same as a good model. Identification makes evaluation possible; it does not establish validity.

8.5.4 Estimation

The estimator should match the indicators and the analysis.

Indicators and data Common approach
approximately continuous and reasonably regular maximum likelihood
continuous but non-normal robust maximum likelihood such as MLR
ordered categorical or binary categorical estimators such as WLSMV, or an appropriate categorical likelihood
incomplete continuous data under a defensible missing-at-random assumption full-information maximum likelihood with an ML-based estimator

Treating a few ordered categories as continuous may or may not be reasonable. The consequences depend on the number of categories, response distributions, sample size, and model. The decision should be reported rather than hidden behind software defaults.

Robust estimation changes standard errors and test statistics to better accommodate violations such as non-normality. It does not repair a substantively misspecified measurement model.

8.5.5 Reading the Parameters

A standardized loading indicates how strongly an indicator is represented by a factor in the specified model. In a simple standardized one-factor model, a loading of \(0.70\) implies

\[ 0.70^2=0.49, \]

so about 49% of the indicator variance is represented by the factor. The interpretation becomes less direct when indicators have several loadings, residual correlations, or other predictors.

Important parameters include:

  • factor loadings: how indicators relate to factors;
  • factor correlations: how latent dimensions relate to one another;
  • residual variances: how much indicator variation remains;
  • intercepts or thresholds: where indicators are located on their response scales;
  • latent means and variances: how factors are distributed when the model identifies them.

Large loadings do not by themselves prove good content coverage. A factor measured by several nearly identical questions can look statistically strong while representing only a narrow part of the intended construct.

8.5.6 Common Confirmatory Factor Models

Several familiar models are different explanations of the same observed covariance structure. They are not interchangeable ways to improve fit.

8.5.6.1 One common factor

All indicators load on one factor:

\[ x_i=\nu_i+\lambda_i\eta+\varepsilon_i. \]

This model is plausible when one broad dimension is expected to account for the shared variation.

8.5.6.2 Correlated factors

Different indicator groups measure distinct but related factors. For example, visual, textual, and speed factors may correlate.

This is often the most transparent starting point when theory distinguishes several domains without proposing a higher-level mechanism.

8.5.6.3 Second-order factor

A second-order model explains correlations among first-order factors through a broader factor:

\[ \boldsymbol{\eta} = \boldsymbol{\Gamma}G + \boldsymbol{\zeta}. \]

Items load on first-order factors, and the first-order factors load on the general factor \(G\). The general factor influences the indicators indirectly through the domain factors.

General factor
      ↓
First-order factors
      ↓
Observed indicators

This model is appropriate only when the correlations among the first-order factors support a meaningful higher-order interpretation.

8.5.6.4 Bifactor model

In a bifactor model, each indicator loads directly on:

  • one general factor;
  • one specific factor.
General factor ─────────────→ all indicators
Specific factor 1 ─────────→ its indicator group
Specific factor 2 ─────────→ its indicator group

The general factor represents variation common across all indicators. Specific factors represent remaining common variation within domains after the general factor is taken into account.

A bifactor model can fit flexibly because it contains many loadings. Better fit does not automatically justify reporting one general score. The general and specific factors must be stable, interpretable, and supported by the indicator design.

Second-order and bifactor models are advanced confirmatory factor models. They are not a middle category between CFA and SEM, and they should not be selected only because a simpler model fits poorly.

8.5.6.5 Cross-loadings

An indicator may genuinely reflect more than one factor:

\[ x_i = \lambda_{i1}\eta_1 + \lambda_{i2}\eta_2 + \varepsilon_i. \]

A cross-loading can be theoretically appropriate when item content spans dimensions. Fixing every non-target loading to zero may be too rigid.

8.5.6.6 Correlated residuals and testlets

Residuals may remain related because indicators share wording, response format, position, stimulus, or context:

\[ \operatorname{Cov}(\varepsilon_i,\varepsilon_j)\neq0. \]

For competence assessments, tasks based on the same reading text or audio passage may share variation beyond the intended competence. For questionnaires, positive and negative wording can create method effects.

Residual correlations should represent a specific source of shared indicator variation. They should not become unexplained patches added until the fit indices turn favorable.

8.5.6.7 Method and response-style factors

A separate factor can represent systematic response behavior rather than the target construct, such as acquiescence or common wording direction. Such factors can be useful, but they require item designs that identify the method effect separately from the substantive factors.

8.5.7 Evaluating a Confirmatory Model

Day 3 of the course begins after the model has been estimated. The output is on screen, and the uncomfortable question is:

Is this model good enough, and what does good enough mean?

A defensible evaluation follows a sequence rather than one cutoff.

8.5.7.1 Step 1: Did the model converge?

Check whether the optimizer reached a solution and whether warnings occurred. Non-convergence can result from poor starting values, an underidentified model, extreme correlations, sparse categories, or an implausible specification.

A non-converged solution should not be interpreted as an ordinary fitted model.

8.5.7.2 Step 2: Are the estimates admissible and plausible?

Inspect:

  • negative residual variances;
  • factor correlations outside the admissible range or extremely close to one;
  • implausibly large standard errors;
  • unstable or weak loadings;
  • thresholds or variances inconsistent with the data;
  • estimates driven by miscoding or reversed items.

A model can display attractive global fit indices and still contain inadmissible or substantively implausible parameters.

8.5.7.3 Step 3: How well does the model reproduce the data globally?

Common global diagnostics include:

Measure Main question General direction
\(\chi^2\) test Is exact fit tenable? smaller relative to degrees of freedom; non-significance is favorable but strongly sample-size dependent
CFI How much does the model improve over an independence baseline? higher
TLI How much improvement remains after a complexity adjustment? higher
RMSEA How much approximation error is estimated per degree of freedom? lower; interpret with its confidence interval
SRMR How large are the average standardized residual discrepancies? lower
AIC and BIC Which candidate balances fit and complexity better? lower, only for comparable candidate models

Historically influential orientation points include CFI and TLI near or above .95, RMSEA near or below .06, and SRMR near or below .08. These values are not laws. Their behavior depends on sample size, model size, factor loadings, estimator, distributions, and the type of misspecification.

The chi-square test asks about exact fit. With large samples, small discrepancies can become statistically detectable. Approximate fit indices answer different questions, but none identifies the substantive source of misfit.

The RMSEA confidence interval is often more informative than the point estimate alone. A wide interval indicates uncertainty; an interval concentrated at high values provides stronger evidence of approximation error.

A fit index is evidence, not a verdict. CFI = .93 does not mean that 93% of the model is correct, and RMSEA = .05 does not prove construct validity.

8.5.7.4 Step 4: Where does the model misfit locally?

Global indices summarize discrepancies. Local diagnostics indicate where they occur:

  • residual covariance or correlation matrices;
  • standardized residuals;
  • modification indices;
  • expected parameter changes;
  • unusual loadings and residual variances;
  • item-level response patterns.

A modification index approximates how much the chi-square statistic might decline if one fixed parameter were freed. It does not establish that the parameter is theoretically correct.

An expected parameter change estimates the likely magnitude of the freed parameter. Ranking modification indices without examining magnitude and item content encourages accidental overfitting.

8.5.7.5 Step 5: Revise, compare, or reject?

Poor fit can point to different problems:

Possible source Possible response
one item spans two constructs consider a justified cross-loading or revise the item
two items share wording or a stimulus model the method relation or improve the instrument
proposed factors are not distinguishable reconsider the number or meaning of factors
an important dimension is missing specify and test an additional factor
reversed items behave differently inspect coding, wording, and response styles
repeated modifications only work in one sample validate in new data
a simple observed score serves the purpose reconsider whether a latent model is necessary

Researchers should distinguish revising the statistical model, revising the measurement instrument, and revising the substantive theory. These are not the same response.

8.5.7.6 Step 6: Validate beyond the original fit

A model repeatedly modified in one sample may describe that sample unusually well. Stronger evidence comes from:

  • testing the structure in a new sample;
  • separating exploration and validation;
  • examining stability across groups or time;
  • testing measurement invariance;
  • relating the construct to external variables;
  • checking robustness to reasonable coding and estimator choices.

Global fit is only one part of validity. A factor can fit statistically while omitting important content, and a useful measure can fail an unnecessarily rigid model.

8.5.8 Beyond Universal Cutoffs

The course introduced two newer approaches. They are useful because they make the limitations of universal thresholds visible, but they should be treated as advanced tools rather than automatic replacements for the full evaluation sequence.

8.5.8.1 Dynamic fit-index cutoffs

Dynamic approaches derive reference values for the particular model under study.104

The general idea is:

  1. estimate the proposed model;
  2. simulate data under correctly specified and deliberately misspecified conditions;
  3. fit the proposed model to the simulated datasets;
  4. examine how CFI, RMSEA, and SRMR behave;
  5. derive model-specific decision regions.

The benefit is calibration to features such as model size, loading strength, and sample size. The cost is a new layer of assumptions: the answer depends on which misspecifications are simulated, how large they are, and which error rates matter.

Dynamic cutoffs are therefore best viewed as a sensitivity analysis. They do not replace parameter inspection, local diagnostics, or substantive reasoning.

8.5.8.2 Machine-learning-based diagnosis

A different innovation uses models trained on many simulated factor models. Instead of comparing one fit index with one threshold, the classifier combines features of the estimated model and predicts whether particular types of misspecification may be present.

This approach can potentially distinguish patterns that a single CFI or RMSEA value cannot. It is closer to a diagnostic aid than to a new universal cutoff.

Its validity is bounded by the training simulations. More computing power cannot compensate for missing or unrealistic training conditions. A classifier should therefore generate hypotheses for inspection, not replace theory or item-level diagnosis.

Course lesson: two models can display nearly the same global fit index and still require different conclusions. The numerical value summarizes discrepancy; it does not diagnose its source.

8.5.9 A Compact CFA Workflow in R

The code is intentionally portable and uses data included with lavaan. A complete analysis also requires careful item coding, missing-data decisions, distribution checks, estimator justification, and validation.

8.5.10 What Should Be Reported?

A transparent report should describe:

  1. the construct and indicators;
  2. the theoretical reason for the factor structure;
  3. item coding, including reversed items;
  4. sample size and missing-data treatment;
  5. estimator and software;
  6. identification and scaling;
  7. loadings, factor correlations, and residual variances;
  8. chi-square, degrees of freedom, CFI, TLI, RMSEA with interval, and SRMR;
  9. local diagnostics and every model modification;
  10. alternative models and validation analyses where relevant.

The conclusion should combine statistical and substantive evidence. The goal is not to make every diagnostic green. It is to construct a measurement argument whose assumptions, evidence, limitations, and intended use are clear.

8.5.11 From Measurement to Structure

CFA specifies how latent constructs are measured. The next step asks how those constructs relate to one another and to observed variables.

In a broad technical sense, CFA belongs to the structural-equation-model family. In the narrower teaching sequence used here:

\[ \text{CFA} = \text{measurement model}, \]

while

\[ \text{SEM} = \text{measurement model} + \text{structural regressions and paths}. \]

This transition is more than a change of software syntax. It moves from evaluating a measurement claim to studying a system of relationships while retaining the measurement model.

  1. McNeish, D., and Wolf, M. G., Dynamic Fit Index Cutoffs for Confirmatory Factor Analysis Models, Psychological Methods, 28(1), 61--88, https://doi.org/10.1037/met0000425.↩︎