Chapter 3: Multigroup Factor Analysis

Example 2 on Multigroup factor analysis: Assessing non-invariance of measurement

Consider the data on the questions D18-D20 which are treated as measures of the factor "obligation to obey the police", for data from Denmark, Norway and Sweden. Fit multigroup models with one factor, and compare models with different specifications of measurement invariance and non-invariance in the items. How well do these different models fit the data, and how do they affect conclusions about cross-national comparisons of the mean of the factor?

Here we consider only three countries and one factor, to keep the command file in Stata relatively short (the commands for all 27 countries and more factors and items would be an obvious extension of these).

Stata commands:

// Example of measurement invariance assessment
// Using three countries, and
// items one factor (obligation to obey) for illustration
* Convert the country variable from string to numeric variables:
* (if not done already earlier)
encode cntry, gen(country)
* Full invariance model:
sem (Obey -> bplcdc@v1 doplcsy dpcstrb) ///
if cntry=="DK" | cntry=="NO" | cntry=="SE", ///
var(7: Obey@1) method(mlmv) ///
group(country) ginvariant(mcons mcoef merrvar)
** Likelihood ratio tests of freeing each of the measurement parameters:
estat ginvariant
estimates store full_invariance
* Scalar invariance: Free error variances for all items
sem (Obey -> bplcdc@v1 doplcsy dpcstrb) ///
if cntry=="DK" | cntry=="NO" | cntry=="SE", ///
var(7:Obey@1) method(mlmv) ///
group(country) ginvariant(mcons mcoef)
estimates store scalar_invariance
* Different levels of non-equivalence for one item (here bplcdc)
** Scalar invariance: Free error variances
sem (Obey -> bplcdc@v1 doplcsy dpcstrb) ///
if cntry=="DK" | cntry=="NO" | cntry=="SE", ///
var(7: Obey@1) method(mlmv) ///
group(country) ginvariant(mcons mcoef merrvar) ///
var(20: e.bplcdc@v11) var(24: e.bplcdc@v12)
estimates store scalar_invariance1
** Metric invariance: Free error variances and intercepts
sem (Obey -> bplcdc@v1 doplcsy dpcstrb) ///
(20: _cons@v31 -> bplcdc) ///
(24: _cons@v32 -> bplcdc) ///
if cntry=="DK" | cntry=="NO" | cntry=="SE", ///
var(7: Obey@1) method(mlmv) ///
group(country) ginvariant(mcons mcoef merrvar) ///
var(20: e.bplcdc@v11) var(24: e.bplcdc@v12)
estimates store metric_invariance1
** Full non-equivalence: Free error variances, intercepts and loadings
sem (Obey -> doplcsy@v1 bplcdc dpcstrb) ///
(20: _cons@v31 -> bplcdc) ///
(20: Obey -> bplcdc@v21) ///
(24: _cons@v32 -> bplcdc) ///
(24: Obey -> bplcdc@v22) ///
if cntry=="DK" | cntry=="NO" | cntry=="SE", ///
var(7: Obey@1) method(mlmv) ///
group(country) ginvariant(mcons mcoef merrvar) ///
var(20: e.bplcdc@v11) var(24: e.bplcdc@v12)
estimates store full_nonequivalence1
* Likelihood ratio tests against full invariance model
lrtest full_invariance scalar_invariance
lrtest full_invariance scalar_invariance1
lrtest full_invariance metric_invariance1
lrtest full_invariance full_nonequivalence1

Stata output and notes

R commands:

# Example of measurement invariance assessment
# Using three countries, and
# items one factor (obligation to obey) for illustration:
library(lavaan)
#
ind <- ESS5Police$cntry=="DK" | ESS5Police$cntry=="NO" |
ESS5Police$cntry=="SE"
# Full invariance model:
ModelSyntax <- '
Obey =~ NA*bplcdc + doplcsy + dpcstrb
Obey ~~ c(1,rep(NA,2))* Obey
'
FittedModel.fullinv <- sem(model = ModelSyntax,
data = ESS5Police[ind,], group="cntry",
meanstructure = TRUE,missing="ml",
group.equal=c("intercepts","loadings","residuals"))
summary(FittedModel.fullinv)
# Scalar invariance: Free error variances for all items
FittedModel.scalar <- sem(model = ModelSyntax,
data = ESS5Police[ind,], group="cntry",
meanstructure = TRUE,missing="ml",
group.equal=c("intercepts","loadings"))
summary(FittedModel.scalar)
# Different levels of non-equivalence for one item (here bplcdc)
## Scalar invariance: Free error variances
FittedModel.scalar1 <- sem(model = ModelSyntax,
data = ESS5Police[ind,], group="cntry",
meanstructure = TRUE,missing="ml",
group.equal=c("intercepts","loadings","residuals"),
group.partial=c("bplcdc~~bplcdc"))
summary(FittedModel.scalar1)
## Metric invariance: Free error variances and intercepts
FittedModel.metric1 <- sem(model = ModelSyntax,
data = ESS5Police[ind,], group="cntry",
meanstructure = TRUE,missing="ml",
group.equal=c("intercepts","loadings","residuals"),
group.partial=c("bplcdc~~bplcdc","bplcdc~1"))
summary(FittedModel.metric1)
## Full non-equivalence: Free error variances, intercepts and loadings
FittedModel.noneq1 <- sem(model = ModelSyntax,
data = ESS5Police[ind,], group="cntry",
meanstructure = TRUE,missing="ml",
group.equal=c("intercepts","loadings","residuals"),
group.partial=c("bplcdc~~bplcdc","bplcdc~1","Obey=~bplcdc"))
summary(FittedModel.noneq1)
# Likelihood ratio tests against full invariance model:
anova(FittedModel.fullinv,FittedModel.scalar)
anova(FittedModel.fullinv,FittedModel.scalar1)
anova(FittedModel.fullinv,FittedModel.metric1)
anova(FittedModel.fullinv,FittedModel.noneq1)

R output and notes

Results for the fitted models are summarized in Table 3.2. Here we consider for illustration seven different specifications for the measurement models across the three countries: The model where full invariance holds, scalar invariance for all of the three items, scalar invariance, metric invariance and complete non-invariance for item D18, and complete non-invariance for items D19 and D20. Note that models with metric invariance in all the items or models with non-invariance in two or more items are not included, because such models would not allow the identification of distinct estimates of factor means for the different countries.

Considering first the goodness of fit of the models, likelihood ratio tests indicate that allowing for non-invariance in any one item would improve the fit compared to the full invariance model. The tests of partial invariance models shown for item D18 suggest that for this item at least the measurement intercepts in particular are significantly different between the countries. The AIC and BIC statistics indicate that the model preferred by each of them includes non-invariance of measurement. So in these data, even with just three items and three culturally and linguistically fairly similar countries, a multigroup analysis suggests significant deviations from exact invariance of measurement.

What matters most for substantive interpretation, however, is whether comparative conclusions about the constructs being measured are affected by different choices for the measurement models. Here they are not. Table 3.2 also shows that all the models considered here yield very similar estimates for the estimated means of the factor. According to all of them, the average level of felt obligation to obey the police is around -0.52 in Norway and around -0.35 in Sweden, on a scale where the mean in Denmark is fixed at 0, and the standard deviation in Denmark fixed at 1. The estimated standard errors of these estimates are around 0.04, so all the differences between the country means are statistically significant. Since all the models give similar results about the factors, here we could without difficulty focus on the simplest model which assumes invariance of measurement.

Table 3.2: Estimated factor means and model assessment statistics for a 1-factor multigroup confirmatory factor analysis model for indicators of Obligation to obey the police, under different specifications of non-invariance of measurement. The models are fitted to data from Denmark, Norway and Sweden.

Estimated mean
(and standard error)
of the factor
[Obligation to obey the police]
Measurement model LR-test againt full invariance model: P-Value AIC BIC Denmark
(Constrained)
Norway Sweden
Full invariance 57417 57501 0 -0.52
(0.04)
-0.36
(0.04)
Scalar invariance
(error variances free)
<0.001 57394 57516 0 -0.51
(0.04)
-0.36
(0.04)
Scalar invariance for item D18
0.10 57417 57513 0 -0.52
(0.04)
-0.36
(0.04)
Metric invariance for item D18
(error variance and intercept free)
<0.001 57384 57493 0 -0.50
(0.04)
-0.37
(0.04)
Complete non-invariance for item D18 <0.001 57381 57503 0 -0.50
(0.04)
-0.37
(0.04)
Complete non-invariance for item D19 <0.001 57397 57519 0 -0.57
(0.05)
-0.34
(0.04)
Complete non-invariance for item D20 <0.001 57396 57518 0 -0.52
(0.04)
-0.37
(0.04)

Go to next chapter >>