Chapter 3: Multigroup Factor Analysis

Invariance vs. non-invariance models in practice: Sensitivity of main conclusions

Whatever methods we use for model assessment, in many applications it is a common conclusion that there is evidence of at least some non-invariance of measurement. This is certainly the case for large cross-national surveys of general populations, where it appears to be very rare that full invariance is formally judged to hold for any measurement scales with multiple items. This then raises the difficult question of what would be the best way to analyze latent constructs in such situations.

When the main purpose of a multigroup analysis is to obtain cross-group (e.g. cross-national) comparisons of the latent factors, the most relevant criterion for assessing the effect of non-invariance of measurement is how different specifications of the measurement models affect the main conclusions about distributions of the factors. If these conclusions are relatively insensitive in this respect, the choice of the measurement models does not matter much – and, in particular, we can with confidence use the simplest choice, the model with full invariance of measurement. Such sensitivity analysis can be done by fitting models with different levels of non-invariance in turn, and comparing the parameter estimates for the distributions of the factors. This is illustrated in Example 2 of this chapter. An even simpler approach has been proposed by [Obe14]. His EPC-interest statistic (expected parameter change in parameters of interest) requires only that the full invariance model is fitted, and gives a good approximation of how much estimates of parameters of interest such as factor means would change if different measurement parameters were freed to be non-invariant across groups.

Ultimately the outcome of such sensitivity analyses may be that the choice matters, i.e. that conclusions about cross-national comparisons do depend on how much non-invariance of measurement we allow in the model specification. In this situation it might seem natural to use the results obtained from the best-fitting non-invariance model. However, this approach also has its problems, even apart from the fact that there may be no partial non-invariance model which both fits well and is identified. Any non-invariance model presents additional complications of interpretation, for example that (as noted previously) comparative conclusions about the means of latent factors are then really based only on those observed items which are specified as invariant. The opposite alternative approach is to base the conclusions on the full invariance model even when it does not fit well according to formal model selection criteria. This is easy to do and ensures that each item is treated in the same way, for all countries; however, it also ignores the observed evidence of non-invariance and thus in effect defines the latent constructs to be measured on a common scale across the countries. In short, all possible choices on how to treat items which are thought to be cross-nationally non-invariant have their disadvantages as well as different advantages. [Kuh15] present some further discussion of these conceptually difficult questions.

Go to next page >>