CHAPTER 2: The comparability of attitude measurements

Measurement equivalence

In this module, we study recent changes in anti-immigration attitudes in 17 European countries. Anti-immigration attitudes are operationalised by means of a scale consisting of three items. Thus, the purpose of this module is to compare an attitude scale over different time points and across countries. However, comparing abstract concepts - such as attitude scales - involves additional methodological issues. Before meaningful comparisons can be made over time and across countries, it is necessary to guarantee that the variable of interest is measured in a sufficiently comparable way. After all, it could be the case that respondents in different countries interpret the items in very divergent ways, due, for example, to culture-specific terms or translation errors. It is also possible that the interpretation of items changes over time. Obviously, if the meaning of items is not constant over time and across countries, then we end up comparing apples and oranges.

The notion of measurement equivalence refers to precisely this important question of the comparability of measurements. An instrument is said to be equivalent when it measures exactly the same attribute under different conditions (e.g. at various time points or in different countries) [Hor92]. When measurement equivalence is absent, our conclusions risk being flawed. Observed differences between countries or time points could reflect method bias rather than substantial differences in the concepts we intend to measure. Similarly, finding no differences would not necessarily guarantee that ‘real’ differences are absent. For these reasons, measurement equivalence should not be taken for granted, but, instead, should be regarded as a hypothesis that needs to be tested.

This second chapter explains how the comparability of measurement scales can be tested in practice, using AMOS. No previous knowledge of AMOS is required to understand this chapter. The chapter is nevertheless both conceptually and statistically more demanding than the other chapters in this module. The other side of the coin is that, once you have completed this chapter, you will have gained knowledge about a very powerful methodological tool that is rapidly gaining in importance in the field of comparative research. Do not let the statistical notation in the first paragraphs of this chapter discourage you. Once you have worked through the theory, the chapter offers a very concrete, step-by-step guide to performing equivalence tests. Those who are not interested in the topic or who consider it too complex, can skip this chapter and go directly to Chapter 3.