Differences in frequency distributions

Identical characteristics of the three requests cannot generate differences, except for random errors, but those aspects that do differ in the formulation of a question can generate differences in responses. Many studies have looked at the differences in response distributions [Sch81]. Table 1.2 presents a summary of the responses of British individuals to the questions presented above. We have made the responses as comparable as possible by collapsing categories that can be clustered1.

Table 1.2: The response distribution for the 9 requests specified in Table 1.1 obtained in the ESS Round 1 pilot study for Great Britain

Table 1.2 shows that quite different results are obtained depending on the method used for the formulation of the answer categories. If we did not know that these answers came from the same 485 people and address the same concepts, we might think that these responses come from different populations or that the questions measure different opinions.

It is not surprising that people sometimes say that they are very satisfied and another time say that they are just satisfied. That can happen by chance, but one can also detect very systematic differences between the methods.

One obvious effect is the effect of the neutral category in the second method, which changes the overall distribution of the answers. Another phenomenon that seems to be systematic is that the third method generates much more ‘satisfied’ responses than do the other two. This has to do with the unipolar character of the scale and the extreme label for the negative category, which is ‘not at all satisfied’. This label seems to move some respondents to the category ‘satisfied’ where they are otherwise ‘neutral’ or ‘dissatisfied’. It also appears that the number of people stating that they are ‘very satisfied’ decreases if more response categories are available to express satisfaction: Method 1 has only 2, Method 2 has 5 and Method 3 has 3 possibilities.

Finally, a very clear effect can be observed for the number of missing values. However, this might have to do with the positioning of the questions and other characteristics of the questionnaire.

Go to next page >>

Footnotes

References