The effect of fashion in questions

The CAHPS survey is considered (Consumer Assessment of Healthcare Providers and Systems), which provides for the distribution of the questionnaire to the former patients of a representative sample of 45 US hospitals.

The questions are formulated on an ordinal scale and reveal the perceived quality of the care, the judgments on the hospitals, on the nursing medical staff, the probability of the recommendation of the structure to the acquaintances, etc.

The methods for distributing the questionnaire and the response rates for each are:

  • sending the questionnaire by paper mail (38%)
  • telephone administration (27%)
  • IVR (Interactive voice response): an operator invites the interviewee by telephone to answer; if the proposal is accepted, an interactive guide voice collects the answers. (21%)
  • mixed mode: questionnaire via paper mail + follow-up telephone (42%)

Each hospital chooses one of the available 75 modes for the 4% of its patients; the rest are assigned a randomly chosen method. 7555 interviews have been collected.

IRT is used (Item Response Theory) Bayesian to compare the probabilities of response to ordinal modalities as the type of administration varies.

The "Never" and "Sometimes" modes, listed in the first two positions, are less used in the paper questionnaire than the other methods, a result consistent with the effect. primacy and recency. In fact, even in other surveys, it has been found that often those who answer the phone are more likely to choose the first and above all the last answer mode read by the interviewer (recency).

It should also be noted that the intermediate mode ("Usually") appears to be chosen above all among those interviewed with a paper questionnaire and the least choice among those interviewed by telephone.

Finally it is noted that the answers for the mixed-mode they differ from the answers resulting from a weighted combination of interviews collected by telephone with interviews carried out with a paper questionnaire: the "Always" mode (the last category listed) is selected most from the interviews mixed-mode and to a lesser extent between the fixed + paper interviews.

Taken from:
An Item Response Theory Approach to Estimating Survey Mode Effects: Analysis of Data from a Randomized Mode experiment
Nel Journal of Survey Statistics and Methodology, volume 5, number 2, June 2017 Mariano and Elliot analyze the difference in the answers given to a questionnaire administered in different ways.

Leave a comment