ILR School

The effect of question and questionnaire characteristics on interviewer and respondent behaviors in CATI surveys

Other Titles


In this paper, we evaluate the joint effects of question, respondent and interviewer characteristics on two proxy indicators of data quality - response time and question misreading - in a telephone survey. We include question features traditionally examined, such as the length of the question and format of response options, and features that are related to the layout and format of interviewer-administered questions. First, we examine how these question features affect the time to ask and answer survey questions and how different interviewers vary in their administration of these questions. Second, we investigate how choices in visual design features in particular, that is design features that require interviewer decisions, contribute to interviewer question misreading. These two measures of question time and question misreading are both proxies for the risk of measurement error in responses to survey questions.

To examine these questions, we use paradata and behavior codes from the Work and Leisure Today (n=450, AAPOR RR3=6.3%) survey and use cross-classified random effects models. Overall, more of the variation in both response time and question misreading is due to question characteristics compared to respondent or interviewer attributes. Additionally, we find that question characteristics related to necessary survey design features and respondent confusion are the primary predictors of response time, with little effect of visual design features of the question. Our results for question misreading show a different pattern. Characteristics related to task complexity and visual design significantly affect question misreading, with little contribution of necessary survey design features. We conclude with implications for survey practice.

Journal / Series

Volume & Issue


Thanks to Beth Cochran, Amanda Ganshert, Jinyoung Lee, and Jerry Timbrook for research assistance!


This work was supported by the National Science Foundation Grant No. SES-1132015. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Date Issued





Effective Date

Expiration Date




Union Local


Number of Workers

Committee Chair

Committee Co-Chair

Committee Member

Degree Discipline

Degree Name

Degree Level

Related Version

Related DOI

Related To

Related Part

Based on Related Item

Has Other Format(s)

Part of Related Item

Related To

Related Publication(s)

Link(s) to Related Publication(s)


Link(s) to Reference(s)

Previously Published As

Government Document




Other Identifiers


Rights URI



Accessibility Feature

Accessibility Hazard

Accessibility Summary

Link(s) to Catalog Record