Maximize your response rate
Did you know that the American Medical Association has a specific recommendation for its authors about questionnaire response rate? Here it is, from the JAMA Instructions for Authors:
Manuscripts reporting survey data, such as studies involving patients, clinicians, the public, or others, should report data collected as recently as possible, ideally within the past 2 years. Survey studies should have sufficient response rates (generally at least 60%) and appropriate characterization of nonresponders to ensure that nonresponse bias does not threaten the validity of the findings. For most surveys, such as those conducted by telephone, personal interviews (eg, drawn from a sample of households), mail, e-mail, or via the web, authors are encouraged to report the survey outcome rates using standard definitions and metrics, such as those proposed by the American Association for Public Opinion Research.
Meanwhile, response rates to questionnaires have been declining over the past 20 years, as reported by the Pew Research Center in “The Problem of Declining Response Rates.” Why should we care about the AMA’s recommendation regarding questionnaire response rates? Many of us will send questionnaires to health care professionals who, like physicians, are very busy and might not pay attention to our efforts to learn about them. Even JAMA authors such as Johnson and Wislar have pointed out that “60% is only a “rule of thumb” that masks a more complex issue.” (Johnson TP; Wislar JS. “Response Rates and Nonresponse Errors in Surveys.” JAMA, May 2, 2012—Vol 307, No. 17, p.1805) These authors recommend that we evaluate nonresponse bias in order to characterize differences between those who respond and those who don’t. These standard techniques include:
- Conduct a follow-up survey with nonrespondents
- Use data about your sampling frame and study population to compare respondents to nonrespondents
- Compare the sample with other data sources
- Compare early and late respondents
Johnson and Wislar’s article is not open access, unfortunately, but you can find more suggestions about increasing response rates to your questionnaires in two recent AEA365 blog posts that are open access:
- Jessica Foster on Maximizing Survey Response Rates: make the questionnaire matter and personalize your communications to the members of your sample;
- Steve Young on Making Evaluation Surveys More Appealing: make responding more fun by using meaningful graphics and new rating tools such as graphic sliders.
Find more useful advice (e.g., make questionnaires short, personalize your mailings, send full reminder packs to nonrespondents) at this open access article: Sahlqvist S, et al., “Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial.” BMC Medical Research Methodology 2011, 11:62