Skip all navigation and go to page content
NN/LM Home About OERC | Contact Us | Feedback |OERC Sitemap | Help | Bookmark and Share

Archive for March, 2014

Maximize your response rate

Did you know that the American Medical Association has a specific recommendation for its authors about questionnaire response rate? Here it is, from the JAMA Instructions for Authors:

Survey Research
Manuscripts reporting survey data, such as studies involving patients, clinicians, the public, or others, should report data collected as recently as possible, ideally within the past 2 years. Survey studies should have sufficient response rates (generally at least 60%) and appropriate characterization of nonresponders to ensure that nonresponse bias does not threaten the validity of the findings. For most surveys, such as those conducted by telephone, personal interviews (eg, drawn from a sample of households), mail, e-mail, or via the web, authors are encouraged to report the survey outcome rates using standard definitions and metrics, such as those proposed by the American Association for Public Opinion Research.

Meanwhile, response rates to questionnaires have been declining over the past 20 years, as reported by the Pew Research Center in “The Problem of Declining Response Rates.” Why should we care about the AMA’s recommendation regarding questionnaire response rates? Many of us will send questionnaires to health care professionals who, like physicians, are very busy and might not pay attention to our efforts to learn about them. Even JAMA authors such as Johnson and Wislar have pointed out that “60% is only a “rule of thumb” that masks a more complex issue.” (Johnson TP; Wislar JS. “Response Rates and Nonresponse Errors in Surveys.” JAMA, May 2, 2012—Vol 307, No. 17, p.1805) These authors recommend that we evaluate nonresponse bias in order to characterize differences between those who respond and those who don’t. These standard techniques include:

  • Conduct a follow-up survey with nonrespondents
  • Use data about your sampling frame and study population to compare respondents to nonrespondents
  • Compare the sample with other data sources
  • Compare early and late respondents

Johnson and Wislar’s article is not open access, unfortunately, but you can find more suggestions about increasing response rates to your questionnaires in two recent AEA365 blog posts that are open access:

Find more useful advice (e.g., make questionnaires short, personalize your mailings, send full reminder packs to nonrespondents) at this open access article: Sahlqvist S, et al., “Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial.” BMC Medical Research Methodology 2011, 11:62

Free Images for Your Evaluation Reports

The current trend in evaluation reporting is toward fewer words and more images. There are a number of companies that offer high-quality, royalty free photographs at minimal cost. (Stockfresh, for example, charges as little as $1 per image.) However, no-cost is even better than low-cost. Freelancers Union, a nonprofit organization dedicated to assisting freelance workers, recently published a list of the best websites for no-cost images.  If you are looking for free images for your presentations or reports, check out their article:

https://www.freelancersunion.org/blog/2014/02/07/best-free-image-resources-online/

(The article also describes the difference between public domain, royalty-free and Creative Commons-licensed images.)

Evaluation Tips: Recipe of Evaluation Techniques

Last week I attended a webinar presentation from Stanley Capela entitled Recipe of Evaluation Techniques for the Real World. This is one of the American Evaluation Association’s (AEA) ongoing 20 minute Coffee Break webinars . The webinars, offered Thursdays at 2pm Eastern time, often present similar tools and tips that are also covered in the Tip a Day blog but allow for audience questions &  answers and networking with the presenters.

Capela’s recipe focused primarily on internal evaluation in a non-profit or government settings where people are seeking realistic answers in response to your assessment efforts. His tips include:

  • Value People’s Timeall time is valuable, regardless of who you are working with, and clear communication on the intent of the evaluation helps to make the best use of everyone’s time.
  • Ethical Conduct – working within the parameters of organization and/or professional association codes of conducts in addition to established support of upper level administration will help to minimize the potential for ethical dilemmas.
  • Know Your Enemies – be aware of those who are resistant to program evaluation and may try to undermine these efforts, and also know that you as an evaluator may be perceived as an enemy by others. Again, clear communication helps!
  • Culture of Accountability – take the time to know the story of those you are working with – where are they coming from? What is their history with previous assessments? Were their needs met, or were there issues that had negative effects on relationships and outcomes?
  • Do Something – avoid cycles of conducting reviews, identifying deficiencies, and outcomes that only include developing correction plans. Also important to note is that program evaluation does not solve management problems.
  • A Picture is Worth 1,000 Words – find ways to integrate charts that direct the reader to the most important information clearly and concisely.
  • Let Go of Your Ego – working from a mindset that accepts the people conducting the program itself will most likely ‘get the credit’, and that your measure of success is doing your job to the best of your ability knowing you made a difference.
  • Give Back – develop of network of trusted colleagues, such as through personal and organization connections on LinkedIn and other platforms, share ideas, and asking questions since others have probably encountered a similar situation or can connect you with those who have.

Hopefully you have found the information we at the Outreach Evaluation Resource Center (OERC) have freely available for you in our updated Evaluation Guides helpful as an additional source of  ideas, strategies, and worksheets to include in your evaluation recipe collection!

Webinars and Workshops about Evaluating Outreach

The National Network of Libraries of Medicine Outreach Evaluation Resource Center (OERC) offers a range of webinars and workshops upon request by network members and coordinators from the various regions. Take a look at the list and see if one of the options appeals to you. To request a workshop or webinar, contact Susan Barnes.

The workshops were designed as face-to-face learning opportunities but we can tailor them to meet distance learning needs by distilling them to briefer webinars or offering them in series of 1-hour webinars.

Don’t see what you’re looking for on this list? Then please contact Susan and let her know!

We’re looking forward to hearing from you.