Skip all navigation and go to page content
NN/LM Home About NER | Contact NER | Feedback |Site Map | Help | Bookmark and Share

Archive for the ‘OERC’ Category

Data Party Like it’s 2099! How to Throw a Data Party

Monday, November 23rd, 2015

[guest post by Karen Vargas]

What’s a “data party?” We attended a program by evaluator Kylie Hutchinson entitled “It’s a Data Party!” at the AEA 2015 conference last week in Chicago.  A data party is another name for a kind of participatory data analysis, where you gather stakeholders together, show them some of the data that you have gathered and ask them to help analyze it.

Isn’t analyzing the data part of your job?  Here are some reasons you might want to include stakeholders in the data analysis stage:

  • It allows stakeholders to get to know and engage with the data
  • Stakeholders may bring context to the data that will help explain some of the results
  • When stakeholders participate in analyzing the data, they are more likely to understand it and use it
  • Watching their interactions, you can often find out who is the person with the power to act on your recommendations

So how do you throw a data party? First of all you need to know what you hope to get from the attendees, since you may only be able to hold an event like this one time. There are a number of different ways to organize the event.  You might want to consider using a World Cafe format, where everyone works together to explore a set of questions, or you could use an Open Space system in which attendees create their own agenda about what questions they want to discuss.  Recently the AEA held a very successful online unconference using MIT’s Unhangout that could be used for an online Data Party with people from multiple locations.

The kinds of questions Kylie Hutchinson suggested asking at a data party include:

  • What does this data tell you?
  • How does this align with your expectations?
  • What do you think is occurring here and why?
  • What other information do you need to make this actionable?

At the end of the party it might be time to present some findings and recommendations that you have.  Considering the work that they have done, they may be more willing to listen.  As Kylie Hutchinson said “People support what they helped create.”


Thursday, November 12th, 2015

Guide from NIH Library Informationists [Guest post by Karen Vargas, OERC]

The Medical Library Association’s October 28 webinar was on Data Visualization, presented by Lisa Federer, NIH Library’s Research Data Informationist.  The webinar was a tour of different aspects of data visualization, including information about elements of design, like color, line, contrast and proximity, as well as loads and loads of specific resources for more information.

For those of you who were not able to attend or would like to know more, Lisa Federer has a LibGuide called Creating Infographics with Inkscape, which contains the resources for a class she taught with NIH Informationist Chris Belter.  The LibGuide includes a Power Point from the lecture part of their class. The slides cover design principles and design elements.  Many of the slides have links to resources that you can use to learn more about the topic.  For example:

Vischeck – a cool tool for finding out what your colors in your chart look like to someone who is color blind

10 Commandments of Typography – suggestions for making font combinations that work

The second part of the class is a hands-on section on using Inkscape, a free, open-source graphics program, to make infographics.  Inkscape allows you to use “vector graphics” to design infographics.  What are vector graphics and why use them? You know images that work when they’re small but get all blurry when they get big? Those images are based on pixels. Vector graphics are based on pathways defined by mathematical expressions like lines, curves, and triangles, so they can get larger and smaller without losing any quality. Sounds hard to do, right? Luckily there are tutorials on Inkscape and it’s easier than you might think (you don’t need to know the math…):

If you want to take a look at other vector graphics editors, there are other free ones, like Apache Open Office Draw, or ones you may already own, like Adobe Illustrator.  Comparisons with links to detailed information can be found in Wikipedia’s “Comparison of Vector Graphics Editors.”

Planning Outcomes-Based Outreach Projects

Wednesday, November 4th, 2015

You are invited to join us for our upcoming webinars in a four-part series on Planning & Evaluating Health Information Outreach Projects.

Register on our training calendar.

Planning Outcomes-Based Outreach Projects
Tuesday, December 1st 10:30 – 11:30 AM ET
Come learn how to connect activities to outcomes with a logic model. Participants will have a chance to share ideas for outreach to community partners and get feedback from others.

Collecting and Analyzing Evaluation Data
Thursday, January 14th 10:30 – 11:30 AM ET
The National Network of Libraries of Medicine Outreach Evaluation Resource Center (OERC) leadership will go over the ins and outs of data collection methods. We will learn how to analyze data for quantitative methods and qualitative methods.

Health Information Outreach Project Planning and Evaluation Showcase
Tuesday, April 12th 10:30 – 11:30 AM ET
Share your completed worksheets and activities from the Planning and Evaluating Health Information Outreach Projects booklets. The showcase is open to all NER network members interested to learn more about getting started with community-based outreach, planning outcomes-based outreach projects, and collecting and analyzing evaluation data.

If you missed the first webinar and would like to view the recording, please contact Michelle Eberle at

If you participate in all four sessions of this project, you will receive 8 Medical Library Association Consumer Health Information Specialization credits.  You will only need 4 other credits to qualify for the MLA CHIS Level 1.

This project is led by Margot Malachowski (Baystate Health), Michelle Eberle (NN/LM NER), Cindy Olney (NN/LM OERC), and Karen Vargas (NN/LM OERC) and sponsored by the NN/LM Healthy Communities COI (Community of Interest).

Boosting Response Rates with Invitation Letters

Wednesday, November 4th, 2015

[guest post by Cindy Olney, OERC]

Today’s topic: The humble survey invitation letter.

I used to think of the invitation letter (or email) as a “questionnaire delivery device.”  You needed some way to get the URL to your prospective respondents, and the letter (or, more specifically, the email) was how you distributed the link. The invitation email was always an afterthought, hastily composed after the arduous process of developing the questionnaire itself.

Then I was introduced to Donald Dillman’s “Tailored Design Method” and learned that I needed to take as much care with the letter as I did the questionnaire. A carefully crafted invitation has been proven to boost response rates. And response rate is a key concern when conducting surveys, for reasons clearly articulated in this quote from the American Association of Public Opinion Research:

“A low cooperation or response rate does more damage in rendering a survey’s results questionable than a small sample, because there may be no valid way scientifically of inferring the characteristics of the population represented by the non-respondents.” (AAPOR, Best Practices for Research)

With response rate at stake, we need to pay attention to how we write and send out our invitation emails.

This blog post features my most-used tips for writing invitation emails, all of which are included in Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method by Dillman, Smyth, and Christian (2014). Now in its fourth edition, this book is the go-to resource for how to conduct all aspects of the survey process. It is evidence-based, drawing on an extensive body of research literature on survey practice.

Plan for Multiple Contacts

Don’t think “invitation email.”  Think “communication plan,” because Dillman et al. emphasized a need for multiple contacts with participants to elicit good response rates. The book outlines various mailing schedules, but you should plan for a minimum of four contacts:

  • A preliminary email message to let your participants know you will be sending them a questionnaire. (Do not include the questionnaire link)
  • An invitation email with a link to your questionnaire (2-3 days after preliminary letter)
  • A reminder notice, preferably only to those who have not responded (one week after the invitation email)
  • A final reminder notice, also specifically to those who have not responded (one week after the first reminder).

 Tell Them Why Their Feedback Matters

Emphasize how the participants’ feedback will help your organization improve services or programs. This simple request appeals to a common desire among humans to help others. If applicable, emphasize that you need their advice specifically because of their special experience or expertise. It is best to use mail merge to personalize your email messages, so that each participant is personally invited by name to submit their feedback.

If you are contacting people who have a relationships with your organization, such as your library users or members of your organization, play up that relationship. Also, make a commitment to share results with them at a later date. (And be sure to keep that commitment.)

Make Sure They Know Who’s Asking

With phishing and email scams abounding, people are leery about clicking on URLs if an email message seems “off” in any way. Make sure they know they can trust your invitation email and survey link. Take opportunities to publicize your institutional affiliation. Incorporate logos or letterhead into your emails, when possible.

Provide names, email addresses and phone numbers of one or two members of your evaluation team, so participants know who to contact with questions or to authenticate the source of the email request. You may never get a call, but they will feel better about answering questions if you give them convenient access to a member of the project team.

It is also helpful to get a public endorsement of your survey project from someone who is known and trusted by your participants.  You can ask someone influential in your organization to send out your preliminary letter on your behalf. Also you or your champion can publicize your project over social media channels or through organizational newsletters or blogs.

And How You Will Protect Their Information

Be explicit about who will have access to individual-level data and will know how they answered specific questions. Be sure you know the difference between anonymity (where no one knows what any given participant specifically said) and confidentiality (where identifiable comments are seen by a few specific people). You can also let them know how you will protect their identity, but don’t go overboard. Long explanations also can cast doubt on the trustworthiness of your invitation.

Provide Status Updates

While this may seem “so high school,” most of us want to act in a manner consistent with our peer group. So if you casually mention in reminder emails that you are getting great feedback from other respondents, you may motivate the late responders who want to match the behavior of their peers.

Gifts Work Better Than Promises

The research consistently shows that sending a small gift to everyone, with your preliminary or invitation letter, is more effective than promising an incentive to those who complete your questionnaire. If you are bothered by the thought of rewarding those who may never follow through, keep in mind that small tokens (worth $2-3) sent to all participants is the most cost effective practice involving incentives. More expensive gifts are generally no more influential than small gifts when it comes to response rates. Also, cash works better than gift cards or other nonmonetary incentives, even if the cash is of less value.

Beyond Invitation Letters

The emails in your survey projects are good tools for enhancing response rate, but questionnaire design also matters. Visual layout, item order, and wording also influence response rate. While questionnaire design is beyond the scope of today’s post, I recommend The Tailored Design Method to anyone who plans to conduct survey-based evaluation in the near future. The complete source is provided below.

Source: Dillman DA, Smyth JD, and Christian LM. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method, 4th edition. Hoboken, NJ: Wiley; 2014.

Please visit WP-Admin > Options > Snap Shots and enter the Snap Shots key. How to find your key