Skip all navigation and go to page content

NEO Shop Talk

The blog of the National Network of Libraries of Medicine Evaluation Office

Archive for November, 2016

The Appreciative Inquiry Holiday Challenge

Wednesday, November 23rd, 2016

hand writing something in to the notebook near christmas toys.

The holiday season is upon us, so I want to give our readers a holiday Appreciative Inquiry challenge.  This is a fun way to practice the Appreciative Inquiry interview. It also provides an opportunity for you and your family to plan a better-than-usual holiday season.  Finally, it gives everyone something to talk about other than politics. (You’re welcome.)

During the coming week, ask yourself and your loved ones the following three questions:

  • What was the best holiday experience you’ve ever had?
  • What made that experience so special? What did you value about it?
  • What could happen to make this year’s holiday season exceptional?

Here’s how I would answer the questions.  My favorite holiday was the one I had as a child, traveling to Arizona to spend Christmas with extended family.  For a kid from Western Pennsylvania, Tucson was exotic.  Christmas lights on saguaro cactuses. Luminarias.  Tree ornaments from Mexico. The best part, though, was a trip to the Catalina mountains.

What I valued about that holiday was the differentness of the setting and seeing how those from another part of the country celebrated the holiday. I also liked the bright sunny days outdoors.

It’s a little too late to book a trip to Arizona for the holidays, but I can still seek out places close by that have a different take on holiday decorations. As for enjoying the outdoors, I live in a place that offers lots of opportunity on that front. My husband and I can easily fit in a hike and a trip to Helen, a Bavarian Alpine village in the North Georgia mountains.

Once you’ve talked with your family, make a list of everyone’s ideas for a great holiday and check them off as they happen. You could even do this as a group on a private Facebook page.  Or go old school and put a written list on your refrigerator door.  See if Appreciative Inquiry doesn’t add some sparkle to your holiday season this year.

Happy Thanksgiving, everyone.

 

 

A Chart Chooser for Qualitative Data!

Friday, November 18th, 2016

Core Values Word Cloud Concept

When people talk about data visualization, they are usually talking about quantitative data. In a previous post, we explained that data visualizations help people perform three primary functions: exploring, making sense of, and communicating data.  How can we report qualitative data in a way that performs those same functions?

We just got some exciting news from the EvergreenData blog that they have developed a Qualitative Chart Chooser. Seriously–it’s a work of art. Actually two works of art because they have two different chart chooser drafts to choose from.

The way it works is this: you think about the story you want to tell with your data, maybe about how something improved over time because of your awesome project. Then using the chart chooser, you look at the “show change over time” category, and then you could select a timeline, before-and-after “change photos,” or a histomap (what’s a histomap?  Take a look at this one).

This chart chooser is a very cool tool. But I wouldn’t wait until it was time to report findings to use it. One thing that we at the NEO suggest is that when you are first planning your project, you should think about the story or stories you want to tell at the end of your project. Maybe when you’re thinking about the story you want to tell, you could look at all these different qualitative charts in the chart chooser.  Which ones would you like to use? Do you want to tell the story of how your program aligns with the goals of your institution (you could try indicator dots)? Or maybe you want to show how the different parts of your project work together as a whole (a dendrogram might work). By looking at these options before you design your evaluation plan, you can be sure that you are gathering the right data from the beginning. Backing up even further in your planning process, if you are having trouble trying to decide what story or stories you want to tell, this Qualitative Chart Chooser can give you ways to think about that.

Here is some more information on qualitative data visualization and storytelling from NEO Shop Talk:

Qualitative Data Visualization, September 26, 2014

More Qualitative Data Visualization Ideas, December 18, 2014

Telling Good Stories About Good Programs, June 29, 2015

DIY Tool for Program Success Stories, July 2, 2015

 

Participatory Evaluation, NLM Style

Friday, November 11th, 2016

Road Sign with directional arrow and "Get Involved" written on it.

This week, I invite you to stop reading and start doing.

Okay, wait. Don’t go yet.  Let me explain. I am challenging you to be a participant-observer in a very important assessment project being conducted by the National Library of Medicine (NLM).

The NEO is part of the National Library of Medicine’s program (The National Network of Libraries of Medicine) that promotes use of NLM’s extensive body of health information resources.  The NLM is devoted to advancing the progress of medicine and improving the public health through access to health information. Whether you’re a librarian, health care provider, public health worker, patient/consumer, researcher, student, educator, or emergency responder fighting health-threatening disasters, the NLM has high quality, open-access health information for you.

Now the NLM is working on a long-range plan to enhance its service to its broad user population.  It is inviting the public to provide input on its future direction and priorities. Readers, you are a stakeholder in the planning process. Here is your chance to contribute to the vision. Just click here to participate.

And, because you are an evaluation-savvy NLM stakeholder, your participation will allow you to experience a strength-based participatory evaluation method in action.  Participatory evaluation refers to evaluation projects that engage a wide swath of stakeholders. Strength-based evaluation approaches are those that focus on getting stakeholders to identify the best of organizations and suggest ways to build on those strengths. Appreciative Inquiry is one of the most widely recognized strength-based approaches. The NEO blog have posts featuring Appreciative Inquiry projects here and here.

While I have no idea if the NLM’s long-range planning team explicitly used Appreciative Inquiry for developing their Request for Information, their questions definitely embody the spirit of strength-based assessment. I’m not going to post all of the question here because I want readers to go to the RFI to see the questions for themselves. But as a teaser, here’s the first question that appears in each area of inquiry addressed in the feedback form:

 “Identify what you consider an audacious goal in this area – a challenge that may be daunting but would represent a huge leap forward were it to be achieved.  Include any proposals for the steps and elements needed to reach that goal. The most important thing NLM does in this area, from your perspective.”

So be an observer: check out the NLM’s Request for Information.  Notice how they constructed a strength-based participant feedback form.

Then be a participant: take a few minutes to post your vision for the future of NLM.

Mean, Median or Mode–What’s the Difference?

Monday, November 7th, 2016

Five stars ratings with shadow on white

Last week I taught the class Finding Information in Numbers and Words: Data Analysis for Program Evaluation at the SCC/MLA Annual Meeting in Galveston, TX.  There is a section of the class where we review some math concepts that are frequently used in evaluation, and the discussion of mean, median and mode was more interesting than I expected.  Mean, median and mode are measures of “central tendency,” which is the most representative score in a distribution of scores.  Central tendency is a descriptive statistic, because it is one way to describe a distribution of scores. Since everyone there had run across those concepts before, I asked the group if anyone knew of any clever mnemonics for remembering the difference.  Several people responded, both in class and afterwards (thanks Margaret Vugrin, Julia Crawford and Michelle Malizia!) Here are a couple of memory tools for you:

Mean: Turns out those mean kids are just average (mean = average)

Median: Just like the median in the road: if you line the values up in order, the median is the number “in the middle”

Mode: Mode has an “O” sound, and O is the first letter in “Often.”  It is the value on the list which occurs most often.

Or

Hey diddle diddle, the Median’s the middle; you add and divide for the Mean. The Mode is the one that appears the most, and the Range is the difference between.

Now that you can remember them, which one should you use?  I think a good way to think about it relates to the ratings you see when you’re trying to pick a hotel or restaurant.  I don’t know about you, but when I’m looking at a restaurant score, when I see 4 stars (out of 5), I’m not comfortable with that number without seeing the breakdown.  There’s a big difference between a 4 where the scores are spread out and one where the scores are heavily skewed towards 5.

Let’s say I’m looking at reviews for hotdog restaurants that have around 4.0 stars.  The first one I look at has a distribution of scores spread out relatively evenly from 5 stars (the most number of ratings) to 1 star (the fewest).  A mean (mean kids are just average) works well here.  To do this you add all the scores up and divide the sum by the number of responses to reach a mean of 3.9.

3-9-star-mean

Here is the chart for a similar hotdog restaurant with a slightly higher 4.2 star rating:

4-2-star-median

While the mean adds up to 4.2,  you can see in the chart that the scores are heavily skewed towards 4 and 5 stars, and it seems that 4.2 does not accurately describe the ratings of that restaurant (remember that central tendency is a descriptive statistic). However, if you use a (middle of the road) median with this kind of distribution, the result is a whopping 5.  To determine the median, you line all the results up in order and select the one in the very middle.  You might want to use a median when the distribution looks skewed.

This particular restaurant analogy doesn’t really work with mode. Mode is used for categories, which really cannot be averaged mathematically.  For example, if you want to know what is the most representative type of restaurant in your city, you might find out that your city has more hotdog restaurants than any other kind of restaurant (that would be awesome, right?).

I hope this helps. If you know any other mnemonics for mean, median, or mode, please send me an email at kjvargas@uw.edu and I will add them to the bottom of this post.

 

Last updated on Monday, June 27, 2016

Funded by the National Library of Medicine under Contract No. UG4LM012343 with the University of Washington.