Skip all navigation and go to page content

NEO Shop Talk

The blog of the National Network of Libraries of Medicine Evaluation Office

Archive for the ‘Library Value’ Category

Inspirational Annual Reporting with Appreciative Inquiry

Friday, March 25th, 2016

Hiker enjoying the view along the Iceberg Lake trail in Glacier National Park

Do you have to file annual reports? How much do you love doing them?

Did I hear someone say “no comment?”

In January, I challenged the outreach librarians of the National Network of Libraries of Medicine Greater Midwest Region (NN/LM GMR) to experiment with a reflective exercise designed to add some inspiration to their annual reporting. The setting was a monthly webinar attended by librarians who led outreach activities at their libraries to promote health information access and use. Because their libraries received funding from the NN/LM GMR, they were required to submit annual reports for their funded activities.

My charge was to teach this group something about evaluation. In response, I presented them with this short (about 15 minute) exercise to be used when they began preparing their annual reports.

When preparing your report, answer these questions. Then write a short paragraph based on your answers and add it to your annual report:

  1. Describe one of the best experiences you had this year conducting outreach for the NN/LM.
  2. What do you value most about that experience?
  3. What do you wish could happen so that you had more experiences like this?

You may recognize these as the three signature questions of the basic Appreciative Inquiry (AI) interview. Appreciative Inquiry is a practice of influencing organizational change by identifying peak experiences and discovering ways to build on them. The book Reframing Evaluation through Appreciative Inquiry (Preskill and Catsambas, Sage, 2006) provides descriptions and examples of how to apply AI to every part of the evaluation process.

My partners for this webinar were host Jacqueline Leskovec, Outreach, Planning and Evaluation Coordinator, and presenter Carmen Howard, who is the Regional Health Sciences Librarian and Visiting Assistant Professor from UIC Library of the Health Sciences Peoria. Carmen headlined the webinar with her presentation about the Nursing Experts: Translating the Evidence (NExT) Guide, which provides resources on evidence-based practice to nurses. Good sport that she was, Carmen helped me demonstrate the exercise to our audience by participating in an AI interview about her outreach project. The outreach librarians then brainstormed ways to use the three questions to prepare their own NN/LM reports. We also talked about how to add their reflective statements to their annual reports, which are entered into an online system.

Soon after that webinar, Carmen wrote an entry about her experience using the three questions that appeared in the NN/LM GMR’s blog The Cornflower. Here is my favorite quote from her entry:

“These three simple questions which only take about 10-15 minutes to answer forced me to stop and reflect on the NExT project. Rather than just being focused on what was next on the to-do list, I was looking back on what had already been accomplished, and better yet, I was thinking about the good stuff.”

The NN/LM GMR outreach librarians have not yet filed their 2016 annual reports, so I can’t tell you how many rose to my challenge. (This exercise was a suggestion, not a requirement.) One other outreach librarian did send an email to say she was using the three questions to have a reflective group discussion with other librarians who participate in NN/LM outreach activities.

I would like to extend the challenge to our readers who may be facing annual reports. Try this exercise and see if you don’t start thinking and writing differently about your efforts over the past year.

If you want to know more about Appreciative Inquiry, we highly recommend this source:

  • Preskill H, Catsambas TT. Reframing Evaluation through Appreciative Inquiry. Thousand Oaks, CA: Sage, 2006.

You also might be interested in the OERC’s other blog posts about Appreciative Inquiry:

If you are interested in earning some continuing education credits from the Medical Library Association while trying your hand at an Appreciative Inquiry project, reach this post: Appreciative Inquiry of Oz: Building on the Best in the Emerald City 

 

Appreciative Inquiry of Oz: Building on the Best in the Emerald City

Friday, February 19th, 2016

Cartoon image of an Emerald City

“One day not very long ago, librarians came to the Emerald City from their libraries in all of the countries of Oz. They came to visit the Great Library of the Emerald City, and to petition the Wizard allow them to borrow books and other items at the Great Library. Their hope was to transport items from one library to another using the Winged Monkeys, who offered their skills for this task after they were set free and got bored.”

Thus begins the latest OERC project – an online class in Appreciative Inquiry (AI), offered through the MidContinental Region’s Librarians in the Wonderful Land of Oz Moodle ‘game’ (i.e. series of online classes worth game points and CE credits from the Medical Library Association).  The game is made up of several ‘challenges’ (online classes) for librarians offered by NN/LM instructors.

In OERC’s challenge, Building on the Best at the Great Library of the Emerald City: Using Appreciative Inquiry to Enhance Services and Programs, the Wizard of Oz makes a deal with the librarians.  He will allow interlibrary loan of the Great Library’s resources if the librarians will assess customer satisfaction of the Great Library’s services and find things to improve.  And students in the class will learn to use a qualitative data collection technique called Appreciative Inquiry to do this assessment.

Sometimes people avoid customer service assessment because they find the methods to be complicated and time-consuming. Negative feedback can be uncomfortable on the part of the listener and the speaker. Appreciative Inquiry, with a focus on identifying and building on organizational strengths, removes that discomfort. A number of OERC workshops touch on Appreciative Inquiry but this Librarians of Oz challenge allows you to practice the technique, something that the OERC has not been able to provide in the traditional webinar or workshop context.  Completing the class is worth 14 MLA CE credits.

The class is free, but in order to take it you will need to register for the game Librarians in the Wonderful Land of Oz .  If you don’t want to take the class, but would still like to learn more about Appreciative Inquiry, I recommend these earlier blog posts:

From Cindy and Karen’s perspective, one of the best parts of this experience is that we finally get the official title of Wizard.  Special thanks to John Game Wizard Bramble of the NN/LM MCR who made all this happen.

 

New Outcome Measurement Resource for Public Libraries

Friday, January 22nd, 2016

Librarian and children looking at globe in library

About six months ago Public Library Association (PLA) initiated a service called Project Outcome, that I have been following with interest. An article entitled “Project Outcome – Looking Back, Looking Forward” by Carolyn Anthony, director of the Skokie Public Library, IL was recently published in Public Libraries Online that describes the successes of libraries using this service over the past 6 months.

Project Outcome is an online resource that provides evaluation tools that are designed to measure the impact of library programs and services, such as summer reading program or career development programming. It also provides ready-made reports and data dashboards that can be used to give libraries and stakeholders immediate data on their programs’ outcomes.  And Project Outcome provides support and peer sharing opportunities to address common challenges and increase capacity for outcomes evaluation.

Here are some of the things that make me the most excited about this service:

  1. Project Outcome has managed to create a structured approach for program outcome evaluation that can be used online by public libraries of all shapes and sizes by people who have not done outcome evaluation before.  Along with tools for collecting data, the resource has tutorials and support for libraries doing outcomes evaluation for the first time.
  2. Continued support and peer sharing as an integral part of the service means that PLA is building a community of librarians who use outcome evaluation.
  3. The stories that are shared by the peers as described in the article will increase the understanding that evaluation isn’t something forced on you from outside, but can be something that helps you to create a better library and enhance the meaning of your library’s programs.
  4. This process teaches librarians to start with the evaluation question (“decide what you want to learn about outcomes in your community”) and a plan for what to do with the findings. And the process ends with successfully communicating your findings to stakeholders and implementing next steps.
  5. Lastly, I love that Project Outcome and the PLA Performance Measurement Task Force are planning the next iteration of their project that will measure whether program participants followed through with their intended outcomes.  It will be very interesting to find out how this long term outcome evaluation comes out.

I’ll end with this statement from Carolyn Anthony, who said “the opportunity to spark internal conversations and shift the way libraries think about their programs and services is what outcome measurement is all about.”

Group Learning about Evaluation

Friday, October 23rd, 2015

Group of young business people talking on business meeting at office.

I recently got to participate in a very successful roundtable at a library conference.  I co-moderated an evaluation roundtable entitled “Library assessment: You’ve measured your success – now how do you get people to listen?” with OU-Tulsa Schusterman Library Associate Director Katie Prentice at the South Central Chapter of the Medical Library Association Annual Meeting in Little Rock, AR.

What makes roundtables unique among educational opportunities at library conferences is that unlike presentations or papers where attendees sit and listen, in a roundtable everyone can participate. It is a moderated discussion on a given topic among the people who attend, and since anyone can chime in, learning is active instead of passive.

About 25 people attended this roundtable and enthusiastically participated in a discussion about library assessment and evaluation data. Katie and I led the discussion with questions starting from what kind of data you collect at your library and leading to what libraries do with the data and how to make it work better for them. Our goal was to use our questions to all issues and solutions to come from the attendees themselves.

As an example, when we asked the question “what would you really like to know about your library and what do you dream of finding out about your users?” one hospital librarian said that she wanted to know how doctors were using the information and how it impacted the patients. Katie Prentice asked “can anyone help her with this?” and another hospital librarian responded that she sends emails to some of her doctors to ask for a sentence or two describing how the information was used.  These sentences, when collected and analyzed, could be a powerful tool to show hospital administration the importance of the library to patient outcomes.

Other kinds of evaluation ideas that were generated from attendees at this roundtable were:

  • using heat map software to determine where people go most often on your website
  • having student workers note what pieces of furniture are being used to improve furniture types and placement in the library
  • using a product like Constant Contact or Mail Chimp to send library newsletters to the doctors and employees at hospitals with assessment data.

While not all roundtables at conferences are this successful, this roundtable demonstrated the ability of librarians brought together in a group to learn from each other and solve problems.

Creative Annual Reports

Friday, October 9th, 2015

Ah, the annual report – at its best we expect to see a glossy booklet with pie charts, short paragraphs and some quotes. At its worst it can be pages of dry text. Our main hope with annual reports is that our stakeholders and others will read them and be impressed with the successes of our organizations.

Last month I ran across the annual report from the Nowra Public Library in New South Wales, Australia, which was so compelling and understandable that over 100,000 people have viewed it on YouTube:

Photo of librarian links to Nowra library video

Since most organizations don’t have the resources to do their own music video (e.g. singers, writers, silly costumes), I thought I would look at a few other examples to consider when it’s time to do your annual report.

One of my all-time favorites is the annual report from the Schusterman Library of The University of Oklahoma-Tulsa. Their annual report is an infographic that shows the data that is collected, but also describes the data in such a way that 1) you have a better feel for what is going on in the library; and also 2) you might think “I didn’t know they would help me with that!”  For example: “7,274 Reference questions answered in person, by phone, by email, and instant message or text on everything from ADHD and child welfare to decision trees, LEED homes, and census reporting.” It is available on their website, and the librarians at the Schusterman Library say they frequently find students looking at it.

The Michigan State University College of Education won a gold ADDY and a Best in Show Award for their 2012 Annual Report (an ADDY is the advertising industry’s largest competition).  Their report featured a tri-fold, die-cut skyline that presented the college’s missions and strengths with an emphasis on “college as community.” The annual report also included a video and a website that gives detailed narratives that show institutional successes in terms of personal stories.

Of course, not all institutions want an unusual annual report.  But it is important to consider the target audience.  Annual reports reach the upper administration, potential funders, and patrons of the library. The success of this years annual report might shape the library users view of the library for years to come.

The OERC is on the Road Again

Friday, September 25th, 2015

The OERC is on the road again.  Today, Cindy and Beth Layton, Associate Director of the NN/LM Greater Midwest Region, are team-teaching Measuring What Matters to Stakeholders at the Michigan Health Sciences Library Association’s annual conference in Flint, MI.

Logo for Michigan Health Sciences Library Association

This workshop covers strategies for using evaluation to enhance and communicate a library’s value to organizational decision-makers and stakeholders who influence decision makers. The workshop combines updated information with material from the NN/LM MidContinental Region’s Measuring Your Impact and the OERC’s Valuing Your Library workshops that have been taught by a number of regional medical library staff members over the past decade.

On Saturday, Karen is presenting a brand-new workshop for the Texas Library Association’s District 8 Conference called Adding Meaning to Planning: A Step-by-Step Method for Involving Your Community in Meaningful Library Planning.

TLA District 8 Logo

The workshop is a method of involving community members in creating pain-free logic models to ensure that the long term vision is always in sight when planning.  Karen wrote a blog entry about the creating “tearless” logic models here. http://nnlm.gov/evaluation/blog/2015/04/10/tearless-logic-models/  This is Karen’s first experience creating and delivering a workshop that is purely about library evaluation.

The NN/LM travel season is about to go into full swing.  We know we aren’t the only ones out and about with presentations, trainings, and exhibits.  So safe travels. And we will see you in a week with another OERC blog post.

Improving Your Data Storytelling in 30 Days

Friday, August 21st, 2015

Here are some more great techniques to help with telling a story to report your evaluation data so it will get the attention it deserves.

Friends at campfire telling storiesJuice Analytics has this truly wonderful collection of resources in a guide called “30 Days to Data Storytelling.” With assignments of less than 30 minutes a day, this guide links to data visualization and storytelling resources from sources as varied as Pixar, Harvard Business Review, Ira Glass, the New York Times, and Bono (yes, that Bono).

The document is a checklist of daily activities lasting no longer than 30 minutes per day. Each activity is either an article to read, a video to watch, or a small project to do.

The resources answer valuable questions like:

  • What do you do when you’re stuck?
  • How do I decide between visual narrative techniques?
  • Where can I find some examples of using data visualization to tell a story?

Easy Email Newsletters for Keeping Stakeholders Informed

Friday, June 19th, 2015

Do you find it difficult to ensure that you are keeping your stakeholders up to date throughout your program? In her AEA Summer Institute class entitled “An Executive Summary is Not Enough: Effective Evaluation Reporting Techniques,” Kylie Hutchinson from Community Solutions suggests a very interesting product for staying in touch with stakeholders. The product is Constant Contact, a tool that allows you to put the latest stats, activities, or planning updates into a newsletter format that goes into the body of an email, allowing stakeholders to easily stay updated on the progress of your program. In addition, you, the evaluator, get feedback on who opens the email and what they click on.

Here is a sample email created for free in just a few moments, showing NN/LM stakeholders what activities took place in May in an imaginary community college outreach project.

Constant Contact

Infographics Basics: A Picture is Worth 1000 Data Points

Friday, April 24th, 2015

Open Access Week at University of Cape Town infographicYou’ve been collecting great data for your library, and now you have to figure out how to use it to convince someone of something, for example how great your library is. Part of the trick is turning that data into a presentation that your stakeholders understand – especially if you are not there to explain it.  Infographics are images that make data easy to understand in a way that gets your message across.

It turns out it doesn’t have to be difficult or expensive to create your own infographics.  Last week I went to a hands-on workshop at the Texas Library Association called “Infographics: One Picture is  Worth 1,000 Data Points,” taught by Leslie Barrett, Education Specialist from the Education Service Center Region 13 in Austin, TX. Using this website as her interactive “handout” http://r13hybrarian.weebly.com/infographicstla.html, Leslie walked us through the process of creating an infographic (and as a byproduct of this great class, she also demonstrated a number of free instructional resources, such as Weebly, Padlet, and Thinglink).

Starting at the top of the page, click on anything with a hyperlink.  You will find a video as well as other “infographics of infographics”  which demonstrate how and why infographics can be used.  There are also a variety of examples to evaluate as part of the learning process.

Finally, there is information on the design process and resources that make infographics fairly easy to create.  These resources, such as Piktochart and Easelly, have free subscriptions for simple graphics and experimenting.

Leslie Barrett allowed us to share this website with you, so feel free to get started making your own infographics!

Image credit: Open Access Week at University of Cape Town by Shihaam Donnelly / CC BY SA 3.0

New Journal: Systematic Reviews

Saturday, April 19th, 2014

Librarians’ expert searching skills provide some great opportunities for collaboration with researchers. Biomed Central’s new open access Systematic Reviews journal is about a specialized type of expert searching that librarians can provide for their communities. More than a source of protocols and a record of others’ work, this journal has great potential for those of us in academia who want to publish articles to share information with our colleagues about what we have done.

Here’s more information from the Aims and Scope:

Systematic Reviews encompasses all aspects of the design, conduct and reporting of systematic reviews. The journal aims to publish high quality systematic review products including systematic review protocols, systematic reviews related to a very broad definition of health, rapid reviews, updates of already completed systematic reviews, and methods research related to the science of systematic reviews, such as decision modeling. The journal also aims to ensure that the results of all well-conducted systematic reviews are published, regardless of their outcome.

It is a long-term goal of the journal to ensure all systematic reviews are prospectively registered in an appropriate database, such as PROSPERO, as these resources for registration become available and are endorsed by the scientific community.

Article types include:

  • Research Articles
  • Commentaries
  • Letters
  • Methodologies
  • Protocols
  • Review Updates

The editors-in-chief comprise an international group hailing from the University of Ottawa; the RAND corporation and UCLA; and the University of York.

Take a look at this journal! It could be a source of inspiration for any librarian whose emphasis is on expert searching.

Last updated on Thursday, May 19, 2016

Funded by the National Library of Medicine under contract # HHS-N-276-2011-00008-C.