Skip all navigation and go to page content
NN/LM Home About OERC | Contact Us | Feedback |OERC Sitemap | Help | Bookmark and Share

Archive for the ‘Practical Evaluation’ Category

Qualitative Data Visualization

Flowchart of text, text with illustrations, then illustrations leading to additional text

Have you thought that only quantitative information can be used for data visualizations, and qualitative data wasn’t an option without first coding or otherwise turning this valuable content into quantitative formats?

I learned about an innovative and compelling approach to creating qualitative data visualizations with illustrations from Fresh Spectrum . They begin the process (as shown in the illustration above) of taking a long narrative such as a focus group transcription, and chunking it into a few paragraphs per concept with a unique illustration for each one. In this case custom illustrations of people were used, but you could use your organization’s existing images or Creative Commons-licensed images for illustrating concepts. The next step for the visualization uses the images with brief captions as an online data dashboard, where visitors can click on the captioned image of interest to them to then access the more detailed narrative. The author describes how to do this within a WordPress portfolio blog template, or a simpler strategy of creating HTML anchor links to each individual section within a longer text. You can see how this works by clicking on an anchor link from the original post (http://freshspectrum.com/blogging-advice/#davidson for example) that leads to the longer narrative at http://freshspectrum.com/blogging-advice/ (a great source of advice for blogging by the way!)

Need more information about reporting and visualizing your data? We at the Outreach Evaluation Resource Center (OERC) have more resources available for you from the Reporting and Visualizing tab of our Tools and Resources for Evaluation Guide at http://guides.nnlm.gov/oerc/tools and welcome your suggestions and comments about the guide.

Freebie Friday: AEA’s Potent Presentation Information

Logo with focus on message, design, and delivery

The American Evaluation Association (eval.org) sponsors a Potent Presentations Initiative (p2i) that has a stated purpose of helping evaluators improve their presentation skills, both within a conference setting and as part of individual practice. P2i challenges evaluators to hone in on three concepts: Their message, their design, and their delivery.

There are a wealth of handouts available as PDF files,Word documents and Powerpoint presentations available from the p2i tools website (http://p2i.eval.org/index.php/p2i-tools/) that sometimes include AEA conference specifications in addition to many great messaging, designing and delivery principles. For an example of each principle be sure to check out the Presentation Preparation Checklist (from 3 months ahead of time to afterwards to include modifications while the information is freshly in mind), How to Design a Research Poster (great infographic visualization and instructions on how to make your data ‘pop’), and the Delivery Glue Handout (did you know as a general rule it takes 16 times the length of your talk to make presentation slides and a script?).

Logic Models for Library Assessment Planning

The Engaged Librarian: Crafting an Effective Assessment Plan to Determine the Impact of a Key Strategic Library Initiative by Sarah Murphy at The Ohio State University (OSU) was presented during the Library Assessment Conference and provided an overview to the use of a logic model as part of library strategic planning. Ms. Murphy’s presentation slides are available by clicking here.

Their project incorporated the theory of change methodology with logic models and used the Kellogg Foundation Logic Model as a template. They storyboarded data within a data dashboard that was both aligned with and broken down by the applicable OSU strategic vision goals. Ms. Murphy reported that the benefits of using a logic model approach included having a flexible but structured way to do library assessment planning, having a collaborative and inclusive approach, creating a project focus, being able to assess linear and iterative programs and services, and the ability to communicate program accomplishments in interesting ways. During the question and answer session they noted they are also Tableau fans (we will write about Tableau for our next post) and like to create data structures in their dashboard to avoid information silos.

If you’d like to learn more about logic models and data dashboards, be sure to check out our freely available Outreach Evaluation Resource Center (OERC) Evaluation Guides, especially Planning Outcomes-Based Outreach Projects. We also offer Data Dashboards: Monitoring Progress toward Program Outcomes as one of our webinars and a recording of Data Dashboards is available by clicking here.

Freebie Friday: Shaping Outcomes Course

logo for Shaping Outcomes class

Do you want to learn more about outcomes-based planning and evaluation (OBPE) for your outreach project but there’s no money in the training budget to do so?

Shaping Outcomes: Making a Difference in Libraries and Museums (shapingoutcomes.org) is available as a free online course that learners can start anytime and work on at their own self-navigated pace. While there are library and museum-specific examples provided in the course the concepts of learning more about target audience needs, how to clarify desired results, developing logic models and evaluating outcomes are applicable for most other organizations’ outreach projects as well.

Modules of the class are broken into five sections (Overview, Plan, Build, Evaluate, Report) with a helpful Glossary to learn OBPE terminology and a Logic Model template. Shaping Outcomes was developed by the Institute of Museum and Library Services (IMLS) and Indiana University/Purdue University Indianapolis (IUPUI) and previously was available as an instructor-led class.

More information specific to developing logic models in health information outreach programs is available from Booklet Two: Planning Outcomes-Based Outreach Projects, part of our resources on our Outreach Evaluation Resource Center (OERC) Evaluation Guides page at http://nnlm.gov/evaluation/guides.html.

 

Freebie Friday: OERC Data Burger Webinar

veggie burger via Flickr
Veggie burger by Dan McKay, Flickr Creative Commons license

The Outreach Evaluation Resource Center (OERC) was pleased to present our Data Burger: A “Good” Questionnaire Response Rate plus Basic Quantitative Data Analysis webinar for the National Network of Libraries of Medicine, New England Region (NN/LM NER) Health Care Workforce Community of Interest (COI) this week. Thanks to NER for freely sharing the 1 hour webcast recording at https://webmeeting.nih.gov/p87ze5jc0do/.

Are you interested in this and other webinar training sessions the OERC has to offer? You can learn more about Data Burger and others from our presentation listing at http://nnlm.gov/evaluation/workshops/ and let us or your Regional office know that you’d like us to present a webinar. We welcome the opportunity to work with you!

 

Freebie Friday: American Evaluation Association Online Public Library

As we at the Outreach Evaluation Resource Center (OERC) learn more about great evaluation resources available at an even better price (free!) in addition to our own freely available resources (http://guides.nnlm.gov/oerc/tools), we will feature some of them here for you to explore on ‘Freebie Friday’ as an occasional series.

To begin our coverage, did you know the American Evaluation Association (AEA) has an online public library of AEA conference presentations, assessment instruments (such as rubrics and logic models), and more available at http://comm.eval.org/communities/resources/libraryview/? The default library setting displays those files most recently updated in the online library and offers keyword searching, but I recommend going straight to the more advanced search functionality at http://comm.eval.org/communities/resources/searchlibrary/.

(more…)

SlideDocs help you create readable reports

Thanks to evolving Internet and mobile technology, most of us now prefer our information presented in tapas-sized quantities, preferably in a visually appealing way. This preference is having an impact on our business communication. Before our eyes, bullet points on PowerPoint slides are gradually giving away to engaging visuals. Infographics are replacing text-dense brochures and executive summaries.

Nancy Duarte, who is motivating  the world to clean up its PowerPoint presentations, is venturing into a new frontier: Written reports. She suggests creating reports using presentation software like PowerPoint, which is more amenable to manipulating layouts. The idea is to present one idea per slide, incorporating visuals, text and much-appreciated white space.

Duarte calls this type of report a SlideDoc and offers a free tutorial on how to create one (presented in, you guessed it, a PowerPoint document). The site includes downloadable templates to help you create your own SlideDoc report. You also can find Duarte’s Diagrammer here, which I wrote about in a previous post.  I believe, at this point, that I should state explicitly that I do not own stock in her company. I just love her approach to presenting ideas and I think evaluation reports would be read and heard by more people if we all started using her guidelines.

As someone who wants to stop producing coma-inducing reports, I have tried my hand at creating more engaging layouts using Microsoft Word. What I learned is this: Take your anticipated timeline for report-writing and double it. Word does not lend itself to interesting layouts, so inserting pictures, graphs, and call-out quotes is like putting shoes on a toddler. You can do it, but it’s probably going to make you late.

Consequently, my results have been underwhelming.

So I‘m excited to try SlideDocs. I already can see how much more flexible PowerPoint will be for arranging text, images, and graphs. I won’t be wasting time adjusting margins and re-positioning graphs that always seem to wander around in Word documents. I also like that I don’t have to learn a new software application, nor do I have to get my audience to download a special reader to view my reports.

Just remember: Shorter isn’t easier when it comes to reports. These “to-the-point” SlideDocs require that you know both your findings and your audience extremely well so you can communicate the most important information in the most succinct way. SlideDocs will help with the layout, but you still have to do the thinking.

Evaluating your social media activities

For public sector and nonprofit organizations, social media can be a cost-effective way to engage with users and supporters. However, social media is not without its cost, particularly in terms of staff time.  So organizations have an interest in assessing the value of their social media activities.

One great resource for social media evaluation is Paine’s book, Measure What Matters.  The book contains detailed guidance for evaluating social media use by different types of organizations. A great supplement to Paine’s book is The Nonprofit Social Media Decision Guide by Idealware, which has worksheets that will help you plan your social media strategies and implement recommendations in Measure What Matters.

Below are the key elements of Paine’s evaluation framework:

  • Begin with a solid social media plan that identifies specific goals and objectives. As with any project, you need a plan for social media that links strategies to the organizational mission and includes objectives with targets and key performance indicators. Objectives for social media in the public-sector often belong in one of two categories: helping users find information they need; or building user awareness, engagement, or loyalty. (To inspire you, The Nonprofit Social Media Decision Guide provides a list of potential objectives on page 52.)
  • Define your target audience: Organizations often have many stakeholder groups, so you want to identify the groups most attuned to social media. On page 54 of The Nonprofit Social Media Decision Guide, you’ll find a worksheet for narrowing down your stakeholder audiences to those most receptive to your social media activities.
  • Pick your metrics: Metrics such as views, followers, and measures of engagement with online content will help you monitor reach. Conversions, defined as the actions you want your social media followers to complete, might include becoming members of your organization or actively recommending your organization to colleagues or friends.
  • Identify a source for benchmarks. Benchmarks provide a basis for comparison to assess progress. Organizations often use their own histories as benchmarks, comparing progress against baseline measures. You also may have access to data from a competing or peer organization that you can use for comparison.
  • Pick a measurement tool: Paine’s book describes different measurement methods for evaluating social media, such as content analysis, web analytics, or surveys.

For more information, check out the resources used for this blog post:

  • Katie Delahaye Paine, Measure What Matters: Online Tools For Understanding Customers, Social Media, Engagement, and Key Relationships. Hoboken, NJ: John Wiley & Sons, Inc, 2011.
  •  Idealware. The Nonprofit Social Media Decision Guide, 2013.

4,000+ free diagram templates

“If people can see what you’re saying, they’ll understand it.”  This quote comes from the Perspectives page of Duarte.com, which now offers a free Diagrammer that provides more than 4,000 free, downloadable diagram templates to help you present your evaluation findings visually.

A handy directory helps you determine the type of diagram you need based on the data relationships you want to portray. You then choose a diagram and download it into a PowerPoint-ready image that is completely customizable. You add text, change font size and color, even move or eliminate parts of the diagram. The PowerPoint slide can be inserted into a slide file for an oral presentation or saved as an image and inserted into a written evaluation report.

I tried my hand at creating a diagram to show how NN/LM train-the-trainer programs encourage the spread of health information resource use. This is what I came up with:

Diagrammer image

I learned about this tool  from an AEA365 blog post by Sheila Robinson. As an example, she included an infographic she designed using the Diagrammer to illustrate American Evaluation Association learning opportunities.

Duarte.com is the company of Nancy Duarte, a master presentation designer who has become a favorite among evaluators on a mission to get their evaluation reports understood and used. If you are interested in punching up the impact of your presentations, you also might want to check out her book “Resonate” (2010, Wiley & Sons) or watch her popular TEDtalk The Secret Structure of Great Talks. Many of her principals can be applied to oral or written presentations.

 

 

 

New Journal: Systematic Reviews

Librarians’ expert searching skills provide some great opportunities for collaboration with researchers. Biomed Central’s new open access Systematic Reviews journal is about a specialized type of expert searching that librarians can provide for their communities. More than a source of protocols and a record of others’ work, this journal has great potential for those of us in academia who want to publish articles to share information with our colleagues about what we have done.

Here’s more information from the Aims and Scope:

Systematic Reviews encompasses all aspects of the design, conduct and reporting of systematic reviews. The journal aims to publish high quality systematic review products including systematic review protocols, systematic reviews related to a very broad definition of health, rapid reviews, updates of already completed systematic reviews, and methods research related to the science of systematic reviews, such as decision modeling. The journal also aims to ensure that the results of all well-conducted systematic reviews are published, regardless of their outcome.

It is a long-term goal of the journal to ensure all systematic reviews are prospectively registered in an appropriate database, such as PROSPERO, as these resources for registration become available and are endorsed by the scientific community.

Article types include:

  • Research Articles
  • Commentaries
  • Letters
  • Methodologies
  • Protocols
  • Review Updates

The editors-in-chief comprise an international group hailing from the University of Ottawa; the RAND corporation and UCLA; and the University of York.

Take a look at this journal! It could be a source of inspiration for any librarian whose emphasis is on expert searching.