The NN/LM Outreach Evaluation Resource Center (OERC) has developed the new webinar Evaluation 2.0: Trends, New Ideas, Cool Tools,” which debuted on August 21 as one of the NN/LM Greater Midwest Region’s monthly Lake Effect webinars. The new webinar presents emerging trends in evaluation practice that emphasize stakeholder interaction and social engagement. It also covers popular tools and methods that allow you to draw others into the evaluation process and raise the visibility of your program or services. A recording of the one-hour session is available for viewing.
Archive for the ‘General’ Category
At this month’s Library Assessment Conference held in Seattle, one panel featured assessment librarians presenting data dashboards they created using Tableau software, Tableau Unleashed: Visualizing Library Data. This presentation includes views of dashboards from University of British Columbia Library (by presenter Jeremy Buhler), UMass Amherst Libraries (by Rachel Lewellen), and Ohio State Libraries (by Sarah Murphy). All of the presenters used Tableau software to produce their dashboards.
Tableau may be the most popular software for creating dashboards right now and the company offers a free version that has a great deal of functionality. In fact, at least one presenter (Sarah Murphy) included dashboards she created using Tableau Public. However, users must be cautioned that any data entered into Tableau Public become public information. That means anyone can see and download your raw data. So, if you use it, be sure all identifying information about individuals is stripped from your files and that you are comfortable with other people downloading your raw data. The presenters also mentioned tips for dashboard design. For additional design guidance, check out the freely downloadable resource A Guide to Creating Dashboards People Love to Use by Juice Analytics.
The Engaged Librarian: Crafting an Effective Assessment Plan to Determine the Impact of a Key Strategic Library Initiative, by Sarah Murphy at The Ohio State University (OSU), was presented during the 2014 Library Assessment Conference, held earlier this month in Seattle, WA. The presentation provided an overview to the use of a logic model as part of library strategic planning. The project incorporated the theory of change methodology with logic models and used the Kellogg Foundation Logic Model as a template. They storyboarded data within a data dashboard that was both aligned with and broken down by the applicable OSU strategic vision goals. Ms. Murphy reported that the benefits of using a logic model approach included having a flexible but structured way to do library assessment planning, having a collaborative and inclusive approach, creating a project focus, being able to assess linear and iterative programs and services, and the ability to communicate program accomplishments in interesting ways. During the question and answer session they noted they are also Tableau fans and like to create data structures in their dashboard to avoid information silos.
To learn more about logic models and data dashboards, check out the freely available NN/LM Outreach Evaluation Resource Center (OERC) Evaluation Guides, especially Planning Outcomes-Based Outreach Projects. Also available is a recording of the one-hour presentation Data Dashboards: Monitoring Progress toward Program Outcomes, part of the NN/LM PSR Midday at the Oasis webinar series.
The American Library Association has announced the release of Community Conversation Workbook, a resource developed for the ALA’s Libraries Transforming Communities (LTC) initiative, which provides librarians with training and resources to enhance their roles as community leaders and change-agents. The initiative’s goal is to help librarians promote the visibility and value of their libraries within their communities. Public discussions are promoted as key community engagement strategies. The workbook provides invaluable guidance to anyone who wants to conduct discussion groups for community assessment purposes. It provides practical advice on every aspect of convening group discussions, including tips on participant recruitment, a list of discussion questions, facilitator guidelines, note-taking tools, and templates for organizing key findings.
Demonstrating value is currently of considerable interest to many libraries and organizations. Such organizations may be interested in exploring other articles and resources related to the LTC initiative, which are available on the LTC web page. Examples showing how libraries are implementing LTC activities are available from the initiative’s digital portal.
Shaping Outcomes: Making a Difference in Libraries and Museums is available as a free online course that learners can start anytime and work on at their own self-navigated pace. While there are library and museum-specific examples provided in the course, the concepts of learning more about target audience needs, how to clarify desired results, developing logic models, and evaluating outcomes are applicable for most any organization’s outreach projects. Modules of the class are broken into five sections; Overview, Plan, Build, Evaluate, and Report, with a helpful Glossary to learn outcomes-based planning and evaluation (OBPE) terminology, and a Logic Model template. Shaping Outcomes was developed by the Institute of Museum and Library Services (IMLS) and Indiana University/Purdue University Indianapolis (IUPUI) and previously was available as an instructor-led class.
More information specific to developing logic models in health information outreach programs is available from NN/LM Outreach Evaluation Resource Center (OERC) Booklet Two: Planning Outcomes-Based Outreach Projects. Additional information is available on the OERC Evaluation Guides page.
With an increase of technology tools available for data reporting and visualization sometimes it’s challenging to know how to best use them to clearly communicate the intended meaning of the data. The concept of visualization literacy and a broader theme of visual literacy are often not included as part of the instructions guiding people in the steps to create their own visualization design. A recent entry by Andrew Kirk on the blog of Seeing Data, a research project in the United Kingdom studying how people understand big data visualizations shown in the media, offers a great review of 8 Articles Discussing Visual and Visualization Literacy that are freely available and well worth a read to better understand both visual and visualization literacy. Their featured articles include resources ranging from the importance of Visual Literacy in an Age of Data to How to Be an Educated Consumer of Infographics, and Seeing Data has asked that you share additional ones with them via blog comments or their Twitter social media account @SeeingData.
Community Health Maps Blog is an initiative designed to share information about free and low cost and easy-to-use applications of Geographic Information Systems (GIS) mapping tools. The goal is to help community-based and other types of small organizations collect and visualize information about their communities with an eye towards using these techniques to support planning and decision-making about community health. The tools discussed on the Community Health Maps Blog can support the collection and visualization of health statistics, demographic information, community resources, and events, thereby facilitating a better understanding of community conditions.
The interactive nature of blogging helps Community Health Maps share information about hardware platforms and software applications available to communities as they consider how, or if, they might use GIS. NLM encourages the submission of blog postings by those who use such resources to carry out projects within their communities, as well as those who have identified additional applications that may be of interest for this purpose.
The National Library of Medicine’s Journal Donation System makes it possible for libraries to determine whether NLM needs any volumes of the print journals they plan to discard. The system can be used by DOCLINE and non-DOCLINE libraries to offer any title, including titles not owned by NLM. The system can be accessed directly through the web site or by searching “Journal Donation System” on NLM’s home page. In the system, click on “Help” for detailed instructions. For additional assistance, contact NLM at (301) 496-0081 or NLMJournalDonation@mail.nlm.nih.gov. NLM will pay shipping for needed volumes. For donations of pre-1871 journal volumes, contact the NLM History of Medicine Division.
Since the beginning of the online donation system in April 2009, over 10,000 gifts have been added to the NLM collection. With the help of libraries planning to discard journal volumes, NLM can build on the success achieved to date.
If you have determined that the use of social media channels is appropriate for your organization, you will quickly encounter hashtags, which are user-controlled categories prefaced with a pound sign. Hashtags were once limited to Twitter but are now used on most social media sites, including Facebook and Google+. Conversational, concise, and consistent use of up to two hashtags per social media message can result in double the amount of user engagement compared to messages without them. For more statistics specific to Twitter and user engagement, Buffer’s blog provides an excellent overview.
What are some of the ways to show that hashtags increase user engagement with your organization’s message? Look for performance indicators of reposts (the use of ‘Share’ on Facebook or retweets on Twitter), replies (comments under the message from Facebook followers, replies to the tweet from Twitter users), the number of clicks to any links included in your message (ideally to your organization’s website and resources), and hashtag usage frequency. For tips on how to track these performance indicators and additional statistics regarding hashtag creation and use, check out this helpful infographic.
For public sector and nonprofit organizations, social media can be a cost effective way to engage with users and supporters. However, social media is not without its cost, particularly in terms of staff time. So organizations have an interest in assessing the value of their social media activities.
One great resource for social media evaluation is Paine’s book, Measure What Matters. The book contains detailed guidance for evaluating social media use by different types of organizations. A great supplement to Paine’s book is The Nonprofit Social Media Decision Guide, by Idealware, which has worksheets that will help plan social media strategies and implement recommendations in Measure What Matters.
Below are the key elements of Paine’s evaluation framework:
- Begin with a solid social media plan that identifies specific goals and objectives. As with any project, you need a plan for social media that links strategies to the organizational mission and includes objectives with targets and key performance indicators. Objectives for social media in the public sector often belong in one of two categories: helping users find information they need; or building user awareness, engagement, or loyalty. (The Nonprofit Social Media Decision Guide provides a list of potential objectives on page 52.)
- Define your target audience: Organizations often have many stakeholder groups, so it’s important to identify the groups most attuned to social media. On page 54 of The Nonprofit Social Media Decision Guide, there is a worksheet for narrowing down stakeholder audiences to those most receptive to social media activities.
- Pick your metrics: Metrics such as views, followers, and measures of engagement with online content will help monitor your reach. Conversions, defined as the actions you want your social media followers to complete, might include becoming members of your organization or actively recommending your organization to colleagues or friends.
- Identify a source for benchmarks. Benchmarks provide a basis for comparison to assess progress. Organizations often use their own histories as benchmarks, comparing progress against baseline measures. You also may have access to data from a competing or peer organization that you can use for comparison.
- Pick a measurement tool: Paine’s book describes different measurement methods for evaluating social media, such as content analysis, web analytics, or surveys.
For more information, check out the resources mentioned in this blog post:
- Katie Delahaye Paine, Measure What Matters: Online Tools For Understanding Customers, Social Media, Engagement, and Key Relationships. Hoboken, NJ: John Wiley & Sons, Inc, 2011.
- Idealware. The Nonprofit Social Media Decision Guide, 2013.