For an example of an elegantly simple program evaluation that yielded great results, check out an article by Michelle Eberle and colleagues in the National Network of Libraries of Medicine New England Region, which appeared in the August 2014 edition of MLA News . The article describes the region’s Clear: Conversations project, a collaboration among five organizations in which librarians and health professionals taught health literacy skills to patients. This innovative project, originated by Health Care Missouri, featured role-plays of patients in which they practice good patient communication skills during a visit to a health care provider (played by volunteers from various health professions).
This project shows that a few relatively simple evaluation activities can clearly show the positive outcomes of a project. For example, after their role-play, participants gave high ratings to their satisfaction with the information they received during their “doctor visit.” When completing the multi-session program, a strong majority said the program improved their comfort with employing effective communication techniques with their own health care providers. More than half of respondents completing the second questionnaire described specific actions they intended to use in future visits to health care providers. Also, the health professional role-players provided their own feedback about how their experiences would affect their own interactions with patients.
The evaluation methods used for the Clear: Conversations project were fairly simple, but well-planned. Eberle and her colleagues developed their evaluation methods in the project planning stage and consulted with the NN/LM OERC on method design. As a result, the team was able to collect information that clearly demonstrated, both to themselves and others, the value of their project.
The OERC would like to highlight more examples of evaluations that are both effective and relatively easy to implement. If you know of other projects that we can showcase in our Elegantly Simple Evaluation series, please contact Cindy Olney at email@example.com.
This week the Outreach Evaluation Resource Center (OERC) enjoyed an intensive time of data information and learning with over 5,500 others during the Tableau Conference in Seattle.
Christian Chabot, Tableau’s CEO, provided the first part of a compelling keynote address of the conference that may seem surprising for a meeting about data: a focus on creativity. Chabot focused on innovation, noting the breakthroughs for technology to first empower user creativity with Adobe’s PostScript (the beginning of desktop publishing), and computer aided design (CAD) instead of relying on subject matter experts to produce the projects in a physical format from start to finish with limited options for revision. He concluded that the world from an artist’s perspective and data analysis perspective are not opposites but that both seek to reveal truth and impart meaning as a part of their work.
For the rest of the keynote Dr. Chris Stolte, Tableau’s Chief Development Officer, showed how in the past software tools required high levels of expertise to use. Dr. Stolte noted that they are trying to incorporate a greater sense of working in a user-friendly way with the data, not the software, to give users creative flow, feedback and flexibility and live demonstrated many new features Tableau has on the horizon.
You can see these demonstrations and learn more about these by watching the keynote recording available at http://tcc14.tableauconference.com/keynote or the overview Tableau wrote at http://www.tableausoftware.com/about/blog/2014/9/keynote-32970.
The American Evaluation Association (eval.org) sponsors a Potent Presentations Initiative (p2i) that has a stated purpose of helping evaluators improve their presentation skills, both within a conference setting and as part of individual practice. P2i challenges evaluators to hone in on three concepts: Their message, their design, and their delivery.
There are a wealth of handouts available as PDF files,Word documents and Powerpoint presentations available from the p2i tools website (http://p2i.eval.org/index.php/p2i-tools/) that sometimes include AEA conference specifications in addition to many great messaging, designing and delivery principles. For an example of each principle be sure to check out the Presentation Preparation Checklist (from 3 months ahead of time to afterwards to include modifications while the information is freshly in mind), How to Design a Research Poster (great infographic visualization and instructions on how to make your data ‘pop’), and the Delivery Glue Handout (did you know as a general rule it takes 16 times the length of your talk to make presentation slides and a script?).
The OERC debuted a brand new webinar for the NN/LM Greater Midwest Region’s monthly Lake Effects webinar series on August 21. The new webinar, Evaluation 2.0: Trends, New Ideas, Cool Tools, presents emerging trends in evaluation practice that emphasize stakeholder interaction and social engagement. It also covers popular tools and methods that allow you to draw others into the evaluation process and raise the visibility of your program or services. The NN/LM GMR makes recordings of Lake Effects presentations publicly available, so click here to listened to the Eval 2.0 webinar.
If you are interested in attending a live presentation of this webinar, please contact the OERC or your National Network of Libraries of Medicine regional office. Descriptions of other OERC webinars that can be offered upon request are listed here.
At the Library Assessment Conference in August, one panel featured assessment librarians presenting data dashboards they created using Tableau software. Our colleagues often ask OERC staff for good examples of library dashboards, so I am excited that the PowerPoint slides of Tableau Unleashed: Visualizing Library Data are publicly available . This document includes views of dashboards from University of British Columbia Library (by presenter Jeremy Buhler), UMass Amherst Libraries (by Rachel Lewellen) and Ohio State Libraries (by Sarah Murphy). All of the presenters used Tableau software to produce their dashboards.
Tableau may be the most popular software for creating dashboards right now and the company offers a free version that has a great deal of functionality. In fact, at least one presenter (Sarah Murphy) included dashboards she created using Tableau Public. However, users must be cautioned that any data entered into Tableau Public become public information. That means anyone can see and download your raw data. So, if you use it, be sure all identifying information about individuals is stripped from your files and that you are comfortable with other people downloading your raw data.
The presenters included some tips for dashboard design, which you will find in their slides. If you want more comprehensive guidance, check out A Guide to Creating Dashboards People Love to Use by Juice Analytics. The guide is free and downloadable.
The Engaged Librarian: Crafting an Effective Assessment Plan to Determine the Impact of a Key Strategic Library Initiative by Sarah Murphy at The Ohio State University (OSU) was presented during the Library Assessment Conference and provided an overview to the use of a logic model as part of library strategic planning. Ms. Murphy’s presentation slides are available by clicking here.
Their project incorporated the theory of change methodology with logic models and used the Kellogg Foundation Logic Model as a template. They storyboarded data within a data dashboard that was both aligned with and broken down by the applicable OSU strategic vision goals. Ms. Murphy reported that the benefits of using a logic model approach included having a flexible but structured way to do library assessment planning, having a collaborative and inclusive approach, creating a project focus, being able to assess linear and iterative programs and services, and the ability to communicate program accomplishments in interesting ways. During the question and answer session they noted they are also Tableau fans (we will write about Tableau for our next post) and like to create data structures in their dashboard to avoid information silos.
If you’d like to learn more about logic models and data dashboards, be sure to check out our freely available Outreach Evaluation Resource Center (OERC) Evaluation Guides, especially Planning Outcomes-Based Outreach Projects. We also offer Data Dashboards: Monitoring Progress toward Program Outcomes as one of our webinars and a recording of Data Dashboards is available by clicking here.
We here at the Outreach Evaluation Resource Center (OERC) and 600 others enjoyed a fabulous week of learning and engagement during the 2014 Library Assessment Conference at the University of Washington in Seattle. We will be covering some of the great assessment resources and information shared here in our blog (http://nnlm.gov/evaluation/blog/) that are of interest to National Network of Library of Medicine (NN/LM) network members.
In the meantime you may be interested in the freely available copies of most of the conference presentations that are posted on the main program schedule website at http://libraryassessment.org/schedule/index.shtml along with photos of the poster session, which will soon be available at http://libraryassessment.org/schedule/2014-posters.shtml.
The OERC will be attending the Library Assessment Conference (LAC) at University of Washington this week, where we will be learning about new trends in library assessment, evaluation, and improvement. Stayed tuned to our blog and we will pass along what we learn. The LAC is sponsored by the Association of Research Libraries.
In the meantime, I want to leave you with a fun data visualization site to explore while we’re gone. David McCandless creates wonderful infographics for his site “Information is Beautiful.” Many are health-related. All are gorgeous. You can find the list of his data visualizations here.
Planning focus groups? You might want to check out the Libraries Transforming Communities Community Conversation Workbook by the American Library Association (ALA).
This workbook is a resource developed for the ALA’s Libraries Transforming Communities (LTC) initiative, which provides librarians with training and resources to enhance their roles as community leaders and change-agents. The initiative’s goal is to help librarians promote the visibility and value of their libraries within their communities. Public discussions are promoted as key community engagement strategies.
To that end, ALA has developed the Libraries Transforming Communities Community Conversation Workbook. This workbook provides invaluable guidance to anyone who wants to conduct discussion groups for community assessment purposes. It provides practical advice on every aspect of convening group discussions, including tips on participant recruitment, a list of discussion questions, facilitator guidelines, note-taking tools, and templates for organizing key findings.
Demonstrating value is of considerable interest to many libraries and organizations these days. Such organizations may be interested in exploring other articles and resources related to the LTC initiative. You can find more information about the initiative at the LTC web page. You can also see how libraries are implementing LTC activities at the initiative’s digital portal:
Do you want to learn more about outcomes-based planning and evaluation (OBPE) for your outreach project but there’s no money in the training budget to do so?
Shaping Outcomes: Making a Difference in Libraries and Museums (shapingoutcomes.org) is available as a free online course that learners can start anytime and work on at their own self-navigated pace. While there are library and museum-specific examples provided in the course the concepts of learning more about target audience needs, how to clarify desired results, developing logic models and evaluating outcomes are applicable for most other organizations’ outreach projects as well.
Modules of the class are broken into five sections (Overview, Plan, Build, Evaluate, Report) with a helpful Glossary to learn OBPE terminology and a Logic Model template. Shaping Outcomes was developed by the Institute of Museum and Library Services (IMLS) and Indiana University/Purdue University Indianapolis (IUPUI) and previously was available as an instructor-led class.
More information specific to developing logic models in health information outreach programs is available from Booklet Two: Planning Outcomes-Based Outreach Projects, part of our resources on our Outreach Evaluation Resource Center (OERC) Evaluation Guides page at http://nnlm.gov/evaluation/guides.html.