Archive for the ‘News’ Category
Coming soon to a computer near you! Chris Lysy of FreshSpectrum is offering a free seven-part data visualization workshop. Chris has provided data viz training for the American Evaluation Association. (His followers also love his cartoon-illustrated evaluation blog. ) He calls himself the Rachel Ray of data visualization, which makes his course description a nice feature for the OERC’s Thanksgiving blog post.
The workshop date is still TBA, but you can join his mailing list now to get full details when they are released.
Also, Thanksgiving activities often include movie-viewing. So here are some fun data visualizations of famous movie quotes by Flowingdata to help you through the last afternoon before the holiday weekend.
Looking for an ‘at a glance’ single page to determine which type of data visualization chart is helpful in order to clearly communicate your results?
This PDF flowchart at http://betterevaluation.org/plan/describe/visualise_data is a very handy reference! The flowchart guides you towards considering the appropriate data visualization chart options after your initial response to the question of “What would you like to show?” answers of comparison, distribution, composition, or relationship. There are brief descriptions of the charts at the Better Evaluation data visualization page that you can click through to get additional information such as a deviation bar graph that includes synonyms, a base definition, examples of how the chart is used, advice about their use, and links to resources for creating them.
Nothing beats qualitative (non-numerical) data collection methods for getting a high volume of rich, interesting information from project participants and stakeholders. The downside is that these methods are resource intensive, so you usually are limited to involving a relatively small number of participants in conversation.
But what if you want to collect a lot of qualitative responses from a lot of people?
If you do, check out the Liberating Structures website. It provides step-by-step instructions for activities to engage large groups in conversations for planning and evaluation. The website offers a menu of 33 activities with extensive planning details, plus ideas for combining activities into an almost unlimited number of group discussion formats.
I participated in a Liberating Structures activity in Denver last month when I attended the Quint*Essential Conference, hosted by five Medical Library Association chapters. Staff from National Network of Libraries of Medicine (NN/LM) regional offices invited all conference attendees to generate and evaluate ideas for future network initiatives. It was a high-energy activity that engaged more than 100 people in providing bold ideas for future activities.
The beauty of Liberating Structures activities is that the guidelines include how to document conversations so meeting facilitators will end their exercises with actual data. In some cases, the data can be quickly analyzed. NN/LM facilitators were able to compile and report results from the Quint discussion in the exhibit hall later that day.
I want to thank Claire Hamasu, the Associate Director of the NN/LM MidContinental Region, for pointing me to the Liberating Structures web site and including me in the Quint Conference activity. I personally look forward to trying more of these activities and hope other readers are inspired to do so as well.
For the past couple of months, the OERC has engaged in an Appreciative Inquiry (AI) interview project to get feedback and advice from users on to our services. Appreciative Inquiry was developed in the 1980s by David Cooperrider and Suresh Srivastva as an approach to bring “collaborative and strength-based change” to organizations. The methods are designed to collect information emphasizing positive aspects of an organization and vision for a better future. Probably the best known AI tool is the interview, which covers three basic areas:
- A peak experience of the interviewee.
- Why the interviewee found that experience so valuable.
- What the interviewee wished could happen to bring about more exceptional experiences.
(You can find the OERC’s adaption of these basic questions here.)
When people are first introduced to AI evaluation processes, they skeptically ask if this approach doesn’t lead to positively biased data. I would say no, because we are asking for descriptive rather than evaluative comments. I call the interviews “constructive conversations without criticism.” You come away from the experience thinking “what could be?” rather than “what’s wrong?” The feedback was painless to me, because our users made recommendations in wishful, rather than judgmental, terms.
I also think AI is a superior way to get frank advice from users if they generally like your organization. When asked for feedback, particularly in interpersonal situations, interviewees may not want to offend the organization’s staff or, worse, cause negative repercussions. When you ask people to talk about dreams and wishes, their imaginations are engaged and fear of being critical falls away. They are free to give you great ideas for moving forward.
If your organization is about to embark on strategic planning of any kind, I highly recommend the AI approach. You can get more information about AI methods at the Appreciative Inquiry Commons or the Center for Appreciative Inquiry websites. For an excellent book on applying AI to evaluation practice, check out Reframing Evaluation through Appreciative Inquiry by Preskill and Catsambas (Sage, 2006).
Note: The OERC will post results of its AI project in a future blog post, when we have completed our analysis.
Recently the AEA365 Evaluation Tip a Day resource the Outreach Evaluation Resource Center (OERC) previously blogged about featured a review and several hot tips for the use of Padlet, a freely available web-based bulletin board system. Their hot tips included use of Padlet as an anonymous brainstorming activity in response to a question or idea, and as a backchannel for students or conference attendees to share resources and raise questions for future discussion.
I took a closer look at Padlet’s bulletin board configuration settings and found them intuitive and easy to use with various backgrounds and freeform, tabular or grid note arrangement display on the bulletin board. Free Padlet accounts can be created by either signing up for one or by linking to an existing Google or Facebook account. Privacy is a key concern that Padlet delivers many options for that are clearly explained including Private (only you and others you invite to participate via email), requiring the use of a password to access the Padlet, and Public to view, write or moderate. A new update feature includes a variety of ways to share Padlet data, ranging from clicking the icon for 6 different social media channels to downloading data as a PDF or Excel/CSV file for analysis.
Please check out a Padlet about the OERC Evaluation Series and leave your input! Posts will be moderated on the Padlet before they display publicly.
Have you thought that only quantitative information can be used for data visualizations, and qualitative data wasn’t an option without first coding or otherwise turning this valuable content into quantitative formats?
I learned about an innovative and compelling approach to creating qualitative data visualizations with illustrations from Fresh Spectrum . They begin the process (as shown in the illustration above) of taking a long narrative such as a focus group transcription, and chunking it into a few paragraphs per concept with a unique illustration for each one. In this case custom illustrations of people were used, but you could use your organization’s existing images or Creative Commons-licensed images for illustrating concepts. The next step for the visualization uses the images with brief captions as an online data dashboard, where visitors can click on the captioned image of interest to them to then access the more detailed narrative. The author describes how to do this within a WordPress portfolio blog template, or a simpler strategy of creating HTML anchor links to each individual section within a longer text. You can see how this works by clicking on an anchor link from the original post (http://freshspectrum.com/blogging-advice/#davidson for example) that leads to the longer narrative at http://freshspectrum.com/blogging-advice/ (a great source of advice for blogging by the way!)
Need more information about reporting and visualizing your data? We at the Outreach Evaluation Resource Center (OERC) have more resources available for you from the Reporting and Visualizing tab of our Tools and Resources for Evaluation Guide at http://guides.nnlm.gov/oerc/tools and welcome your suggestions and comments about the guide.
For an example of an elegantly simple program evaluation that yielded great results, check out an article by Michelle Eberle and colleagues in the National Network of Libraries of Medicine New England Region, which appeared in the August 2014 edition of MLA News . The article describes the region’s Clear: Conversations project, a collaboration among five organizations in which librarians and health professionals taught health literacy skills to patients. This innovative project, originated by Health Care Missouri, featured role-plays of patients in which they practice good patient communication skills during a visit to a health care provider (played by volunteers from various health professions).
This project shows that a few relatively simple evaluation activities can clearly show the positive outcomes of a project. For example, after their role-play, participants gave high ratings to their satisfaction with the information they received during their “doctor visit.” When completing the multi-session program, a strong majority said the program improved their comfort with employing effective communication techniques with their own health care providers. More than half of respondents completing the second questionnaire described specific actions they intended to use in future visits to health care providers. Also, the health professional role-players provided their own feedback about how their experiences would affect their own interactions with patients.
The evaluation methods used for the Clear: Conversations project were fairly simple, but well-planned. Eberle and her colleagues developed their evaluation methods in the project planning stage and consulted with the NN/LM OERC on method design. As a result, the team was able to collect information that clearly demonstrated, both to themselves and others, the value of their project.
The OERC would like to highlight more examples of evaluations that are both effective and relatively easy to implement. If you know of other projects that we can showcase in our Elegantly Simple Evaluation series, please contact Cindy Olney at firstname.lastname@example.org.
This week the Outreach Evaluation Resource Center (OERC) enjoyed an intensive time of data information and learning with over 5,500 others during the Tableau Conference in Seattle.
Christian Chabot, Tableau’s CEO, provided the first part of a compelling keynote address of the conference that may seem surprising for a meeting about data: a focus on creativity. Chabot focused on innovation, noting the breakthroughs for technology to first empower user creativity with Adobe’s PostScript (the beginning of desktop publishing), and computer aided design (CAD) instead of relying on subject matter experts to produce the projects in a physical format from start to finish with limited options for revision. He concluded that the world from an artist’s perspective and data analysis perspective are not opposites but that both seek to reveal truth and impart meaning as a part of their work.
For the rest of the keynote Dr. Chris Stolte, Tableau’s Chief Development Officer, showed how in the past software tools required high levels of expertise to use. Dr. Stolte noted that they are trying to incorporate a greater sense of working in a user-friendly way with the data, not the software, to give users creative flow, feedback and flexibility and live demonstrated many new features Tableau has on the horizon.
You can see these demonstrations and learn more about these by watching the keynote recording or the overview Tableau wrote at http://www.tableausoftware.com/about/blog/2014/9/keynote-32970.
The OERC debuted a brand new webinar for the NN/LM Greater Midwest Region’s monthly Lake Effects webinar series on August 21. The new webinar, Evaluation 2.0: Trends, New Ideas, Cool Tools, presents emerging trends in evaluation practice that emphasize stakeholder interaction and social engagement. It also covers popular tools and methods that allow you to draw others into the evaluation process and raise the visibility of your program or services. The NN/LM GMR makes recordings of Lake Effects presentations publicly available, so click here to listened to the Eval 2.0 webinar.
If you are interested in attending a live presentation of this webinar, please contact the OERC or your National Network of Libraries of Medicine regional office. Descriptions of other OERC webinars that can be offered upon request are listed here.
At the Library Assessment Conference in August, one panel featured assessment librarians presenting data dashboards they created using Tableau software. Our colleagues often ask OERC staff for good examples of library dashboards, so I am excited that the PowerPoint slides of Tableau Unleashed: Visualizing Library Data are publicly available . This document includes views of dashboards from University of British Columbia Library (by presenter Jeremy Buhler), UMass Amherst Libraries (by Rachel Lewellen) and Ohio State Libraries (by Sarah Murphy). All of the presenters used Tableau software to produce their dashboards.
Tableau may be the most popular software for creating dashboards right now and the company offers a free version that has a great deal of functionality. In fact, at least one presenter (Sarah Murphy) included dashboards she created using Tableau Public. However, users must be cautioned that any data entered into Tableau Public become public information. That means anyone can see and download your raw data. So, if you use it, be sure all identifying information about individuals is stripped from your files and that you are comfortable with other people downloading your raw data.
The presenters included some tips for dashboard design, which you will find in their slides. If you want more comprehensive guidance, check out A Guide to Creating Dashboards People Love to Use by Juice Analytics. The guide is free and downloadable.