Skip all navigation and go to page content
NN/LM Home About MAR | Contact MAR | Feedback |Site Map | Help | Bookmark and Share

Archive for the ‘Evaluation’ Category

Elegantly Simple Evaluation: Talking to Health Care Providers about Patient Health Literacy

Sunday, October 19th, 2014

By Yawar Ali and Cindy Olney

As the child of a physician living in South Texas, I’ve witnessed a deficiency of health literacy in patients. I volunteered in my dad’s clinic over spring break. I also participated on a medical relief trip with my father to a nonprofit charitable hospital in Pakistan. At both places, I witnessed difficulty in patient health literacy. – Yawar Ali

 In June 2014, Yawar Ali, a rising junior from the South Texas High School for Health Professions, taught physicians and physician assistants in his father’s medical clinics about patient health literacy. He also introduced them to MedlinePlus as an important tool for their patients. Yawar evaluated his project and discovered valuable insight that helped him improve the impact of his project.

Yawar conducted this health information outreach project as an internship offered through the  ¡VIVA! (Vital Information for a Virtual Age) project.  ¡VIVA! is a high school-based initiative in which students are trained to promote MedlinePlus to their classmates, teachers, families, and community members.  It is a student organization led by librarians of the South Texas Independent School District, located in the Lower Rio Grande Valley. The National Library of Medicine (NLM) funds the project.

He developed his presentation using health literacy materials available through the Medical Library Association and presented to three doctors and three PAs.  He taught them seven steps for addressing low patient health literacy and introduced them to MedlinePlus.

Yawar incorporated elegantly simple evaluation techniques into his project. Right after the presentation, he asked participants to complete a short evaluation form, asking them how likely they were to use the steps and promote MedlinePlus to patients.  They all responded positively, indicating good intentions.

Two weeks after the training, Yawar visited all of the health care providers to conduct brief semi-structured interviews. He asked if they had tried the steps and collected their feedback on the techniques. He also checked to see if they had promoted MedlinePlus to their patients. With some persistence, he was able to conduct a complete interview with each participant.

The feedback he received is of interest to anyone hoping to initiate health information outreach in partnership with primary care clinics, particularly in medically underserved areas:

  • The majority of Yawar’s participants tried teach-back, open-ended questions, and other techniques with their patients; but they were conflicted because such techniques added time to patient appointments. This interfered with their ability to stick to their busy schedules.
  • The health care providers were impressed with MedlinePlus, but they had convenient access to print materials from a database (Healthwise) that was integrated with the clinic’s Electronic Health Records (EHR) system. Furthermore, it was easier to document that they were adhering to the meaningful use requirements of the Medicare and Medicaid EHR Incentive Programs when they got patient information from Healthwise.
  • While the Healthwise database was more convenient for the providers, they recognized that the print information they were providing was limited. They believed their patients could get more comprehensive information from MedlinePlus, but the clinicians did not have a convenient way to promote the resource.

Their feedback prompted a speedy response. The project team secured MedlinePlus brochures from NLM that Yawar delivered to the clinics. The fix was relatively simple, but critical. The team may have never known about this necessary adjustment without Yawar’s elegantly simple evaluation.

Credit:  Yawar and Cindy would like to thank ¡VIVA! project team members Lucy Hansen, Sara Reibman, and Ann Vickman, for their help on this project.

The ¡VIVA! project has been funded in whole or in part with federal funds from the National Library of Medicine, National Institute of Health, under Contract No. HHSN-276-2011-0007-C with the Houston Academy of Medicine-Texas Medical Center Library.

Audit an Evaluation MOOC

Friday, October 10th, 2014

Have you always been curious about Massive Open Online Courses (MOOCs) and haven’t yet checked them out, or have started a MOOC and not completed it because you found the format confusing or didn’t have the time to complete all the assignments in it?

Evaluating Social Programs is currently being offered by the Massachusetts Institute of Technology (MIT) within one of the easier-to-navigate open access MOOC formats, EdX. The course is now closed for registration for credit, but can still be accessed for free to audit with complete access to the instructional materials, activities, tests and discussions within the MOOC. It is estimated that full participation in each week of the course will take approximately 4 hours, but of course you are welcome to skip around and access the information that is of interest to you. I am currently auditing the MOOC while taking note of new resources and information that is likely to be of interest to you from a health program evaluation perspective this month, and will write a summary blog post in November.

The class description is:

This four and a half-week course on evaluating social programs will provide a thorough understanding of randomized evaluations and pragmatic step-by-step training for conducting one’s own evaluation. Through a combination of lectures and case studies from real randomized evaluations, the course will focus on the benefits and methods of randomization, choosing an appropriate sample size, and common threats and pitfalls to the validity of the experiment. While the course is centered around the why, how and when of Randomized Evaluations, it will also impart insights on the importance of a needs assessment, measuring outcomes effectively, quality control, and monitoring methods that are useful for all kinds of evaluations.

If you are interested in participating in Evaluating Social Programs and have not previously taken an EdX MOOC, I highly recommend checking out the 15 minute self-navigated DemoX which provides a very user-friendly overview of how to make the best use of the features within EdX.

Qualitative Data Visualization

Friday, October 10th, 2014

An innovative and compelling approach to creating qualitative data visualizations with illustrations is provided by Fresh Spectrum. The process begins by taking a long narrative such as a focus group transcription, and chunking it into a few paragraphs per concept with a unique illustration for each one. One option is to use your organization’s existing images or Creative Commons-licensed images for illustrating concepts. The next step for the visualization uses the images with brief captions as an online data dashboard, where visitors can click on the captioned image of interest to them to access the more detailed narrative. One example describes how to do this within a WordPress portfolio blog template, or a simpler strategy of creating HTML anchor links to each individual section within a longer text, which then leads to the longer narrative.

The NN/LM Outreach Evaluation Resource Center (OERC) has more resources available from the Reporting and Visualizing tab of the OERC Tools and Resources for Evaluation Guide.

Qualitative Data Visualization

Friday, October 3rd, 2014

NN/LM Outreach Evaluation Resource Center New OERC Blog posting! This is to let you know that a new OERC Blog article has become available. You can find this article online here. For simplicity’s sake, we’ve posted the article below:

Have you thought that only quantitative information can be used for data visualizations, and qualitative data wasn’t an option without first coding or otherwise turning this valuable content into quantitative formats?

I learned about an innovative and compelling approach to creating qualitative data visualizations with illustrations from Fresh Spectrum . They begin the process (as shown in the illustration above) of taking a long narrative such as a focus group transcription, and chunking it into a few paragraphs per concept with a unique illustration for each one. In this case custom illustrations of people were used, but you could use your organization’s existing images or Creative Commons-licensed images for illustrating concepts. The next step for the visualization uses the images with brief captions as an online data dashboard, where visitors can click on the captioned image of interest to them to then access the more detailed narrative. The author describes how to do this within a WordPress portfolio blog template, or a simpler strategy of creating HTML anchor links to each individual section within a longer text. You can see how this works by clicking on an anchor link from the original post (http://freshspectrum.com/blogging-advice/#davidson for example) that leads to the longer narrative at http://freshspectrum.com/blogging-advice/ (a great source of advice for blogging by the way!)

Need more information about reporting and visualizing your data? We at the Outreach Evaluation Resource Center (OERC) have more resources available for you from the Reporting and Visualizing tab of our Tools and Resources for Evaluation Guide at http://guides.nnlm.gov/oerc/tools and welcome your suggestions and comments about the guide.

My NCBI – ORCID Author Data Integration with SciENcv

Friday, September 26th, 2014

SciENcv users will soon be able to create SciENcv profiles using the data stored in their ORCID records. By linking an ORCID account to an NCBI account, users will be able to create SciENcv profiles using the personal statement, education, employment, publications and research awards information stored in ORCID records.

http://www.nlm.nih.gov/pubs/techbull/so14/so14_sciencv_orcid.html

Elegantly Simple Evaluation: Documenting Outcomes of a New England Health Literacy Project

Friday, September 26th, 2014

NN/LM Outreach Evaluation Resource Center New OERC Blog posting! This is to let you know that a new OERC Blog article has become available. You can find this article online here. For simplicity’s sake, we’ve posted the article below:

For an example of an elegantly simple program evaluation that yielded great results, check out an article by Michelle Eberle and colleagues in the National Network of Libraries of Medicine New England Region, which appeared in the August 2014 edition of MLA News . The article describes the region’s Clear: Conversations project, a collaboration among five organizations in which librarians and health professionals taught health literacy skills to patients. This innovative project, originated by Health Care Missouri, featured role-plays of patients in which they practice good patient communication skills during a visit to a health care provider (played by volunteers from various health professions).

This project shows that a few relatively simple evaluation activities can clearly show the positive outcomes of a project. For example, after their role-play, participants gave high ratings to their satisfaction with the information they received during their “doctor visit.”   When completing the multi-session program, a strong majority said the program improved their comfort with employing effective communication techniques with their own health care providers. More than half of respondents completing the second questionnaire described specific actions they intended to use in future visits to health care providers. Also, the health professional role-players provided their own feedback about how their experiences would affect their own interactions with patients.

The evaluation methods used for the Clear: Conversations project were fairly simple, but well-planned. Eberle and her colleagues developed their evaluation methods in the project planning stage and consulted with the NN/LM OERC on method design. As a result, the team was able to collect information that clearly demonstrated, both to themselves and others, the value of their project.

The OERC would like to highlight more examples of evaluations that are both effective and relatively easy to implement.  If you know of other projects that we can showcase in our Elegantly Simple Evaluation series, please contact Cindy Olney at olneyc@uw.edu.

NISO September Virtual Conference: Library Data in the Cloud

Sunday, September 21st, 2014

Virtual conferences are 5-6 hour conferences held online in webinar-like formats, with occasional breaks in the schedule for participants. The longer length allows the depth of coverage of a conference coupled with the convenience of a webinar.

Date: September 24, 2014

Time: 11:00 am – 5:00 pm Eastern

Event webpage: http://www.niso.org/news/events/2014/virtual/data_in_the_cloud/

ABOUT THE VIRTUAL CONFERENCE

Cloud computing seems to be a growing trend, no matter the industry or type of information system. Library systems are no stranger to this trend; just about every major systems provider has a cloud-based solution available. While many factors for selecting a cloud system are similar to those for any information system decision, there are some special issues and challenges for storing your data in the cloud, including security, privacy, ownership, interoperability, and transferability.

In NISO’s September 24 virtual conference, Library Data in the Cloud—to be held from 11:00 am to 5:00 pm EDT—libraries that have explored the use of cloud systems will discuss their experiences, their concerns, issues encountered, and lessons learned.

TOPICS AND SPEAKERS

Keynote Speaker – Rick McMullen, PhD, Director of the Arkansas High Performance Computing Center and Research Professor of Computer Science and Computer Engineering, University of Arkansas

Integrated Library Systems Moving to the Cloud – Joseph R. Matthews, author and library consultant

Big Data Processing in the Cloud: a Hydra/Sufia Experience – Zhiwu Xie, Ph.D., Associate Professor and Technology Development Librarian, Center for Digital Research and Scholarship University Libraries, Virginia Tech

Cloud Computing in Library Instruction – Laura Fargo McKinnon, JD, MLIS, Department Head, Research & Instructional Services, University of North Texas Libraries and Kris Helge, Scholarly Communications Librarian, University of North Texas Libraries

Data Publication and Sharing with Globus – Steve Tuecke, Deputy Director, Computation Institute, University of Chicago; Co-Founder of the Globus Project

eResource Management in the Cloud – Jeffrey D. Kuskie, Electronic Resource Manager, Dr. C.C. and Mabel L. Criss Library, University of Nebraska at Omaha

Security and Data Ownership in the Cloud – Andrew K. Pace, Executive Director, Networked Library Services, OCLC; Councilor-at-large, American Library Association

Privacy in the Cloud – Speaker TBA

Conference Roundtable – Discussion with speakers and Q&A

REGISTRATION

Registration is per site (access for one computer) and closes at 4:00 pm Eastern on September 24, 2014 (the day before the virtual conference). Discounts are available for NISO members and students. All virtual conference registrants receive access to the recorded version for one year.

Can’t make it on the day of the virtual conference? All registrants receive access to the recorded version for one year. Take advantage of the Virtual Conference subscription package (www.niso.org/news/events/2014/virtual/#subscription) for all six of the 2014 Virtual Conferences and save 33%. (Previously held 2014 virtual conferences available in recorded versions.)

For more information and to register, visit the event webpage: http://www.niso.org/news/events/2014/virtual/data_in_the_cloud/

American Evaluation Association Potent Presentation Information

Sunday, September 21st, 2014

NN/LM Outreach Evaluation Resource Center New OERC Blog posting! This is to let you know that a new OERC Blog article has become available. You can find this article online here. For simplicity’s sake, we’ve posted the article below:

The American Evaluation Association (eval.org) sponsors a Potent Presentations Initiative (p2i) that has a stated purpose of helping evaluators improve their presentation skills, both within a conference setting and as part of individual practice. P2i challenges evaluators to hone in on three concepts: Their message, their design, and their delivery.

There are a wealth of handouts available as PDF files, Word documents and PowerPoint presentations available from the p2i tools website (http://p2i.eval.org/index.php/p2i-tools/) that sometimes include AEA conference specifications in addition to many great messaging, designing and delivery principles. For an example of each principle be sure to check out the Presentation Preparation Checklist (from 3 months ahead of time to afterwards to include modifications while the information is freshly in mind), How to Design a Research Poster (great infographic visualization and instructions on how to make your data ‘pop’), and the Delivery Glue Handout (did you know as a general rule it takes 16 times the length of your talk to make presentation slides and a script?).

Value of the Library to Society

Monday, September 8th, 2014

http://theweek.com/article/index/265775/what-the-death-of-the-library-means-for-the-future-of-books

Is There a Shortage of Qualified Librarians for the Future’s Needs?

Monday, September 8th, 2014

“Labor shortages also will hit shrinking and slow-growing professions such as plant operators, librarians and sea captains because there simply aren’t enough young workers to fill the remaining positions after current workers retire.”

Well, I think out there in I-Need-a-Job Land there is a disconnect with many people thinking information and communication technologies (and, sadly, a general dumbing down of the US in math, science, and reading) is making librarianship obsolete. There should be more than rays of hope in this.

http://online.wsj.com/articles/help-wanted-librarians-sea-captains-1409631004