Skip all navigation and go to page content

OERC Blog

A Weblog from the NN/LM Outreach Evaluation Resource Center

Archive for the ‘News’ Category

Updated Community Health Status Indicators (CHSI)

Friday, March 27th, 2015

Peer county mapsThe OERC is excited to get the word out about the Center for Disease Control and Prevention (CDC)’s newly updated and redesigned Community Health Status Indicators (CHSI).  The new CHSI 2015 represents the collaboration of public health partners in the public, non-profit and research communities, including the National Library of Medicine.

The OERC recommends the CHSI as a possible resource in the data gathering portion of planning outreach projects or needs assessments. CHSI 2015 is an interactive online tool that produces health profiles for all 3,143 counties in the United States. Each profile includes key indicators of health outcomes that describe the population health status of a county. What makes CHSI 2015 an important tool is that it includes comparisons to “peer counties” – groups of counties that are similar to each other based on 19 variables, including population size, percent high school graduates and household income.

CHSI Summary Comparison ReportFor each county, CHSI 2015 provides a Summary Comparison Report.  Using Karen Vargas’ childhood home of Union County, PA as an example, this report (right) shows that in the case of the overall cancer death rate, Union County does better than most of its peer counties.  But in the case of stroke death rate, they do worse.

Distribution Display – bar chartsBy selecting a specific indicator, such as coronary heart disease death rate, the interactive CHSI 2015 will produce a bar chart showing Union County in comparison to its peer counties, as well as the US median and the Healthy People 2020 target (left).

More detailed, downloadable data for each peer county can also be found (below), as well as a web page detailing the sources of the data for each indicator. CHSI 2015 provides a helpful How to use CHSI web page that explains each feature and provides helpful hints.

Distribution data in a downloadable format

CHSI 2015 is designed to complement other available sources of community health indicators including the Robert Wood Johnson Foundation’s County Health Rankings and Roadmaps.

Five reasons to attend the AEA Summer Institute 2015

Friday, March 20th, 2015

Registration is now open for the American Evaluation Association’s annual Summer Evaluation Institute.  The Institute, held in Atlanta, runs for 2.5 days and features 26 half-day training sessions.   Here are five reasons I make a point of attending this Institute every year.

  • Great instructors. The training is offered by some of the most experienced evaluators in the field.
  • A continuing education bargain. Training costs about $80-90 per half-day session, less for students.
  • CDC presence. Historically, AEA co-sponsored this annual event with Atlanta-based Centers for Disease Control and Prevention.  While the CDC no longer co-sponsors the Institute, you will meet lots of CDC staff members and consultants.
  • Networking opportunities. Between lunch and breaks, you get eight opportunities to chat with your colleagues.
  • Great location. The Institute is held at the Crown Plaza Atlanta Perimeter at Ravinia, located in a park-like setting on Atlanta’s perimeter near shopping and restaurants. The hotel is on the MARTA (mass transit) red line, so you can get from the airport to the hotel without facing Atlanta’s legendary traffic. Because I live near Atlanta, I haven’t stayed in the hotel; but I’ve never heard any complaints.

Full-day pre-Institute workshops are held, for an additional charge, on the Sunday before the Institute. You can attend pre-conference sessions without registering for the Institute itself.  For example, beginners might want to take “Introduction to Evaluation” taught by Tom Chapel, the Chief Evaluation Officer at the CDC. Chapel organizes the workshop around the CDC’s six-step framework for program evaluation.

The AEA Institute 2015 runs June 1-3, with pre-session workshops conducted on May 31. The cost for the Institute is $395 for members and $480 for nonmembers, with a special student rate of $250. The price covers five training sessions (your choice among the 26 offerings), snacks, and lunch.  Pre-Institute workshops are an additional $150 (all participants).

AEA Summer Institute Home Page

What Shapes Health? A Story about Data Visualization

Friday, March 13th, 2015

Medicaid birth map 2006-08

It was an amazing a-ha moment. We kind of blinked at each other, and then simultaneously said ‘We got to do something.’ – Dr. Nancy Hardt, University of Florida

This week on National Public Radio’s (NPR) All Things Considered was a story of what happened when Dr. Nancy Hardt, an OB-GYN, used data from Medicaid birth records to see where children were born into poverty in Gainesville, FL to try and identify ways to intervene and prevent poor childhood health outcomes. She was surprised to see a 1 square mile high-density ‘hot spot’ of births in dark blue appear in her map above. Dr. Hardt was encouraged to share her map with Sheriff Sadie Darnell, who pulled out a map of her own of Gainesville.

Sheriff Darnell’s map showed an exact overlay with the ‘hot spot’ on Dr. Hardt’s map of the highest crime rates in the city. By visiting the area they identified many things in the community that were barriers to good health including hunger, substandard housing, and a lack of medical care facilities – the closest location for uninsured patients was a 2 hour bus ride each way to the county health department. You’ll want to check out the rest of A Sheriff and A Doctor Team Up to Map Childhood Trauma to learn more about a mobile health clinic, what data from additional maps showed, and other steps they have taken since to help improve health outcomes for the community.

This story is the latest from the NPR series What Shapes Health, which was inspired in response to a recent Robert Wood Johnson Foundation poll about what beliefs and concerns Americans have regarding health. You can read an overview and download the full report of their results at http://www.rwjf.org/en/library/research/2015/01/what-shapes-health.html.

OERC’s Tools and Resources for Evaluation Guide

Friday, March 6th, 2015

Data Dashboard Example

Have you ever found yourself trying to do an evaluation activity, but needing that one helpful tool? Or perhaps you need a step-by-step guide on how to do a community assessment, or are looking for ways to build evaluation into a project that you are planning?

The OERC has an online guide called Tools and Resources for Evaluation that you and your library can use to evaluate your programs. Here are some of the types of tools and resources described in the Guide.

Community Oriented Outreach

  • Tips on successful collaborations and tools for improving collaboration with community networks
  • Toolkits for practical participatory evaluation and processes for conducting outcome-based evaluations

Evaluation Planning

  • Step-by-step guides on incorporating evaluation planning into your outreach projects
  • Instructions on using logic models for program planning

Data Collection and Analysis

  • Tips for questionnaire development
  • Resources for statistical methods of data analysis
  • Guides for analyzing qualitative and quantitative data

Reporting and Visualizing

  • Help with creating popular data dashboards
  • Descriptions of data visualization methods
  • Tools and TEDtalks about how you will present your data

Qualitative Evaluation Week

Friday, February 27th, 2015

The American Evaluation Association (AEA) just concluded a week-long blog theme about qualitative evaluation, which we’ve summarized below for your reference and to consider as part of your own assessment efforts:

  1. The Role of Context – the authors of this entry previously shared five high quality elements of qualitative evaluation,  and this entry referenced them while emphasizing the need for evaluators to understand what role setting, relationships, and other context factors play in data as well.
  2. Purposeful Sampling – a great explanation on why to avoid convenience sampling (interviewing people because they happen to be around) and using caution with your qualitative evaluation terminology to consider not using the word ‘sampling’ due to people’s association of it with random probability.
  3. Interviewing People who are Challenging – establishing rapport leads to good qualitative data, but what does an interviewer do if there seems to be conflict with the interviewee? Details about how to manage your own feelings and approach with a curious mindset are very helpful!
  4. Asking Stupid Questions – this example from a bilingual HIV/AIDS training is especially insightful about the importance of clarifying sexual terms, putting aside concerns the evaluator may have about looking ‘stupid,’ and outcomes that led to deeper engagement and discussion from the group.
  5. Practical Qualitative Analysis – many helpful tips and lessons shared, including the reminder of being sure to group our participants’ responses that answer the same question together even if these replies come from different parts of the survey or interview.
  6. Providing Descriptions – sometimes there are concerns expressed that evaluation is ‘only looking at the negative,’ and by including full details about your qualitative inquiry collection and analysis as an additional resource or appendix you can help explain the steps of the process that otherwise may not be evident.

Need more information about qualitative and other types of evaluation? The Outreach Evaluation Resource Center (OERC) has resources available including our Tools and Resources for Evaluation guide and our freely available Planning and Evaluating Health Information Outreach booklet series.

Assessing a Military Community

Friday, February 20th, 2015
Returning soldier kisses daughter after return from Iraq.
Photo by The U.S. Army

http://publiclibrariesonline.org/2015/01/library-services-for-the-new-normal-of-miltary-families/

This week the OERC would like to highlight a community assessment that led to a public library’s outreach to the military community of Cumberland County, North Carolina. This project is described in “Library Services for the ‘New Normal’ of Military Families” by Jennifer Taft and Cynthia Olney, which appears in the November/December issue of Public Libraries. This article demonstrates how assessments can be used to involve community members in the process, leading to their commitment to the success of the outreach project.

Cumberland County is the home of Fort Bragg, one of the nation’s largest military bases. The military community there includes service members, their families, and the people and organizations that work to support the installation. The Cumberland County Public Library and Information Center was looking for ways to better serve this population. Getting an LSTA grant allowed the library to conduct a thorough assessment of the military community and the organizations that serve them.

The assessment process involved the community every step of the way:

  1. Library staff participated in the Fayetteville Community Blueprint Network, made up of organizations that provide support for service members, veterans and their families.
  2. The Library hosted a community forum on post-traumatic stress, which was met with enthusiasm.
  3. The assessment team used key informant interviews and focus groups to collect data.
  4. They also met with additional groups of military parents.
  5. Results were validated by presenting them to Living in the New Normal Committee, a group comprised of representatives from community organizations that work with military families.
  6. The Library created marketing and program strategies based on these results.
  7. These strategies were validated by presenting them to an advisory group of representative from organizations in Cumberland County or Fort Bragg.

The community assessment described in the article helped the library staff to understand the military community in greater depth. For example, the project team originally believed that post-traumatic stress would be a major topic of interest. They learned, instead, that the military deployment cycle is the single biggest disruptor to family. This cycle, from preparing for the service member’s departure, to adjustment to a single-parent situation, to then reintegrating the returning service member back into the family, is increasingly stressful the more often it occurs. This finding had significant impact on the library’s planning going forward. Other key findings are covered in the article, along with the library’s plan to respond to the community’s needs.

One of the clearest manifestations of the success of the assessment was the willingness of the community to support the library programs. For example, Community Blueprint Network organizational members participated in a library birthday celebration for the Army. Local military museums agreed to lend museum pieces to the library for display. Most significantly, the library’s requests to participate in on-post activities, which had not been approved before, were now met with enthusiasm by post personnel who had participated in the assessment process.

The OERC Welcomes Karen Vargas

Friday, February 13th, 2015

Karen Vargas

The OERC welcomes evaluation specialist Karen Vargas, MSLS, who joined the staff on February 2.  Karen will create and present training sessions on various evaluation topics, contribute regularly to the OERC blog and LibGuide, and provide one-to-one evaluation assistance to the National Network of Libraries of Medicine.

Although Karen is new to the OERC staff, she is well known within NN/LM circles, particularly in the South Central Region. She joined the SCR Regional Medical Library in 2003 as the consumer health outreach coordinator, and later became the region’s outreach and evaluation coordinator.  Karen worked on a number of evaluation projects during her time at the SCR. Prior to joining the National Network, she worked at the Houston Public Library.

Karen will be telecommuting from Houston, Texas. We look forward to her contributions to the OERC.  She agreed to participate in an interview with Cindy Olney so we could introduce her to our blog readers.

What made you want to become part of NN/LM?

I liked all the opportunities to provide training offered through NN/LM. You get to see how this job impacts people’s lives.

 What was your favorite evaluation project?

I really enjoyed doing the evaluation of the resource library outreach subcontract program at NN/LM SCR. We involved the resource library directors and outreach contacts in identifying the outcomes that they were interested in. We started with friendly general questions that developed into a process that they were willing to use. We managed to develop specific methods that led to good data. I liked helping people identify what was important to them and figuring out how to track their progress. It makes it more likely they will actually participate in the evaluation. Some of the libraries are still using the forms that we developed.

What do you find challenging about doing that project?

Multi-site evaluation was a challenge.  We were working with libraries in five states.  It was important that everyone follow directions, but we weren’t always successful in getting that point across. Some people using our forms were not the ones we originally trained and the ones we trained didn’t train their co-workers.  If I were to do it over again, we would have written out the directions more explicitly and had special training sessions for everyone who was collecting data for the project. We could have recorded the training, too.

What made you want to work with the OERC?

We’re helping organizations see the value of their programs. At SCR, we helped network members working on projects to focus on what is working and not waste time on parts that aren’t working. We were helping them feel good about what they were doing.  It’s important, for advocacy, to find out what stakeholders want. But it’s also good to figure out what you want. That’s what makes evaluation exciting. People figuring out what makes them satisfied, what they would see as success, and then figuring out how to measure it.

What I’ve like best about working with NN/LM the last 11 years, is that, when we took people through the planning steps of writing an award proposal, they often did the project even if we didn’t fund it. They get so involved in planning that they find some way to do the project. I liked seeing them get excited.

What type of evaluation skills are you most interested in developing?

I would like to learn more about interviewing.

What else would you like readers to know about you?

 I have a little girl named Sophia. She’s two years old and plays harmonica, ukulele, recorder, and can even get a sound on a trombone!

 

52 Weeks of Better Evaluation

Friday, January 30th, 2015

BetterEvaluation.org is an international collaboration that encourages sharing of evaluation methods, approaches and processes for improvement. BetterEvaluation offers yearly blog themes for their staff and guest writers to focus on, and have wrapped up the highlights of their ’52 Weeks of BetterEvaluation’ 2014 theme in a post at http://betterevaluation.org/node/4682 For 2015 they are featuring ’12 Months of BetterEvaluation’ with multiple posts during a month, starting with impact evaluation in January.

A ‘top 5′ selection from the ‘52 Weeks of BetterEvaluation‘ post that is likely to be of interest to National Network of Libraries of Medicine (NN/LM) members includes

  1. Top ten developments in qualitative evaluation over the past decade (link to part 1, part 2)
  2. Fitting reporting methods to evaluation findings and audiences (link)
  3. Infographics, including step by step instructions in piktochart (link)
  4. Innovation in evaluation (link)
  5. Presenting data effectively (link)

Focused Outreach Vermont (from NN/LM NER)

Friday, January 16th, 2015

If you are planning or currently conducting an outreach project, you might want to take a look at the Focused Outreach Vermont article in the National Network of Libraries of Medicine, New England Region (NN/LM NER) newsletter (posted January 13, 2015). NN/LM NER’s Focused Outreach Project uses carefully planned outreach activities and strong community-based collaboration to connect underserved communities with NLM resources and services. The Ner’eastah article, which is an abstract of a full report, highlights outreach results through a succinct description of evaluation findings.

I particularly applaud NN/LM NER’s reporting method. They provide a quick overview, featuring the results of their efforts, with easy access to full details for those who want it. The full report describes the project’s community assessment process and findings. You also get a more thorough description of documented outcomes, laid out in a highly readable format. A nice added feature is the infographic in the beginning of the report.

This is a great example of how to use evaluation to publicize and advocate for successful programs!

nnlm ner

We would like to report more projects that demonstrate effective use of evaluation methods. If you have an example to share, send it to Cindy Olney at olneyc@uw.edu.

Freebie Friday: Mobile Data Solutions Course

Friday, January 9th, 2015

Mobile Course Screenshot

Are you curious about the use of smart phones, tablets, or other mobile data resources to collect data for your assessment project, but are seeking more information on how to determine if this is the right approach for your project or program and how to process the data you collect using this method?

Check out http://techchange.org/media/mobile-data-solutions/, which was created as part of the Mobile Solutions Technical Assistance and Research (mSTAR) project, with expertise provided by U.S. Agency for International Development’s (USAID) Digital Development Lab and designed by TechChange.

The primary goal of this freely available and accessible online course (free registration is required to access it) is to learn more about mobile tools, processes, and strategies for data collection in order to use mobile devices (referred to as mobile data solutions) to their full potential in doing so. The course will take about 2 hours to complete and can be done at your own pace over time. Your progress in the course is saved so you’ll be taken to the point where you stopped to continue learning the next time you access it.

The learning objectives of the course are

  • Describe examples of mobile data solutions from collection through visualization
  • Articulate the benefit of using these solutions
  • Analyze the challenges and limitations associated with mobile data solutions
  • Assess whether or not particular mobile data solutions are appropriate for a project, program or problem
  • Outline how to design a project or activity to include mobile data solutions
  • Explain the steps involved in implementing mobile data solutions
  • Summarize how to analyze, visualize, and share mobile data

 

Last updated on Saturday, 23 November, 2013

Funded by the National Library of Medicine under contract # HHS-N-276-2011-00008-C.