National Network of Libraries of Medicine nnlm.gov	Archives	Search nnlm.gov: nnlm home | About the archives
This page was archived on: May 01, 2008 | View page metadata		Document content is not current. Links may be broken.

Dragonfly, Summer 2000 Success with Outreach: Introduction to Planning and Evaluation

by Cathy Burroughs, Network Librarian, Evaluation
NN/LM Pacific Northwest Region

Editor's note: This is the first in a series of Dragonfly articles on "Success with Outreach." An earlier version of this article appeared in 3 Sources, December 1999, 17 (6). (Used here with permission.)

According to the American Heritage Dictionary, 2nd college edition, the term outreach means: "To reach or go beyond; surpass... A systematic attempt to provide services beyond conventional limits, as to particular segments of a community." Several concepts in this definition fit with what health information outreach means. When conducting outreach, we do surpass the boundaries of traditional on-site medical library services, and we do reach out to address the problems or needs of a targeted clientele.

Are we systematic in our attempts? That probably depends on what is meant by systematic. We think our outreach efforts must certainly help bridge the gap between information needed and information found by the priority audiences we serve. We can point to numbers of classes, demonstrations, exhibits, and information services provided. We receive positive feedback from our class evaluations and other ways. But, there's always a nagging sense that we do not know enough about the impact we are making and what we could do that would be more effective. And in the end, what differences are we making as a result of our efforts?

These and other good questions can be addressed with the help of a planning and evaluation process. The process of evaluation nudges us to get to know our audience and ask the right questions at the beginning so that it is much easier to plan for and get useful results at the end. Whether formally or informally, if we are setting priorities, creating objectives, and assessing progress, outreach will benefit from a systematic approach embraced by planning and evaluation.

Thus, this article begins a series to discuss the ins and outs of these two concepts -- planning and evaluation: how they work together and what we can do to benefit from their use. Here's an outline of the topics to be included in the series:

• Introduction to program evaluation
• Problem identification: Assessing the community
• Getting to goals: Developing measurable objectives
• Planning activities: Tailoring theory-based strategies to needs of target audience
• Planning the evaluation: Figuring out the questions and how to answer them
• Coming full circle: Utilizing results

The series will be based in large part on a forthcoming guide about planning and evaluating outreach, developed in collaboration with the NLM and the NN/LM, Pacific Northwest Region. The guide will be published later this year; readers can refer to a draft at the following web site: http://www.nnlm.nlm.nih.gov/pnr/eval/. The same web site provides details and resources of the planning and evaluation project -- conceived, funded and overseen by NLM -- including a list of its stellar advisors and fascinating white papers about best practices in outreach and evaluation in seven related disciplines.

Introduction to Program Evaluation

Evaluation in a sense is a constant process of obtaining feedback from the community you work with during the entire cycle of developing and implementing the outreach program.

And even though it's ideal to begin thinking about evaluation at the start, it's also never too late to bring evaluation into an outreach effort.

When combined together, planning and evaluation help us:

• Set goals and objectives
• Tailor activities to the needs of those we want to reach
• Be accountable for our objectives
• Schedule ways to track or measure progress
• Discover unanticipated benefits
• Improve outreach strategies
• Justify outreach or provide evidence for additional funding
• Encourage partnerships with other organizations
• Increase understanding and support for outreach in the target community

Overall, evaluation sharpens an outreach focus; provides accountability to our funders, managers, or administrators; improves quality and effectiveness; and increases understanding of what we've achieved. As demonstrated by many studies in the library literature, when shared with the outreach community, evaluation research contributes to a growing knowledge base from which we can all benefit.

There are many types of health information outreach initiatives that vary according to site, setting, need, resources, and other characteristics of the community to be reached. If you consider that an outreach initiative is really a program, the process of conducting various types of assessment and evaluation will clearly benefit its development.

The basic idea of program development and evaluation involves a process of prioritizing and refining outreach efforts through a continuous cycle of feedback from the community. Effective program development means you treat community members as major stakeholders in the whole process.

Evaluation textbooks represent the process of program development in a 6-phase circular flow chart. Below is a sketch of each stage in this program development cycle. You'll see where various types of evaluation conducted in each stage help planners make decisions, assess quality, track progress, test theories or assumptions, and determine results.

• Stage 1 - Assessing Needs
  A community assessment, including input from key stakeholders as well as the targeted audience, helps test, revise, or refine assumptions about the needs and priorities of the program.
• Stage 2 - Developing Goals and Objectives
  Clarifying the cause of problems and needs identified in the community assessment helps identify overall goals and develop realistic objectives to achieve them. A critical step in this phase is to involve community stakeholders in developing mutual goals and objectives.
• Stage 3 - Developing Activities and Strategies
  Specific plans and strategies might benefit from behavior change theories borrowed from the field of health communications. Audience assessment is conducted prior to outreach to help tailor activities and then again while the program is ongoing to help track progress and monitor effectiveness.
• Stage 4 - Evaluating Implementation and Outcomes
  What do you want to know about your outreach initiative? Planning for evaluation is ideally accomplished before activities begin. Identify evaluation objectives first, and prioritize what can be done with available resources.
• Stage 5 - Gathering Data and Assessing Results
  The challenge of planning an evaluation involves decisions about how best to gather and analyze the measurements you obtain.
• Stage 6 - Utilizing and Reporting Results
  At each stage of outreach development, useful information is gathered about the targeted audience, the activities conducted, and the intended effects of these activities. This information will help ensure further outreach success in your own program and may contribute to the success of other programs as well.

Some negative perceptions and fears about evaluation stem from a feeling that it involves a lot of effort and that results are not always used. It's true that asking questions for the sake of asking might be an interesting exercise, but will not necessarily be useful or contribute to a decision-making process. And it's important that evaluation be realistic in its ambition. As a general rule of thumb, allocate 5 percent of the total program budget to evaluation efforts. This means making priorities about what you want to find out by trying to decide in advance why you are asking the question and what you will do with the results.

So while planning and evaluation sound complex, it can be simple and still be of benefit. Plus there are lots of resources to help. The evaluation guide will provide helpful tips as well as point you to other resources. The research section of the Medical Library Association, and your local university can provide assistance. There are also increasing numbers of web-based resources.

URL: http://nnlm.gov/pnr/news/200007/success-1.html