Skip all navigation and go to page content

OERC Blog

NN/LM Outreach Evaluation Resource Center

Archive for the ‘Library Value’ Category

Institute for Research Design in Librarianship: 9 days in southern CA; full scholarships available

Monday, December 2nd, 2013

Do you want to learn about how your user groups and communities find and use information? Do you want to gather evidence to demonstrate that your work is making a difference?

Exciting news! You can work on these questions, and questions like them, June 16-26, 2014!

The Institute for Research Design in Librarianship is a great opportunity for an academic librarian who is interested in conducting research. Research and evaluation are not necessarily identical, although they do employ many of the same methods and are closely related. This Institute is open to academic librarians from all over the country. If your proposal is accepted, your attendance at the Institute will be paid for, as will your travel, lodging, and food expenses.

The William H. Hannon Library has received a three-year grant from the Institute for Museum and Library Services (IMLS) to offer a nine-day continuing education opportunity for academic and research librarians. Each year 21 librarians will receive instruction in research design and a full year of support to complete a research project at their home institutions. The summer Institute for Research Design in Librarianship (IRDL) is supplemented with pre-institute learning activities and a personal learning network that provides ongoing mentoring. The institutes will be held on the campus of Loyola Marymount University in Los Angeles, California.

The Institute is particularly interested in applicants who have identified a real-world research question and/or opportunity. It is intended to

“bring together a diverse group of academic and research librarians who are motivated and enthusiastic about conducting research but need additional training and/or other support to perform the steps successfully. The institute is designed around the components of the research process, with special focus given to areas that our 2010 national survey of academic librarians identified as the most troublesome; the co-investigators on this project conducted the survey to provide a snapshot of the current state of academic librarian confidence in conducting research. During the nine-day institute held annually in June, participants will receive expert instruction on research design and small-group and one-on-one assistance in writing and/or revising their own draft research proposal. In the following academic year, participants will receive ongoing support in conducting their research and preparing the results for dissemination.”

Your proposal is due by February 1, 2014. Details are available at the Institute’s Prepare Your Proposal web site.

Factoid: Loyola Marymount is on a bluff above the Pacific Ocean, west of central LA.

New, Improved, and Available Now!

Thursday, September 12th, 2013

The 2nd Edition of the Planning and Evaluating Health Information Outreach Projects series of 3 booklets is now available online:

Getting Started with Community-Based Outreach (Booklet 1)
What’s new? More emphasis and background on the value of health information outreach, including its relationship to the Healthy People 2020 Health Communication and Health Information Technology topic areas

Planning Outcomes-Based Outreach Projects (Booklet 2)
What’s new? Focus on uses of the logic model planning tool beyond project planning, such as providing approaches to writing proposals and reports.

Collecting and Analyzing Evaluation Data (Booklet 3)
What’s new? Step-by-step guide to collecting, analyzing, and assessing the validity (or trustworthiness) of quantitative and qualitative data, using questionnaires and interviews as examples.

These are all available free to NN/LM regional offices and network members. To request printed copies, send an email to nnlm@uw.edu.

Non-508 compliant pdf versions of all three booklets are available here: http://nnlm.gov/evaluation/guides.html#A2 .

The Planning and Evaluating Health Information Outreach series, by Cynthia Olney and Susan Barnes, supplements and summarizes material in Cathy Burroughs’ groundbreaking work from 2000, Measuring the Difference: Guide to Planning and Evaluating Health Information Outreach. Printed copies of Burroughs’ book are also available free—just send an email request to nnlm@uw.edu.

The Value of Academic Libraries

Friday, September 23rd, 2011

Last year the Association of College and Research Libraries issued a very substantial and thorough review of the research that has been done on how to measure library value:  “The Value of Academic Libraries: A Comprehensive Research Review and Report” by Megan Oakleaf.  Althought its focus is academia, there are sections reviewing work in public libraries, school libraries, and special libraries.  I recommend the section on special libraries, which has quite an emphasis on medical libraries, including references to past work regarding clinical impacts.

For those who are interested in approaches that libraries have taken to establishing their value, there is potential benefit in reading the entire report cover-to-cover.  For those who want a quick overview, these are sections that I recommend:

  • Executive Summary
  • Defining “Value”
  • Special Libraries

For more information about this report, see “A tool kit to help academic librarians demonstrate their value” from the 9/14/2010 issue of the Chronicle of Higher Education.

The full report is available at http://www.acrl.ala.org/value/.

The Critical Incident Technique and Service Evaluation

Thursday, September 22nd, 2011

In their systematic review of clinical library (CL) service evaluation, Brettle et al. summarize evidence showing that CLs contribute to patient care through saving time and providing effective results.  Pointing out the wisdom of using evaluation measures that can be linked to organizational objectives, they advocate for using the Critical Incident Technique to collect data on specific outcomes and demonstrate where library contributions make a difference.  In the Critical Incident Technique, respondents are asked about an “individual case of specific and recent library use/information provision rather than library use in general.”  In addition, the authors point to Weightman, et al.’s suggested approaches for conducting a practical and valid study of library services:

  • Researchers are independet of the library service.
  • Respondents are anonymous.
  • Participants are selected either by random sample or by choosing all members of specific user groups.
  • Questions are developed with input from library users.
  • Questionnaires and interviews are both used.

Brettle, et al., “Evaluating clinical librarian services: a systematic review.”  Health Information and Libraries Journal, March 2010.  28(1):3-22.

Weightman, et al., “The value and impact of information provided through library services for patient care: developing guidance for best practice.”  Health Information and Libraries Journal, March 2009.  26(1):63-71.

Value of Academic Libraries: IMLS Grant and ACRL Project

Monday, July 12th, 2010

The Institute of Museum and Library Services has awarded a grant that will test and implement methodologies measuring the return on investment (ROI) in academic libraries. The goals are to provide evidence and a set of tested methodologies that academic libraries will be able to use in demonstrating their value. The University of Tenessee, Knoxville will be conducting this study in collaboration with the University of Illinois at Urbana-Champaign and the Association of Research Libraries. Dr. Carol Tenopir, professor in the School of Information Sciences, is the project’s lead investigator. This news item from the UT Knoxville’s Tennessee Today provides more details: “UT Shares in Grant to Study Value of Academic Libraries.”

In a complementary project, the Association of College and Research Libraries has selected Dr. Megan Oakleaf to conduct a review of the “quantitative and qualitative literature, methodologies and best practices currently in place for demonstrating the value of academic libraries.” The Association plans to issue a completed report later this year.

Hospital Libraries and IRS-Mandated Community Benefit: MLA Poster

Tuesday, July 6th, 2010

A belated note about an interesting item at the Medical Library Association meeting in DC this past May. Christine Chastain-Warheit gave a fascinating 5-minute “Lightning Round” presentation, “Can Hospital Librarians Demonstrate Internal Revenue Service-mandated Community Benefit for Their Nonprofit Organizations? Reflecting on Value Provided and Connecting the Hospital Library to Community Benefit.” She pointed out that the IRS Community Benefit standard for not-for-profit hospitals includes activities that promote health in response to community needs. Community Benefit is the basis of the tax-exemption of not-for-profit hospitals. Her institution has agreed that the library’s outreach activities can be included in calculating hospital community benefit efforts for IRS reporting (Poster presented at Medical Library Association Annual Meeting, May 23, 2010). This approach could have good potential for libraries in not-for-profit hospitals demonstrating their value to their institutions and their institutions’ communities, so it’s definitely something to watch. Ms. Chastain-Warheit is Director of Medical Libraries at Christiana Hospital in Newark, DE.

ROI in MLA News

Thursday, November 20th, 2008

In the October, 2008 issue of the MLA News, Terrance Burton presented a quick overview of how the business, dollor-based Return on Investment approach (profit gained from invested dollars) can be expanded to a view of what value is derived from whatever inputs you choose to work with.  This can be more relevant in the library world, where profit is not the main point.  In fact, in many health-related institutions served by libraries, profit is not the main point.  Money is indeed invested in libraries, but it can be challenging to assess the outputs from that investment.  Burton suggests that we identify outcomes that we want to measure, use a mix of approaches (quantitative, qualitative, speculative), and accept that any measure will be flawed.  Despite flaws, using mixed approaches can provide various indicators to bolster a case.

Healthcare Services Managers’ Decision-Making and Information

Thursday, November 20th, 2008

MacDonald, J; Bath, P; Booth, A.  “Healthcare services managers:  What information do they need and use?”  Evidence-Based Library and Information Practice.  3(3):18-38.

This paper presents research results that provide insights into how information influences healthcare managers’ decisions.  Information needs included explicit Organizational Knowledge (such as policies and guidelines), Cultural Organizational Knowledge (situational such as buy-in, controversy, bias, conflict of interest; and environmental such as politics and power), and Tacit Organizational Knowledge (gained experientially and through intuition).  Managers tended to use internal information (already created or implemented within an organization) when investigating an issue and developing strategies.  When selecting a strategy, managers either actively looked for additional external information–or else they didn’t, and simply made a decision without all of the information that they would have liked to have.  Managers may be more likely to use external information (ie, research-based library resources) if their own internal information is well-managed.  The article’s authors suggest that librarians may have a role in managing information created within an organization in order to integrate it with externally created information resources.

Library Assessment Challenges

Monday, August 11th, 2008

The Library Assessment Conference took place in Seattle from August 4-7 and at the opening session, the audience heard three academic library directors’ perspectives on the “Most Important Challenge for Library Assessment.”

Susan Gibbons, Dean of the University of Rochester’s River Campus Libraries, opened with the observations about the attractions of qualitative data: they give you a sense of “precision” and a “correct” answer, they’re perceived as weighty, and their collection can be automated. She emphasized the importance of thinking about what we are counting, and why, and provided the example of the decrease in reference questions answered at the University of Rochester by 10,000 from 1996 to 2006. To learn about the “why” behind this quantitative finding, the library used qualitative approaches. For example, they asked students to take pictures of what they carry with them all the time, to map out their daily movements, to indicate what is useful/not useful by writing on a printed copy of the library’s web page, and to imagine what they would wish for if they had a magic wand. They learned that all students carry cell phones but that their library’s phone number did not appear on its home page on the web (in fact, 40% of ARL libraries’ home pages lack phone numbers!) and that students’ peak time period for studying is from 11pm to 1am. Those findings led to better visibility of the library’s phone number on the web and near library computers. Through the magic wand exercise they learned the importance of providing skills and tools to graduate students early in their careers. She emphasized that local assessment methods are required since every campus is unique and accountability is local. If opportunities are available for wide staff participation in assessment, changes are easier and work better.

Rick Luce, Director of Libraries at Emory University, characterized assessment as a method of planning for improvement–a catalyst for change rather than a quick fix. Performance measures are an organization’s vital signs through metrics that show innovation, research leadership, brand identity, and gains in market share. Successful organizations offer something that others can’t do, do poorly, or have difficulty doing well. Satisfaction can be studied through questionnaires that function as “happiness meters,” investigation into what’s important, and looking at how an organization rates against the best in an industry. He cautioned that assessment efforts can be hampered by pitfalls such as lack of accountability, too many initiatives, forgetting larger organizational drivers, and lack of discipline. He reminded us that time and patience are needed for real change in organizations. Providing a brief mention of the “Hedgehog” concept (a single, simple idea that guides great organizations’ efforts to be the best) from Jim Collins’ book Good to Great, he urged us to understand what we are passionate about, what we are best at, and what drives our economic engines.

Betsy Wilson, Dean of Libraries at the University of Washington, provided her perspective that the most important challenge for libraries is accelerating relevance. Assessment can help by providing fuel for that acceleration. So, it is extremely important for libraries that assessment becomes part of their organizational lifeblood, turning cultures of complaint into cultures of assessment.

More from MLA on Library Value

Tuesday, July 15th, 2008

This year’s Medical Library Association annual meeting in Chicago had several good sessions in which speakers presented experiences and approaches to assigning dollar values to library services and activities. These included:

  • “A Calculator for Measuring the Impact of Health Sciences Libraries and Librarians” presented by Betsy Kelly and Barb Jones of the MidContinental Region, National Network of Libraries of Medicine–Their calculators include the Valuing Library Services Calculator and the Cost Benefit and ROI Calculator. These have the potential to be very useful tools.
  • “Connecting with Administrators: Demonstrating the Value of Library Services” presented by Edward J. Poletti of the Central Arkansas Veterans Health Care System in Little Rock, AR–He and VA Library colleagues conducted value studies of shared electronic resources, ILL, and literature searches. His presentation included a list of sources of dollar values such as Fortney’s “Price History for Core Clinical Journals in Medicine and Nursing 2003-2007” and “Doody’s core titles in the health sciences 2007: list overview and analysis.” This paper received honorable mention for the MLA Research Award, and a summary is available at the MLA Federal Libraries Section blog.
  • “Bridging the Gap: Using Dollar Values to Demonstrate the Value of Library Services” presented by Julia Esparza of Louisana State University Health Sciences Center in Shreveport, LA–Her experience with assigning and tracking dollar values included analysis of copying/printing costs and article costs.
  • “Quantum Physics and Hospital Library Assessment” presented by Michele Klein-Fedyshin of UPMC Shadyside, Pittsburgh, PA–Assessment must be locally relevant and there are various possible foci, such as the financial impact of local consortia, the impact of library services on nursing certification, prevention of hospital acquired infections, cost savings from library contributions to pay-for-performance, library as drug information center, etc.

Last updated on Saturday, 23 November, 2013

Funded by the National Library of Medicine under contract # HHS-N-276-2011-00008-C.