Archive for the ‘General’ Category
For public sector and nonprofit organizations, social media can be a cost effective way to engage with users and supporters. However, social media is not without its cost, particularly in terms of staff time. So organizations have an interest in assessing the value of their social media activities.
One great resource for social media evaluation is Paine’s book, Measure What Matters. The book contains detailed guidance for evaluating social media use by different types of organizations. A great supplement to Paine’s book is The Nonprofit Social Media Decision Guide, by Idealware, which has worksheets that will help plan social media strategies and implement recommendations in Measure What Matters.
Below are the key elements of Paine’s evaluation framework:
- Begin with a solid social media plan that identifies specific goals and objectives. As with any project, you need a plan for social media that links strategies to the organizational mission and includes objectives with targets and key performance indicators. Objectives for social media in the public sector often belong in one of two categories: helping users find information they need; or building user awareness, engagement, or loyalty. (The Nonprofit Social Media Decision Guide provides a list of potential objectives on page 52.)
- Define your target audience: Organizations often have many stakeholder groups, so it’s important to identify the groups most attuned to social media. On page 54 of The Nonprofit Social Media Decision Guide, there is a worksheet for narrowing down stakeholder audiences to those most receptive to social media activities.
- Pick your metrics: Metrics such as views, followers, and measures of engagement with online content will help monitor your reach. Conversions, defined as the actions you want your social media followers to complete, might include becoming members of your organization or actively recommending your organization to colleagues or friends.
- Identify a source for benchmarks. Benchmarks provide a basis for comparison to assess progress. Organizations often use their own histories as benchmarks, comparing progress against baseline measures. You also may have access to data from a competing or peer organization that you can use for comparison.
- Pick a measurement tool: Paine’s book describes different measurement methods for evaluating social media, such as content analysis, web analytics, or surveys.
For more information, check out the resources mentioned in this blog post:
- Katie Delahaye Paine, Measure What Matters: Online Tools For Understanding Customers, Social Media, Engagement, and Key Relationships. Hoboken, NJ: John Wiley & Sons, Inc, 2011.
- Idealware. The Nonprofit Social Media Decision Guide, 2013.
In the summer of 1946, an Oklahoma newspaper editor sent a young reporter to complete a story on a state psychiatric hospital, where he found neglected, half-naked inmates, crowded together in filthy, dilapidated buildings, and fed on rotten food. He soon went back, taking along a photographer, and then he went to visit Oklahoma’s other state mental hospitals. His blistering series of newspaper articles about the institutions launched a grassroots reform movement: less than a year later, the state legislature voted huge budget increases for state hospitals, restructured the state hospital administration, and re-wrote the state’s commitment laws.
The young journalist was Mike Gorman (1913–1989). His work in Oklahoma earned him a Lasker Award in 1948, and changed the course of his career. Several decades later he would be called “the country’s greatest modern missionary for mental health.” Mike Gorman’s papers are now online at the National Library of Medicine’s Profiles in Science Web site, an NLM digital project that provides online access to the archival collections of more than 30 Nobel Laureates and other leading innovators in scientific and medical fields. The presentation features correspondence, photographs, speeches and addresses given by Gorman, speeches he wrote for members of Congress and several U.S. Presidents, along with published articles and reports from the Gorman collection. Visitors to the site can view his first series of articles for the Daily Oklahoman, drafts of speeches Gorman wrote for Presidents Truman and Kennedy, and the public service announcements issued by the Citizens for the Treatment of High Blood Pressure.
Gorman’s Oklahoma experience taught him that newspaper exposés alone would not produce substantive changes. Public attention to social problems faded quickly, and entrenched social and political practices did not change without constant agitating from outside. Gorman would spend the rest of his life providing that agitation: gathering the facts about mental illness and other diseases; speaking to governors, legislators, professional groups, and the public; testifying to Congressional appropriations committees; and writing books and articles.
Gorman came to Washington, D.C. in 1951 to be a member of President Truman’s Commission on the Health Needs of the Nation, and in 1953 became executive director of the National Committee Against Mental Illness, a lobbying and advocacy organization founded by philanthropist and health care activist Mary Lasker. In that post, he became perhaps America’s best known lobbyist and publicist in the crusade for psychiatric hospital reform and the community mental health center movement. Gorman played a key role in shaping many of the social programs of the Kennedy and Johnson administrations, including the Mental Retardation Facilities and Community Mental Health Centers Construction Act of 1963. During the 1970s and 1980s he also directed two other advocacy groups, Citizens for the Treatment of High Blood Pressure, which helped coordinate a highly successful national hypertension education and screening program, and the National Initiative for Glaucoma Control.
The National Library of Medicine (NLM) has a deep interest in the publishing models used by scientific journals, from the viewpoints of practical and efficient use of titles that are indexed for MEDLINE, and the clear and accurate preservation of the scientific literature for use by future generations. Now you have the opportunity to participate in the development of a National Information Standards Organization (NISO) Recommended Practice that provides guidance on the presentation and identification of electronic journals!
PIE-J: The Presentation & Identification of E-Journals, a NISO (National Information Standards Organization) Recommended Practice, was published just over a year ago, having been approved on March 25, 2013. In just over 12 months, the full Recommended Practice document has received well over 4500 downloads, while two PIE-J brochures have received a total of more than 2000 downloads. All three documents can be accessed from the PIE-J website. This level of download activity suggests that PIE-J is meeting a need, and it is essential that librarians, publishers, and other e-journal providers be aware of its existence. The PIE-J Standing Committee, co-chaired by Sally Glasser (Hofstra University) and Ed Cilurso (Taylor & Francis), is charged with responding to specific questions about the Recommended Practice, gathering comments for a full review of the Recommended Practice document, and promoting PIE-J.
If you have written to publishers or providers about PIE-J, the Standing Committee would like to hear from you, whether the result was positive or negative. Likewise, the Committee would greatly appreciate hearing from publishers and providers who have made changes to their websites based on PIE-J and user feedback, intend to make changes based on PIE-J during a future website redesign, or feel that the recommended practices are not feasible. Please write the Committee with the subject “PIE-J feedback.” Standing Committee members have been busy making the rounds at various conferences and meetings. Next up are NASIG (May 1-4, Fort Worth, TX), the Society of Scholarly Publishers (SSP) conference (May 28-30, Boston), and ALA Annual (June 28-July 1, Las Vegas). If you plan to attend any of these conferences, please look out for NISO’s PIE-J presentations! Also, on Monday, May 12, 2014, at 12 PM PDT co-chairs Sally Glasser and Ed Cilurso will be speaking about PIE-J at NISO’s monthly Open Teleconference.
The Standing Committee recently posted a template to the PIE-J website for librarians wishing to contact publishers and providers with concerns about the presentation of e-journals on their websites. The template includes suggested wording but is completely customizable. If you (or your users) have experienced an access or display issue that is due to the way in which e-journals are presented online, use the template to let publishers and providers know how PIE-J can help. Regina Reynolds, who was on the original PIE-J Working Group and has continued on the Standing Committee, recently published the freely accessible article, “PIE-J: Presentation and Identification of E-Journals: What’s the Point?” in Insights: the UKSG Journal, vol. 26, no. 3 (Nov. 2013). The article provides an excellent overview of PIE-J.
Terrence Sejnowski, PhD, will give the 2014 Joseph Leiter NLM/Medical Library Association (MLA) Lecture, “The BRAIN Initiative: Connecting the Dots,” on Thursday, June 12, 2014, at 10:00 am PDT at the National Library of Medicine. The lecture will be recorded and broadcast live on the Web. Dr. Sejnowski is a pioneer in computational neuroscience and his goal is to understand the principles that link brain to behavior. His laboratory uses both experimental and modeling techniques to study the biophysical properties of synapses and neurons and the population dynamics of large networks of neurons. New computational models and new analytical tools have been developed to understand how the brain represents the world and how new representations are formed through learning algorithms for changing the synaptic strengths of connections between neurons. By studying how the resulting computer simulations can perform operations that resemble the activities of the hippocampus, Dr. Sejnowski hopes to gain new knowledge of how the human brain is capable of learning and storing memories. This knowledge ultimately may provide medical specialists with critical clues to combating Alzheimer’s disease and other disorders that rob people of the critical ability to remember faces, names, places and events.
Dr. Sejnowski is an Investigator with the Howard Hughes Medical Institute and holds the Francis Crick Chair at The Salk Institute for Biological Studies. He is also a Professor of Biology at the University of California, San Diego, where he is co-director of the Institute for Neural Computation and co-director of the Temporal Dynamics of Learning Center. He has published over 400 scientific papers and 12 books, including The Computational Brain, with Patricia Churchland. He is a member of the Institute of Medicine, National Academy of Sciences, and the National Academy of Engineering, one of only 13 living persons to be a member of all 3 national academies. Dr. Sejnowski was instrumental in shaping the BRAIN Initiative that was announced from the White House on April 2, 2013, and serves on the Advisory Committee to the Director of NIH for the BRAIN Initiative.
On May 14, 2014, the Board of Regents of the National Library of Medicine (NLM), the Friends of the NLM, and the Medical Library Association are co-sponsoring a symposium The National Library of Medicine, 1984-2014: Voyaging to the Future, to be held at the Natcher Center on the NIH campus in Bethesda, MD. The purpose of the symposium is to review the influence of NLM’s long range planning over the past 30 years; to reflect on key factors that contributed to successes and setbacks; and to consider opportunities for the future, all as background for the next NLM long range planning effort, to commence in 2015. The symposium is free, but registration is required. A preliminary program is also available. The symposium will be available for remote simultaneous viewing and also archived for future viewing.
In conjunction with this event, NLM is collecting written recollections and images reflecting the Library’s impacts over the last 30 years, as well as ideas for future opportunities and directions. Anyone who has advised and worked with/for NLM and/or benefited from its programs and services is encouraged to submit contributions to a moderated blog. Comments will be accepted throughout the year.
SurveyMonkey recently launched a mobile app for the iPad and iPhone, providing the ability to create, send, and monitor surveys from a phone or tablet device. The app is free, although you need a SurveyMonkey account to use it. With the new app, there’s no longer a need to rely on a computer to design and manage surveys. The app also allows convenient viewing of data from any location with Internet access. Another notable benefit is that the analytic reports are optimized for mobile devices and are easy to read on small screens. Although there is not yet an Android app, all SurveyMonkey pages and surveys are optimized for any mobile device, so surveys are easy to take regardless of the operating system used.
The American Medical Association has specific recommendations for its authors about questionnaire response rates included in the JAMA Instructions for Authors. One of the guidelines is that survey studies should have sufficient response rates (generally at least 60%) and appropriate characterization of nonresponders to ensure that nonresponse bias does not threaten the validity of the findings. However, response rates to questionnaires have been declining over the past 20 years, as reported by the Pew Research Center in The Problem of Declining Response Rates. Fortunately, suggestions about increasing questionnaire response rates are available in two recent AEA365 blog posts that are open access:
Additional useful advice, such as making questionnaires short, personalizing your mailings, and sending full reminder packs to nonrespondents, is included in this open access article: Sahlqvist S, et al., “Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial.” BMC Medical Research Methodology 2011, 11:62.
The National Library of Medicine has announced the launch of a new user interface for the IndexCat database, which offers a faster response time to searches; full record displays in search results; and record sorting and refinements. NLM uses the same search engine for its main Web site, as well as MedlinePlus, MedlinePlus en Español, the Directory of the History of Medicine Collections search engine, and the History of Medicine Finding Aids Consortium.
IndexCat simultaneously searches the digitized version of the printed Index-Catalogue of the Library of the Surgeon General’s Office; eTK for medieval Latin texts; and eVK2 for medieval English texts; and LocatorPlus. A post in the NLM Circulating Now blog offers additional information on IndexCat. There also are a number of new and revised FAQs and Help pages to assist with searching IndexCat. Additional details and illustrations are available in the most recent edition of the NLM Technical Bulletin.
The current trend in evaluation reporting is toward fewer words and more images. There are a number of companies that offer high-quality, royalty free photographs at minimal cost. Stockfresh, for example, charges as little as $1 per image. However, no-cost is even better than low-cost. Freelancers Union, a nonprofit organization dedicated to assisting freelance workers, recently published a list of the best websites for no-cost images. If you are looking for free images for your presentations or reports, check out their article, which also describes the difference between public domain, royalty-free and Creative Commons-licensed images.
Stanley Capela recently presented the webinar Recipe of Evaluation Techniques for the Real World, one of the American Evaluation Association’s (AEA) ongoing 20-minute Coffee Break webinars. The webinars, offered Thursdays at 11:00 am Pacific time, often present similar tools and tips that are also covered in the Tip a Day blog but allow for audience questions & answers and networking with the presenters. Capela’s recipe focused primarily on internal evaluation in non-profit or government settings where people are seeking realistic answers in response to assessment efforts. His tips include:
- Value People’s Time – all time is valuable, regardless of who you are working with, and clear communication on the intent of the evaluation helps to make the best use of everyone’s time.
- Ethical Conduct – working within the parameters of organizational and/or professional association codes of conducts in addition to established support of upper level administration will help to minimize the potential for ethical dilemmas.
- Know Your Enemies – be aware of those who are resistant to program evaluation and may try to undermine these efforts, and also know that you as an evaluator may be perceived as an enemy by others. Again, clear communication helps!
- Culture of Accountability – take the time to know the story of those you are working with – where are they coming from? What is their history with previous assessments? Were their needs met, or were there issues that had negative effects on relationships and outcomes?
- Do Something – avoid cycles of conducting reviews, identifying deficiencies, and outcomes that only include developing correction plans. Also important to note is that program evaluation does not solve management problems.
- A Picture is Worth 1,000 Words – find ways to integrate charts that direct the reader to the most important information clearly and concisely.
- Let Go of Your Ego – working from a mindset that accepts the people conducting the program itself will most likely ‘get the credit,’ and that your measure of success is doing your job to the best of your ability and knowing you made a difference.
- Give Back – develop a network of trusted colleagues, such as through personal and organization connections on LinkedIn and other platforms, share ideas, and ask questions, since others have probably encountered a similar situation or can connect you with those who have.