Archive for the ‘Communications Tools’ Category
A hot trend in marketing research is the micro-survey. Also known as the bite-sized survey, these questionnaires are short (about three questions) with the goal of collecting focused feedback to guide specific action. The micro-survey is a technique for overcoming what is arguably the biggest hurdle in survey assessment: Getting people to respond to your questionnaire. It is a technique that is particularly useful for populations where mobile technology use is on the rise, and where there is competition for everyone’s attention in any given moment. To better expect respondents to answer questionnaires, don’t burden them with long, matrix-like questions or require them to flip through numerous web pages. Keep things simple, or respondents will be lost before they ever get to the submit button.
The trick to micro-surveys is to keep them short, but administer multiple questionnaires over time. For example, break down a traditional membership or customer questionnaire into several micro-surveys and distribute them periodically. The length of the survey is not the only factor contributing to response rate. Follow the Dillman method, which provides time-tested guidelines for administering surveys. Also, take a look at Champagne’s Nine Principles of Embedded Assessment. His website has articles and YouTube videos on how to implement these principles. If you want to try doing a micro-survey, check out the effective practices described in this blog article from the marketing research company Instantly.
Have you ever found yourself trying to do an evaluation activity, but needing that one helpful tool? Or perhaps you need a step-by-step guide on how to do a community assessment, or are looking for ways to build evaluation into a project that you are planning. The NN/LM Outreach Evaluation Resource Center (OERC) has developed the online guide Tools and Resources for Evaluation to assist with program evaluation. Following are some of the types of tools and resources described in the Guide.
Community Oriented Outreach
- Tips on successful collaborations and tools for improving collaboration with community networks.
- Toolkits for practical participatory evaluation and processes for conducting outcome-based evaluations.
- Step-by-step guides on incorporating evaluation planning into your outreach projects.
- Instructions on using logic models for program planning.
Data Collection and Analysis
- Tips for questionnaire development.
- Resources for statistical methods of data analysis.
- Guides for analyzing qualitative and quantitative data.
Reporting and Visualizing
- Help with creating popular data dashboards.
- Descriptions of data visualization methods.
- Tools and TED talks about how to present your data.
The American Evaluation Association (AEA) just concluded a week-long blog theme about qualitative evaluation. Following are some highlights to consider using in your own assessment efforts:
- The Role of Context: the authors of this entry previously shared five high quality elements of qualitative evaluation, and this entry referenced them while emphasizing the need for evaluators to understand what role setting, relationships, and other context factors play in data as well.
- Purposeful Sampling: a great explanation on why to avoid convenience sampling (interviewing people because they happen to be around) and using caution with your qualitative evaluation terminology to consider not using the word ‘sampling’ due to peoples’ association of it with random probability.
- Interviewing People who are Challenging: establishing rapport leads to good qualitative data, but what does an interviewer do if there seems to be conflict with the interviewee? Details about how to manage your own feelings and approach with a curious mindset are very helpful!
- Asking Stupid Questions: this example from a bilingual HIV/AIDS training is especially insightful about the importance of clarifying sexual terms, putting aside concerns the evaluator may have about looking ‘stupid,’ and outcomes that led to deeper engagement and discussion from the group.
- Practical Qualitative Analysis: many helpful tips and lessons shared, including the reminder of being sure to group our participants’ responses that answer the same question together even if these replies come from different parts of the survey or interview.
- Providing Descriptions: sometimes there are concerns expressed that evaluation is ‘only looking at the negative,’ and by including full details about your qualitative inquiry collection and analysis as an additional resource or appendix you can help explain the steps of the process that otherwise may not be evident.
BetterEvaluation.org is an international collaboration that encourages sharing of evaluation methods, approaches, and processes for improvement. BetterEvaluation offers yearly blog themes for their staff and guest writers to focus on, with highlights of the 2014 theme published as a blog posting, 52 Weeks of BetterEvaluation. For 2015 they are featuring 12 Months of BetterEvaluation, with multiple posts each month, starting with impact evaluation in January. Following are five selections from 2014 that may be of special interest to NN/LM Network members:
- Top ten developments in qualitative evaluation over the past decade, Part 1 and Part 2.
- Fitting reporting methods to evaluation findings and audiences.
- Infographics, including step-by-step instructions in piktochart.
- Innovation in evaluation.
- Presenting data effectively.
Monitoring and Evaluation (M&E) is a form of assessment used to help improve the performance and achievement of program results and often used by both non-government organizations (NGOs) and government agencies. It utilizes a staircase diagram with six questions related to program planning, monitoring, and evaluation; with information about clarifying the differences. While not specific to health information outreach programs, the Measuring Success Toolkit, from the Urban Reproductive Health Initiative, is about health program planning, monitoring and evaluation. The toolkit provides helpful resources from the initiative’s multi-country perspective of working with the urban poor and the significant health disparities they face that may be helpful to consult when working with health information outreach partners in underserved communities. The Toolkit includes subject-specific M&E resources such as maternal & child health and HIV/AIDS, and the resources within the toolkit are selected by M&E experts and reviewed quarterly following established criteria, to identify important resources from diverse perspectives that include accurate, up-to-date information.
In January, 2015, the NIH Manuscript Submission (NIHMS) system will be getting a new interface design, which will streamline the login and manuscript submission processes and provide relevant help information on each screen. The NIHMS sign-in routes will be available from the homepage, with options based on a funding agency or signing in through NCBI. The new homepage will also include a graphic overview of the NIHMS process, allowing you to hover over each step for more information or to click on “Learn More” to read the complete overview in the FAQ. Once you are signed in to NIHMS, you will be directed to your Manuscript List. From this page you can manage and track your existing submissions, submit a new manuscript, and search for a record. You can also click on any headings in the information box to expand a topic and read the help text. The initial deposit will still require you to enter a manuscript and journal title, deposit complete manuscript files, and specify funding information and the embargo.
Key updates will include:
- Assigning an NIHMSID to a record only after files have been uploaded, i.e., at the Check Files step;
- A streamlined deposit process with clearly defined and explained actions in each step;
- Requiring the Submitter to open the PDF Receipt to review the uploaded files and confirm that the submission is complete before advancing to the next step;
- Relevant help information on each page; and
- Requiring the Reviewer to add funding before approving the initial deposit.
The National Library of Medicine’s Refugee Health Information Network (RHIN) resource was a national collaborative partnership with the principal focus of creating and making available a database of quality multilingual/multicultural, public health resources to professionals providing care to resettled refugees and asylees. In October, 2014, NLM’s Specialized Information Services (SIS) broadened the scope of RHIN by rebranding it HealthReach. This was done to better meet the needs of the diverse non-English and English as a second language speaking audiences. HealthReach continues to recognize the importance of providing refugee and asylee specific information while expanding the information provided to meet the needs of most immigrant populations. Over the next several months new resources will be added to the web site. There is also a new Twitter feed, @NLM_HealthReach. There isn’t much change between the old RHIN and the new HealthReach; this was intentional to help with the continuity of service through the transition.
A recent AEA365 Evaluation Tip-a-Day featured a review and several hot tips for Padlet, a freely available web-based bulletin board system. The hot tips include the use of Padlet as an anonymous brainstorming activity in response to a question or idea, and as a backchannel for students or conference attendees to share resources and raise questions for future discussion. Padlet’s bulletin board configuration settings are intuitive and easy to use with various backgrounds and freeform, tabular, or grid note arrangement display on the bulletin board. Free Padlet accounts can be created by either signing up directly or by linking to an existing Google or Facebook account. Padlet includes many privacy options that are clearly explained, including “Private” mode, requiring the use of a password for you and those you invite to participate to access the Padlet, and “Public” mode to view, write or moderate. A new update feature includes a variety of ways to share Padlet data, ranging from choosing the icon for six different social media channels to downloading data as a PDF or Excel/CSV file for analysis. For a trial run of this resource, visit the NN/LM Outreach Evaluation Resource Center’s Padlet about the OERC Evaluation Series booklets and leave your input! Posts will be moderated on the Padlet before they display publicly.
The Centers for Disease Control and Prevention (CDC) has a freely available ‘how to’ resource Introduction to Program Evaluation for Public Health Programs: A Self Study Guide. Examples of public and community health programs that can be considered for program evaluation include direct service interventions, community-based mobilization efforts, research initiatives into issues such as health disparities, advocacy work, and training programs. The guide is available online or as a PDF document that consists of a six-step process (from Engaging Stakeholders to Ensure Use of Evaluation Findings), a helpful Glossary of program evaluation terminology, and Resources for additional publications, toolkits, and more to support public and community health program evaluation work. A related CDC guide A Framework for Program Evaluation is one of several resources featured in the Evaluation Planning section of the NN/LM OERC Tools and Resources for Evaluation web page.
An innovative and compelling approach to creating qualitative data visualizations with illustrations is provided by Fresh Spectrum. The process begins by taking a long narrative such as a focus group transcription, and chunking it into a few paragraphs per concept with a unique illustration for each one. One option is to use your organization’s existing images or Creative Commons-licensed images for illustrating concepts. The next step for the visualization uses the images with brief captions as an online data dashboard, where visitors can click on the captioned image of interest to them to access the more detailed narrative. One example describes how to do this within a WordPress portfolio blog template, or a simpler strategy of creating HTML anchor links to each individual section within a longer text, which then leads to the longer narrative.
The NN/LM Outreach Evaluation Resource Center (OERC) has more resources available from the Reporting and Visualizing tab of the OERC Tools and Resources for Evaluation Guide.