5 Steps to an Evaluation
Step 5: Collect Data, Analyze, and Act
The fifth step in designing your evaluation is to implement the evaluation - Collect data, Analyze, and Act! This is the time to reflect upon what you have learned, gather insights, and inform programming improvements. As part of an evaluation, you should:
- Collect data before, during, and after your program has completed
- Complete analysis after the completion of data collection
- Act upon your analysis by sharing evaluation results with stakeholders, and if needed, adapt future iterations of your program to address gaps identified through the evaluation
- Ensure the privacy and confidentiality of all participants.
- Privacy - No participant should ever feel or be forced to reveal information to the evaluator that the participant does not wish to reveal.
- Confidentiality - Personal information about the participant that has been revealed to the evaluator should not be directly linked to the individual in the dataset or results shared in a way that identifies the participant.
- Select a sampling strategy:
- Is your target population small enough that all participants will be included in data collection?
- Or do you need to sample participants? If so, there are many considerations to determine a sampling strategy.
- More information on sampling strategies
- While collecting data, be sure to:
- Gather informed consent of respondents
- Ensure the privacy and confidentiality of respondents and their data
- Carry out data analysis using appropriate quantitative or qualitative approaches
- Quantitative data are information gathered in numeric form.
- Analysis of quantitative data requires statistical methods.
- Results are typically summarized in graphs, tables, or charts.
- More information on quantitative analysis.
Quantitative analyses Description Frequencies Describes how many times something has occurred within a given interval, such as a particular category or period of time.
For example, the number of training participants who are classroom teachers is a frequency.
PercentageThe given number of units divided by the total number of units and multiplied by 100. Percentages are a good way to compare two different groups or time periods.
For example, if 50 of 100 training participants are library staff, 50% of training participants are library staff.
Ratio The numerical relationship between two groups.
For example, the ratio of the number of LGBTQIA+ participants at an event (25) to the number of total participants (300) would be 25/300, or 1:12.
Mean, Median, Mode Three measures of the most typical values in your dataset (also called measures of central tendency). A mean, or average, is determined by summing all the values and dividing by the total number of units in the sample. A median is the 50th percentile point, with half of the values above the median and half of the values below the median. A mode is the category or value that occurs most frequently within a dataset.
For example, if a list of post-test scores are 65%, 70%, 85%, 90%, 90%, the mean is 80% (400/5), the median is 85%, and the mode is 90%.
- Qualitative data are information gathered in non-numeric form, usually in text or narrative form.
- Analysis of qualitative data relies heavily on interpretation.
- Qualitative data analysis can often answer the 'why' or 'how' of evaluation questions.
- More information on qualitative analysis.
Steps to Analyzing Qualitative Data Review your data Before beginning any analysis, it is important that you understand the data you have collected by reviewing them several times. For example, if your data consist of interview transcripts, read and re-read the transcripts until you have a general understanding of the content. As you are reviewing, write notes of your first impressions of the data; these initial responses may be useful later as you interpret your data. Organize your data Qualitative data sets tend to be very lengthy and complex. Once you have reviewed your data and are familiar with what you have, organize your data so that they are more manageable and easy to navigate. This can save you time and energy later. Depending on the evaluation question(s) you want to answer, there are a variety of ways to group your data, including by date, by data collection type (such as focus group vs. interview), or by question asked. Code your data Coding is the process of identifying and labeling themes within your data that correspond with the evaluation questions you want to answer. Themes are common trends or ideas that appear repeatedly throughout the data. You may have to read through your data several times before you identify all of the themes within them. Interpret your data Interpretation involves attaching meaning and significance to your data. Start by making a list of key themes. Revisit your review notes to factor in your initial responses to the data.
- Share results with stakeholders
- Sharing information gathered in your evaluation with stakeholders will ensure that they understand your program successes and challenges
- Use creative data communication strategies and techniques to effectively present results and engage stakeholders in discussions
- Act upon those results to ensure program adaptation and improvement to address any identified gaps or challenges
- More information on data visualization and communication strategies