Now that you’ve held your exercise, it’s time to evaluate it and reflect on the experience; identify, prioritize, and plan next steps; and summarize and share findings.
Click on a step to learn more.
Finish the evaluation
Start by reviewing your evaluation plan to remind yourself what you hoped to learn. Next, prepare, analyze, and interpret your data so that you are able use your results.
Tasks for finishing the evaluation
Click on a task to learn more
Prepare
The data that you collected likely consists of completed paper surveys or questionnaires, observation notes, or audio recordings of interviews or focus groups. Typically, data in this format is not very useful and must be entered and organized before you can begin any type of analysis. Whether you’ve collected quantitative or qualitative data, you’ll need to enter it in a logical format that you can analyze. If you used an online survey tool, such as Google Forms, you can typically export responses to a CSV, Excel, or PDF file. Steps include:
- Clean up the data you have collected
- Enter quantitative data into Excel or a similar spreadsheet
- Type qualitative data into Word or a similar word processing program
- Check the data for errors (i.e., duplicates, irrelevant responses, grammar errors, etc.) and ensure that participant’s words are attributed correctly if interviews or focus groups are used.
Example data preparation
Analyze
Quantitative analysis uses numerical or statistical approaches to help you understand, illustrate, and summarize data. Most exercise evaluations will use descriptive statistics, which can easily be calculated in an Excel spreadsheet or similar software, to help identify main features of the data. Examples include:
- Frequency count or measure of the number of times that a response occurs
- Percentage of people who gave a certain response
- Mean or average response
- Range of responses
Example qualitative data analysis from the North Platte River Basin Drought THIRA Workshop
Question: How helpful was today’s workshop in advancing your organization’s planning for drought?
|
|
Response
|
Not helpful at all |
Slightly |
Moderately |
Very |
Extremely helpful |
Total |
|
Response value
|
(1) |
(2) |
(3) |
(4) |
(5) |
|
|
Frequencies
|
1 |
3 |
6 |
14 |
1 |
25 |
|
Percent
|
4% |
12% |
24%
|
56% |
4% |
25 |
| Range |
Lowest score = 1, Highest score = 5 |
| Mean |
3.44 |
Qualitative analysis on the other hand, includes things like finding patterns and themes in the written survey questions that people answered or in the stories that people shared during interviews. Other qualitative data can include field notes written by a researcher and focus groups. Your evaluation plan will help you decide which approach you should use. Approaches that may be used for qualitative analysis include:
- Code or organize written survey responses into categories and count the number of responses in each
- Review notes or transcripts to see what themes emerge and explore how different people responded to each of the themes
- Depending on the amount of data and your available software, you can analyze qualitative data by highlighting themes on paper copies of responses, creating tables in Word or Excel, or with another qualitative coding software
Example qualitative data analysis from the North Platte River Basin Drought THIRA Workshop
Comments on the surveys and by interview participants related to “helpfulness of the workshop for drought planning”
|
|
Being able to hear from other agencies and people working at different levels (state, local), including learning what others can do, and seeing other perspectives
|
|
Increase awareness of drought impacts
|
|
The potential for having consistency and coordination across organizations and geographic location
|
|
Readiness includes knowledge of who/what organization to contact for what purpose
|
Interpret
Once you’ve analyzed your data, think critically about it and draw out key findings. Refer back to the questions that you outlined in your evaluation plan to help you determine what your data mean. Here are some things to consider:
- Did any patterns or themes emerge that help you tell a story about your exercise?
- Why were some outcomes achieved, and others not achieved?
- Are outcomes different for different groups of people?
- What has surprised you about the data?
- What do you need to find out more about?
- How do your results compare with data collected during previous exercises?
Example interpretation of evaluation findings from the North Platte River Basin Drought THIRA Workshop
Goals of the workshop: Assess whether the workshop would enhance planning for droughts by increasing familiarity with drought impacts and building trust and cooperation between groups
|
|
Self-reported familiarity with the impacts of drought increased before and after the workshop and continued through the follow-up assessment approximately nine months later
|
|
Participants perceived the primary importance of the workshop related to increasing their capacity to coordinate and collaborate with other stakeholders. The most frequently mentioned reason for helpfulness was that it brought together and helped them to learn about other organizations whose capacities and resources could be important for coordinating to deal with an extended drought. Immediately after the workshop, more than 80% of persons thought the workshop increased the likelihood of involving other organizations in their planning processes at least “slightly.”
|
Identify lessons learned
The lessons learned are the knowledge and data you gain from conducting a project, or in this case a drought scenario-based exercise. They’re the tips and guidance that you learn from your experience and that of others. In fact, this website is built upon the lessons learned by the National Drought Mitigation Center and other organizations in holding preparedness exercises. Lessons may be positive, such as the importance of setting clear expectations, or negative such as packing too much into the agenda.
Take time to reflect on your exercise and identify what went wrong and why so that you (and others) don’t repeat mistakes and can take advantage of best practices. One way to do this is to hold a meeting with the development team and key stakeholders within two weeks of conducting the exercise to discuss what worked and what didn’t. Meeting as a group as soon as possible after the exercise is important so that the event is still fresh in their minds and you can get the most out of the meeting.
Tips for capturing lessons learned
Use the Impacts Worksheet #1: Identify Lessons Learned (pdf) to help guide your meeting and doducment your results.
Plan next steps
Suggested next steps, whether they come from the exercise itself or are suggested by the planning team, are essential in ensuring that you keep the momentum going after the exercise concludes. At the exercise, participants meet new people, talk with others that have similar interests, get energized, and are ready to implement what they’ve learned. Once they return home, they go back to their normal day-to-day responsibilities and that newly gained momentum is quickly lost. Identifying actionable next steps can help you, the development team, and key stakeholders to:
- Find practical ways to reach your drought preparedness goals
- Focus on the details that you need to do in order to succeed
- Think about the best strategy for the future
- Save on time, energy, and resources
Tips for developing next steps
Use the Impacts Worksheet #2: Plan Next Steps (pdf) to help you identify concrete, comprehensive action steps.
Summarize and share findings
If you have evaluated your exercise, identified lessons learned, and planned your next steps, you will want to make sure that these findings get shared with stakeholders and/or funders. As part of your evaluation plan, you identified who should learn about your findings, the kind of information they need, how they might use it, and formats that would best meet these needs. Take time to review your plan and make sure that you communicate your findings in a way that encourages people to read them and take action. While you may need a formal document for funders, avoid a long, text-heavy document for key stakeholders, exercise participants, and the general public. Think about your audience and tailor the content to their needs.
Formats for sharing findings
Example methods for sharing findings