This episode from the IEG podcast 'What have we learned' features Brenda Barbour and Jos Vaessen discussing the opportunities and challenges of using innovative technologies with evaluation data.
This resource and the following information was contributed by Simon Davies.
Authors and their affiliation
Brenda Barbour, Independent Evaluation Group, World Bank
Jos Vaessen, Independent Evaluation Group, World Bank
Year of publication
Type of resource
The conversation begins with a discussion about how an increase in available data and computational capacity has created opportunities to generate actionable and relevant evaluative insights. It is recommended, however, that evaluators avoid the temptation to use a data-driven approach, and instead start with honing good evaluation questions rather than looking at the data that is available.
The idea of 'boutique innovations' is introduced. These are innovations using specific innovative methods to answer specific questions. An example provided is the use of sentiment analysis to understand the perceptions of particular stakeholders or groups, using publicly data from platforms such as Twitter or more conventional media platforms.
Advantages of using technologies such as artificial intelligence were presented, including; efficiency in analysing large data sets; accuracy of analysis due to the potential to extract data more consistently from large datasets; and increase in the breadth of analysis, for example by accessing large sets of images to extract and classify data.
The effect of COVID-19 was discussed, in particular the challenge of not being able to collect data in person. Remote interviews were possible but have both advantages and disadvantages. Additionally, desk reviews could be carried out more smartly, for example using text analytics and machine learning for content analysis.
The conversation concludes by suggesting that data innovations serve as complimentary tools for evaluative analysis, alongside strong evaluation processes.
Who is this resource useful for?
- Advocates for evaluation;
- Commissioners/managers of evaluation;
- Evaluation users;
- Those involved in evaluation capacity strengthening;
How have you used or intend on using this resource?
This podcast is useful for learning about the different ways innovative technology can be applied to evaluation, including as solutions to emerging challenges, such as COVID-19. It is also helpful for reinforcing the importance of evaluative thinking and processes over simply diving straight into using new technologies.
Why would you recommend it to other people?
This brief and well articulated conversation provides the listener with an overview on how to begin thinking about the use of innovative tech with evaluation data. The podcast interview format creates an easy entry point for busy professionals. The conversation could be helpful to those making a start with innovative data technologies, or to help spark a conversation with colleagues on the topic.