Big Data and Evaluation - Use and Implications

Alice Macfarlan's picture 12th May 2015 by Alice Macfarlan

Big data is emerging as a new world currency. This form of digital data, generated almost automatically by the online interactions of people and products and services, creates a wealth of constantly updating information that can be used to support decision-making and aid monitoring and evaluation. This week we take a look at some of the ways that Big Data is being used in evaluation, and raise some questions about the implications of this emerging form of information - questions that will be explored further in a public seminar at RMIT University next week.

What is big data?

Big data refers to data that are so large and complex that traditional methods of collection and analysis are not possible. The amount and variety of big data has increased exponentially over the past decade. It can come from sources such as mobile phone services or Internet banking, web content such as news, social media interactions, and search history. It can also be produced by physical sensors such as satellite images and traffic information (See UNDP 2013).

You can watch a video by Global Pulse on the power of big data here, in which Andreas Weigend, former Chief Data Scientist for Amazon.com and Peter Hirshberg, former head of Enterprise Marketing for Apple, Inc., offer a range of examples and visualisations of how Real-Time, or big data, can augment and improve decision making in the 21st century:

How is big data being used in monitoring and evaluation?

In the field of evaluation, big data has the potential to complement traditional data sources, such as survey data or official statistics by adding unique and up-to-date data that can be used to form a complete and present picture of a situation. 

For example, a project monitoring traffic incidents in an area of Victoria (Australia) might turn to VicRoads open data sets to extract information about the whereabouts of traffic incidents. In another example, the Sustainable Water, Energy and Environmental Technology Laboratory (the SWEETLAB) develops and implements technologies to improve data collection in development areas. Their use of remotely reporting electronic sensors for assessing use of water filters and cookstoves in Rwanda, for example, can collect objective electronic data on the use of these devices to help monitor the sustainability of this interventions.

Other projects track real-time events based on the analysis of online content. This is valuable in providing baselines of current situations, and updating this with snapshots over time to see how situations change, which can be used in the design and evaluation of programmes and interventions. One example of this, the Global Database of Events, Language and Tone (GDELT), monitors the world's news media in print, broadcast, and web format to report on how the world is "feeling" and what "events" (300 categories of physical activities such as riots, protests, and diplomacy actions) are happening.

The UN Global Pulse runs a number of projects that utilise social media data to monitor social and environmental issues. For example, one project analyses social media conversations to understand public perceptions of sanitation, providing a baseline of how public discourse of sanitation changed over time so as to allow monitoring of the effectiveness and reach future health campaigns. Similarly, other Global Pulse projects use Twitter to measure global engagement on climate change and to evaluate the impact of the Every Women Every Child movement since its 2010 launch, by monitoring changes in related public health dialogue in Twitter conversations.

Emerging issues in the management and regulation of big data

As the use of big data is growing, in both M&E and other areas, new ways of looking at the patterns and rules of digital content are emerging.  For example, the open data movement (e.g. Creative Commons, Linked Open Data projects, Government Linked Open Data, Open Science, Free Access to Law), among other trends take large data sets of underused data (government data sets, for example) and open them up to the public for reuse. These have great potential to be picked up by individuals or organisations who can use this data to effect change, supported by the underlying idea that the use of big data leads to better decision making. But there is also the lingering question of the regulation of big data, particularly as technologies advance to the point that the collection, structuring and analysis of data occurs almost automatically. 

Global Pulse’s White Paper, Big Data for Development: Opportunities and Challenges (2012), brings up the issue of privacy when dealing with big data, noting that “in many cases, the primary producers— i.e. the users of services and devices generating data—are unaware that they are doing so, and/or what it can be used for” (p.24). For example, while users of social media sites might have 'consented' to their data being used in a myriad of ways, they may not be fully aware of what this entails. 

This article by the Guardian voices some of the concerns that the use of big data can raise for the data producers, particularly when this data is used by corporations, but also in cases of misplaced good intentions, such as the Samaritans Twitter app that monitored user tweets for signs of depression before being discontinued after public backlash. What these concerns point to is a rising awareness that the current protocols around the use of big data are likely to undergo a process of examination. In terms of using big data for development and evaluation purposes, it's important for initiatives that draw on big data to be aware of the issues of privacy and careful handling of data, and to make this part of an ongoing discussion about how best to manage data rights. 

In the broader context, these discussions about regulations take on a new level of concern when you consider the combined amount of individual-level information that social media companies, Google, and mobile and internet companies have (Global Pulse, 2012). The question of how to regulate a world that is informational, rather than local or global, is one that Pompeu Casanovas  (Director of Advanced Research and founder of the Institute of Law and Technology, Universitad Autónoma de Barcelona) will ask in his upcoming seminar series, A New Rule of Law, as he discusses the idea that democracy on the web is emerging as a new important world issue akin to the Rule of Law of last century. As Global Pulse notes, "Because privacy is a pillar of democracy, we must remain alert to the possibility that it might be compromised by the rise of new technologies, and put in place all necessary safeguards" (2012, p.24).

It’s clear that big data is proving to be a useful tool for social good in the  monitoring and evaluation. However, the ongoing debate and discussion about how to devise the best protocols and principles for dealing with big data will be an interesting and important one to watch and engage in.

Find out more

You can hear Pompeu Casanovas discuss these issues more at the seminar 'A New Rule of Law' (The Seminar Series) at RMIT University, Australia. He will be joined by Timos Sellis (RMIT), Guido Governatori (NICTA), Margaret Jackson (RMIT), and Philip Chung (AustLII). 

You can also find out more about UN Global Pulse and its projects here.

References

UNDP (2013) Discussion Paper: Innovations in Monitoring and Evaluation. Retrieved from http://www.outcomemapping.ca/download/UNDP%20Discussion%20Paper%20Innovations%20in%20Monitoring%20and%20Evaluation.pdf 

UN Global Pulse (2012). Big Data for Development: Challenges & Opportunities [White Paper]. Retrieved from http://www.unglobalpulse.org/sites/default/files/BigDataforDevelopment-UNGlobalPulseJune2012.pdf

A special thanks to this page's contributors
Author
BetterEvaluation Website and Engagement Coordinator, BetterEvaluation and ANZSOG.
Melbourne, Australia.

Comments

Simon Hearn's picture
Simon Hearn

Thanks for this blog. It provides a nice overview. One question I have about big-data and evaluation is what kind of effect is it having on evaluation practice?

Is it improving conventional evaluation practice, and are there cases of where this is happening? I noted the link to the evaluation of improved cook stoves in Rwanda but I'm not convinced this counts as big data as described above.

Or is it enabling a new kind of evaluation practice to emerge - citizen led, collective evaluation? E.g. the use of apps like Ushahidi.

On another tack, I found the following blogs from the Open Knowledge Foundation quite useful in reminding us of the power of small data: 

Readers may also be interested in the following papers:

 

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.