Big data and evaluation: Use and implications

Blurry and stylized image of a large crowd

Big data is emerging as a new world currency. This form of digital data, generated almost automatically by the online interactions of people and products and services, creates a wealth of constantly updating information that can be used to support decision-making and aid monitoring and evaluation.

This week we take a look at some of the ways that Big Data is being used in evaluation, and raise some questions about the implications of this emerging form of information.

What is big data?

Big data refers to data that are so large and complex that traditional methods of collection and analysis are not possible. The amount and variety of big data has increased exponentially over the past decade. It can come from sources such as mobile phone services or internet banking, web content such as news, social media interactions, and search history. It can also be produced by physical sensors such as satellite images and traffic information (See UNDP 2013).

You can watch a video by Global Pulse on the power of big data, in which Andreas Weigend, former Chief Data Scientist for Amazon.com and Peter Hirshberg, former head of Enterprise Marketing for Apple, Inc., offer a range of examples and visualisations of how real-time, or big data, can augment and improve decision making in the 21st century:

 

How is big data being used in monitoring and evaluation?

In the field of evaluation, big data has the potential to complement traditional data sources, such as survey data or official statistics by adding unique and up-to-date data that can be used to form a complete and present picture of a situation.

For example, a project monitoring traffic incidents in an area of Victoria (Australia) might turn to VicRoads open data sets to extract information about the whereabouts of traffic incidents. In another example, the Sustainable Water, Energy and Environmental Technology Laboratory (the SWEETLAB) develops and implements technologies to improve data collection in development areas. Their use of remotely reporting electronic sensors for assessing use of water filters and cookstoves in Rwanda, for example, can collect objective electronic data on the use of these devices to help monitor the sustainability of this interventions.

Other projects track real-time events based on the analysis of online content. This is valuable in providing baselines of current situations, and updating this with snapshots over time to see how situations change, which can be used in the design and evaluation of programmes and interventions. One example of this, the Global Database of Events, Language and Tone (GDELT), monitors the world's news media in print, broadcast, and web format to report on how the world is "feeling" and what "events" (300 categories of physical activities such as riots, protests, and diplomacy actions)are happening.

The UN Global Pulse runs a number of projects that utilise social media data to monitor social and environmental issues. For example, one project analyses social media conversations to understand public perceptions of sanitation, providing a baseline of how public discourse of sanitation changed over time so as to allow monitoring of the effectiveness and reach future health campaigns. Similarly, other Global Pulse projects use Twitter to measure global engagement on climate change and to evaluate the impact of the Every Women Every Child movement since its 2010 launch, by monitoring changes in related public health dialogue in Twitter conversations.

Emerging issues in the management and regulation of big data

As the use of big data is growing, in both M&E and other areas, new ways of looking at the patterns and rules of digital content are emerging. For example, the open data movement (e.g. Creative Commons, Linked Open Data projects, Government Linked Open Data, Open Science, Free Access to Law), among other trends take large data sets of underused data (government data sets, for example) and open them up to the public for reuse. These have great potential to be picked up by individuals or organisations who can use this data to effect change, supported by the underlying idea that the use of big data leads to better decision making. But there is also the lingering question of the regulation of big data, particularly as technologies advance to the point that the collection, structuring and analysis of data occurs almost automatically.

Global Pulse’s White Paper, Big Data for Development: Opportunities and Challenges (2012), brings up the issue of privacy when dealing with big data, noting that “in many cases, the primary producers— i.e. the users of services and devices generating data—are unaware that they are doing so, and/or what it can be used for” (p.24). For example, while users of social media sites might have 'consented' to their data being used in a myriad of ways, they may not be fully aware of what this entails.

This article by the Guardian voices some of the concerns that the use of big data can raise for the data producers, particularly when this data is used by corporations, but also in cases of misplaced good intentions, such as the Samaritans Twitter app that monitored user tweets for signs of depression before being discontinued after public backlash. What these concerns point to is a rising awareness that the current protocols around the use of big data are likely to undergo a process of examination. In terms of using big data for development and evaluation purposes, it's important for initiatives that draw on big data to be aware of the issues of privacy and careful handling of data, and to make this part of an ongoing discussion about how best to manage data rights.

In the broader context, these discussions about regulations take on a new level of concern when you consider the combined amount of individual-level information that social media companies, Google, and mobile and internet companies have (Global Pulse, 2012). The question of how to regulate a world that is informational, rather than local or global, is one that Pompeu Casanovas (Director of Advanced Research and founder of the Institute of Law and Technology, Universitad Autónoma de Barcelona) will ask in his upcoming seminar series, A New Rule of Law, as he discusses the idea that democracy on the web is emerging as a new important world issue akin to the Rule of Law of last century. As Global Pulse notes, "Because privacy is a pillar of democracy, we must remain alert to the possibility that it might be compromised by the rise of new technologies, and put in place all necessary safeguards" (2012, p.24).

It’s clear that big data is proving to be a useful tool for social good in the  monitoring and evaluation. However, the ongoing debate and discussion about how to devise the best protocols and principles for dealing with big data will be an interesting and important one to watch and engage in.

You can also find out more about UN Global Pulse and its projects here.

UNDP (2013) Discussion Paper: Innovations in Monitoring and Evaluation. Retrieved from http://www.outcomemapping.ca/download/UNDP%20Discussion%20Paper%20Innovations%20in%20Monitoring%20and%20Evaluation.pdf

UN Global Pulse (2012). Big Data for Development: Challenges & Opportunities [White Paper]. Retrieved from http://www.unglobalpulse.org/sites/default/files/BigDataforDevelopment-UNGlobalPulseJune2012.pdf

'Big data and evaluation: Use and implications' is referenced in: