Big Data

Big data refers to data that are so large and complex that traditional methods of collection and analysis are not possible. The amount and variety of big data has increased exponentially over the past decade. 'Data exhaust' is one source, which relates to data produced passively as a byproduct of user interactions with a system, such use of a mobile phone service or Internet banking. Online information is another, giving an indication of human intent, emotions, and wishes by collecting the information contact in web content such as news and social media interactions (e.g., Facebook, blogs, twitter), and online search history. Data can also come from physical sensors such as satellite images and traffic information (UNDP 2013).

One trend in big data is the promotion of open data (or open knowledge). This entails the free-release of de-identified data sets to the public. This allows underused data sets (for example, government data sets) that contain valuable data to be accessed by people who can turn this raw data into something useable and useful. For example, VicRoads has released a number of data sets about crashes and road networks, which will be useful for people who are wanting to evaluate traffic interventions. The Open Knowledge website has more information about this trend.

Resources

BetterEvaluation Blog Posts

Big Data and Evaluation - Use and Implications - Alice Macfarlan takes a look at some of the ways that Big Data is being used in evaluation, and raises some questions about the implications of this emerging form of information.

Websites

  • Global Pulse - Global Pulse is an innovation initiative launched by the Executive Office of the United Nations Secretary-General, which conceives and co-ordinates research on big data for development through a network of innovation labs.

  • Open Knowledge Foundation Australia - A site that shares knowledge about open data sets.

Papers

Videos 

  • Introduction to Big Data - by Hilary Mason, Chief Data Scientist at Bitly

  • ​Big Data Intro by Global Pulse - Andreas Weigend, former Chief Data Scientist for Amazon.com and Peter Hirshberg, former head of Enterprise Marketing for Apple, Inc., offer a range of examples and visualisations of how Real-Time, or big data, can augment and improve decision making in the 21st century

​Examples

Overview

  • Big Data, Big Ruse - This article from Stephen Few of Perceptual Edge provides a critique of many of the arguments used to promote big data - in particular concerns that without adequate analysis and visualisation, big data will just increase the problems of monitoring.

References

UNDP (2013) Discussion Paper: Innovations in Monitoring and Evaluation. Retrieved from http://www.outcomemapping.ca/download/UNDP%20Discussion%20Paper%20Innovations%20in%20Monitoring%20and%20Evaluation.pdf 

UN Global Pulse (2012). Big Data for Development: Challenges & Opportunities [White Paper]. Retrieved from http://www.unglobalpulse.org/sites/default/files/BigDataforDevelopment-UNGlobalPulseJune2012.pdf

Updated: 17th November 2016 - 4:48am
This Option is useful for:
A special thanks to this page's contributors
Author
BetterEvaluation Website and Engagement Coordinator, BetterEvaluation and ANZSOG.
Melbourne, Australia.

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.