Big data refers to data that are so large and complex that traditional methods of collection and analysis are not possible.
The amount and variety of big data has increased exponentially over the past decade. 'Data exhaust' is one source, which relates to data produced passively as a byproduct of user interactions with a system, such use of mobile phone services or internet banking. Online information is another, giving an indication of human intent, emotions, and wishes by collecting the information contact in web content such as news and social media interactions (e.g., Facebook, blogs, Twitter), and online search history. Data can also come from physical sensors such as satellite images and traffic information (UNDP 2013).
One trend in big data is the promotion of open data (or open knowledge). This entails the free release of de-identified data sets to the public. This allows underused data sets (for example, government data sets) that contain valuable data to be accessed by people who can turn this raw data into something useable and useful. For example, VicRoads has released a number of data sets about crashes and road networks, which will be useful for people who are wanting to evaluate traffic interventions. The Open Knowledge website has more information about this trend.
The UN Global Pulse runs a number of projects that utilise social media data to monitor social and environmental issues. For example, one project analyses social media conversations to understand public perceptions of sanitation, providing a baseline of how public discourse of sanitation changed over time so as to allow monitoring of the effectiveness and reach future health campaigns. Similarly, other Global Pulse projects use Twitter to measure global engagement on climate change and to evaluate the impact of the Every Women Every Child movement since its 2010 launch, by monitoring changes in related public health dialogue in Twitter conversations.
BetterEvaluation Blog Post
UNDP (2013) Discussion Paper: Innovations in Monitoring and Evaluation. Retrieved from http://www.outcomemapping.ca/download/UNDP%20Discussion%20Paper%20Innovations%20in%20Monitoring%20and%20Evaluation.pdf
UN Global Pulse (2012). Big Data for Development: Challenges & Opportunities [White Paper]. Retrieved from http://www.unglobalpulse.org/sites/default/files/BigDataforDevelopment-UNGlobalPulseJune2012.pdf
'Big data' is referenced in:
- Rainbow Framework :