Big data for development: challenges & opportunities

This white paper by UN Global Pulse examines the use of Big Data in development contexts. Using a number of examples, it highlights how this data type can be leveraged to provide early warnings of disruptions and crises, and can give real-time awareness and feedback of situations and interventions. It also delves into a conversation about the implications of Big Data use.

Extract

"Innovations in technology and greater affordability of digital devices have presided over today’s Age of Big Data, an umbrella term for the explosion in the quantity and diversity of high frequency digital data. These data hold the potential—as yet largely untapped—to allow decision makers to track development progress, improve social protection, and understand where existing policies and programmes require adjustment. 

Turning Big Data—call logs, mobile-banking transactions, online user-generated content such as blog posts and Tweets, online searches, satellite images, etc.—into actionable information requires using computational  techniques to unveil trends and patterns within and between these extremely large socioeconomic datasets. New insights gleaned from such data mining should complement official statistics, survey data, and information 
generated by Early Warning Systems, adding depth and nuances on human behaviours and experiences—and doing so in real time, thereby narrowing both information and time gaps. 

With the promise come questions about the analytical value and thus policy relevance of this data—including concerns over the relevance of the data in developing country contexts, its representativeness, its reliability—as well as the overarching privacy issues of utilising personal data. This paper does not offer a grand theory of technology-driven social change in the Big Data era. Rather it aims to delineate the main concerns and 
challenges raised by “Big Data for Development” as concretely and openly as possible, and to suggest ways to address at least a few aspects of each. 

It is important to recognise that Big Data and real-time analytics are no modern panacea for age-old development challenges. That said, the diffusion of data science to the realm of international development nevertheless  constitutes a genuine opportunity to bring powerful new tools to the fight against poverty, hunger and disease."

Contents

• INTRODUCTION    6 
• SECTION 1: OPPORTUNITY    8 
  • 1.1. DATA INTENT AND CAPACITY    8 
    • The Data Revolution    8 
    • Relevance to the Developing World    9 
    • Intent in an Age of Growing Volatility    11 
    • Big Data for Development: Getting Started    13 
    • Capacity: Big Data Analytics    17 
  • 1.2 SOCIAL SCIENCE AND POLICY APPLICATIONS    19 
    • A Growing Body of Evidence    20 
• SECTION II: CHALLENGES    24 
  • 2.1 DATA    24 
    • Privacy    24 
    • Access and Sharing    25 
  • 2.2 ANALYSIS    26 
    • Getting the Picture Right    27 
    • Interpreting Data    29 
    • Defining and Detecting Anomalies in Human Ecosystems    33 
• SECTION III: APPLICATION    35 
  • 3.1 WHAT NEW DATA STREAMS BRING TO THE TABLE    35 
    • Know Your Data    35 
    • Applications of Big Data for Development    36 
  • 3.2. MAKING BIG DATA WORK FOR DEVELOPMENT    39 
    • Contextualisation is Key    39 
    • Becoming Sophisticated Users of Information    40 
• CONCLUDING REMARKS ON THE FUTURE OF BIG DATA FOR DEVELOPMENT    42