Using technologies for monitoring and evaluation in insecure settings
Operating in insecure environments is one of the more critical tests for humanitarian, development, and peacebuilding organizations alike. Access constraints or even direct attacks make monitoring and evaluation extremely challenging.
Technologies like mobile phones, radios, Internet platforms and GPS trackers promise new solutions for collecting vital data or tracking implementation of projects.
In some of the most insecure humanitarian contexts, monitoring aid programmes are particularly important to humanitarian organisations. Did assistance get to the right people? Have aid trucks safely reached their intended location? How do affected people assess the services provided? Amidst access constraints, attacks and the risk of aid diversion, those questions remain difficult to answer.
Access restrictions, high costs, poor infrastructure and high levels of uncertainty require tools that can function without constant electricity supply, across large distances and without advanced computing skills. We identified six technology types that meet these criteria: handheld devices for digital data collection, mobile phone-based feedback mechanisms, remote sensing with satellites or delivery tracking, communication with online platforms and broadcasting with radios and other forms of media. M&E practitioners are increasingly optimistic about the use of these technologies when working in insecure environments.
Examples of technologies used in insecure environments
An international NGO runs a large, long-standing operation throughout Afghanistan, which involves multiple and time-intensive surveys every month. Transporting paper questionnaires from remote areas to local hubs often took days. Entering the responses from the paper questionnaire into the database, aggregation and analysis would take many hours. In 2013, the NGO bought 20 tablet computers for about 300 USD a piece. It set up these tablets with open-source software. the investment necessary to roll out digital data collection paid for itself after just one survey.
IDPs in Iraq had limited access to reliable news from local media and information about available aid services. As one man in Dahuk noted: ‘We don’t know the organisations or their names; we have never spoken with them. We don’t know anything about their work’. In July 2015, an inter-agency group of UN agencies and NGOs launched a nationwide toll-free hotline. The Erbil-based call centre is run by two coordinators and four Iraqi operators, three of whom are female. The operators collect information from agencies every week in order to answer straightforward questions from callers. With more complex queries, the call centre’s coordinators contact the relevant agency or cluster for an answer and get back to the caller within three days.
Strict access constraints to Syria and dynamic changes on the ground make it very difficult for humanitarian responders to have a clear picture of the situational needs. The UN body for space imagery, UNOSAT, thus aimed to contribute to context understanding using satellite technology. Satellite imagery and analysis were used to assess conflict developments and their impact. The same tools could be used to monitor humanitarian programming in similarly inaccessible situations.
(For more information on these examples, see the SAVE Toolkit: Technologies for monitoring in insecure environments)
Types of technologies
Several technologies can be used in insecure settings to collect and communicate data from aid programs:
The rising usage of basic mobile phones in volatile environments can make communicating with communities easier. Many organisations are expanding their outreach and feedback systems into accepting messages via SMS, calls or interactive voice response, in which pre-recorded messages are used to provide or gather information.
‘Handhelds’ such as smartphones, tablets or more basic digital devices are becoming cheaper and easier to use. Many aid organisations have found it convenient and cost-effective to use digital devices for data collection instead of traditional paper-based surveys. Aid actors can now tap into a substantial market for digital form creation and data collection applications. Digital data entry linked to electronic databases also facilitates automatic data analysis and visualisation.
Where physical access is constrained, there are a number of aerial technologies to gather information about objects or areas without having a physical presence. This is also referred to as ‘remote sensing’ and can reveal context conditions and other observable changes such as construction, agricultural developments or population movement. Additional remote sensing tools, like radars and infrared, can further augment this. Similarly, devices linked to ‘navigation satellites’ make it possible to identify and trace the location of deliveries or staff and to visualise information on maps.
Radio remains the most popular technology that people use to receive news and updates in many resource-constrained contexts. In many insecure environments, it is the most reliable way to reach communities. It is often understood as a one-way communication tool, but radio programming can also be used for active engagement, involving or supporting communities in creating their own shows.
Finally, where online communication platforms are popular, aid agencies can use them for their monitoring, feedback and accountability efforts. This includes social media networks such as Facebook and Twitter, as well as instant messaging applications like WhatsApp. These tools make it possible to transmit information and messages via online connections – often free of charge.
Take your time
Practitioners are well advised not to rush implementation. Consulting with partners who already use the technology, and developing a thorough understanding of who influences and spreads information in the specific context, is critical for success. Also, factor in time to win the support and trust from colleagues and affected people for the solution, and work with users when inventing, designing and testing tools. Challenges can stem from inherent biases of some technologies – for example, men are more likely to own the household phone, making it difficult for women to use phones anonymously. Concerns can also arise from technical problems or be linked to resistance to change among staff. It is important to proactively address such concerns before they develop into general mistrust and rejection of a particular technology.
Do a proper risk-benefit analysis
There are contexts or situations where certain technologies can do more harm than good. It takes some stamina to resist the urge to be innovative. But experimenting with untested technology on the back of affected people should not be an option. Do not use technology when data collected cannot be adequately protected and is so sensitive that it could put people at risk; when acceptance of a certain tool is very low and using it can create security risks; when the lack of infrastructure makes a project too costly; or when your organisation cannot guarantee long-term implementation.
Watch out for digital risks and privacy concerns
Risks related to digital security are insufficiently understood and addressed, and not just in the humanitarian sector. Even though aid practitioners are becoming more aware that digital data may be intercepted, or that mobile communication can be tracked, many organizations still opt against using encryption to secure their data. Increasing digitisation means increasing dependence on tools, which makes a potential attack on digital systems more harmful. At the same time, intrusion becomes more rewarding when attackers get their hands on greater amounts of information. Furthermore, technology-enabled aid often depends on for-profit actors, including businesses like Google, Dropbox or Facebook. It is important to keep in mind that aid organisations forfeit some control of their data by using third party tools. For a brief overview, see this toolbox for online privacy: https://www.privacytoolbox.gppi.net/obscure-me/
Contribute to improving practice
Because of potential dilemmas and risks of technology use in conflict-affected environments, it is important that humanitarian practitioners continue to aggregate experience with the use of technology in insecure environments. It is important to develop precedence before new actors create technology practice uninformed by humanitarian experience and principles.
- The Impact Tracker Technology (ITT) catalogue
Features a compilation of low cost, information communications technology-based tools intended to help experts working in the development sector collect data, communicate with their clients and measure their impact.
- IFRC World Disasters Report 2013
In this comprehensive guidebook, the IFRC collects diverse experiences and case studies from multiple humanitarian contexts, including insecure settings. The volume includes a wealth of detailed resources, in addition to a general analysis of information technology in humanitarian assistance.
- GPPi’s Toolbox for Online Privacy
Provides a starting point to protect privacy and security online when collecting more and more digital data and also in everyday life.
- Technologies for monitoring in insecure environments:
In this toolkit from the SAVE research project, downloadable as pdf, users can find a summary of applications, their pros and cons as well as many links to more detailed information.
- Case Studies: How organisations have been implementing the use of mapping drones in humanitarian contexts
The DG ECHO funded Initiative on Drones in Humanitarian Action produced a series of case studies describing the experiences of organizations that have used this technology in the past.
- SMS Feedback and Accountability in Somalia
This case study explores the development and implementation of an innovative, mobile phone-based feedback mechanism by the Danish Refugee Council (DRC) in Somalia. In a complex conflict setting, characterised by major insecurity, lack of access for humanitarian actors, and limited civil participation in state structures, the project piloted the use of mobile phones and internet-based technologies to strengthen communication and feedback between beneficiaries, aid agencies, Somali communities, and the diaspora.
- Seven key insights in using ICT to improve the Ebola response
Using ICT to Improve Ebola Response: During a Technology Salon in Washington, DC, on How Can ICTs Improve the Ebola Response, participants identified seven lessons on ICT use in the Ebola response that can inform future use.
- Groundtruthing OpenStreet Map Building Damage Assessment: Haiyan Typhoon – The Philippines - Final Assessment Report
The limits of remote satellite assessments: Comparing field-level information with crowd-sources maps. In this final report from the American Red Cross, USAID and the REACH Initiative, the accuracy of crowdsourced remote damage assessments based on satellite images was found to be very mixed. The report gives some useful recommendations for future projects.
- The International Rescue Commitee Commodity Tracking System (CTS)
Operating in Syria, with no access for its own staff in large areas, the IRC was searching for a solution to keep track of aid deliveries sent in and shipped throughout the country. Based on the widely-used OpenDataKit (ODK) software, the team developed a tracking solution using an android app and printed stickers. Documentation for understanding and learning how to use and adjust the tool is available as an online manual developed by the IRC’s CTS team
- Boda boda talk talk
South Sudan presents a challenging context for the use of technologies. Many people do not have mobile phones, nor smartphones or Internet access, and the mobile network is patchy, especially in rural areas. Internews developed a reliable information and communications channel both to keep communities updated and alert, and to hear their feedback called 'Boda boda talk talk'. Shows were pre-recorded, stored on USB sticks and then attached to and played using speakers that were physically brought to select locations by bike.
- Combining ‘missed calls with radio and other media
This article explains how development professionals are increasingly pairing radio and ‘missed calls’ to facilitate interactive communication.
- The Pact handbook
Is intended to give people involved in international development a framework for getting started with mobile technology. It can be paired with other resources and literature, country-specific information or training slides for more examples and context. The handbook provides instructions for developing and implementing strategies for using mobile, introduces different platforms that can be used and provides an overview of Pact’s own work with technologies.
- Dialling Down Risks: Mobile Privacy and Information Security in Global Development Projects
Referencing the huge opportunities but also inherent risks with using mobile phones for development projects, this paper proposes guiding privacy and security principles and a framework for project planning and evaluation.
- The use of ICT in contemporary mixed migration flows to Europe
This briefing paper produced by DRC and the Regional Mixed Migration Secretariat for the Horn of Africa and Yemen provides an analysis of how migrants use applications such as WhatsApp, Facebook, and other online communication platforms.
- UNITAR describes experiences with using Satellite imagery
UNITAR describes experiences with using Satellite imagery and analysis to assess conflict developments and their impact. The same tools could be used to monitor humanitarian programming in similarly inaccessible situations.
This theme page draws heavily on the SAVE Toolkit, Using technologies for monitoring and evaluation in insecure settings, and has been reproduced with the authors' permission. Readers are encouraged to view this resource for further information.
Dette, R. Steets, J. and Sagmeister, E. (2016) Using technologies for monitoring and evaluation in insecure settings. SAVE Toolkit. Retrieved from: http://www.gppi.net/fileadmin/user_upload/media/pub/2016/SAVE__2016__Toolkit_on_Technologies_for_Monitoring_in_Insecure_Environments.pdf