Using technologies for monitoring and evaluation in insecure settings

Operating in insecure environments is one of the more critical tests for humanitarian, development, and peacebuilding organizations alike. Access constraints or even direct attacks make monitoring and evaluation extremely challenging.

Technologies like mobile phones, radios, Internet platforms and GPS trackers promise new solutions for collecting vital data or tracking implementation of projects.

In some of the most insecure humanitarian contexts, monitoring aid programmes are particularly important to humanitarian organisations. Did assistance get to the right people? Have aid trucks safely reached their intended location? How do affected people assess the services provided? Amidst access constraints, attacks and the risk of aid diversion, those questions remain difficult to answer.

Access restrictions, high costs, poor infrastructure and high levels of uncertainty require tools that can function without constant electricity supply, across large distances and without advanced computing skills. We identified six technology types that meet these criteria: handheld devices for digital data collection, mobile phone-based feedback mechanisms, remote sensing with satellites or delivery tracking, communication with online platforms and broadcasting with radios and other forms of media. M&E practitioners are increasingly optimistic about the use of these technologies when working in insecure environments.

Example

Examples of technologies used in insecure environments

• An international NGO runs a large, long-standing operation throughout Afghanistan, which involves multiple and time-intensive surveys every month. Transporting paper questionnaires from remote areas to local hubs often took days. Entering the responses from the paper questionnaire into the database, aggregation and analysis would take many hours. In 2013, the NGO bought 20 tablet computers for about 300 USD a piece. It set up these tablets with open-source software. the investment necessary to roll out digital data collection paid for itself after just one survey.

• IDPs in Iraq had limited access to reliable news from local media and information about available aid services. As one man in Dahuk noted: ‘We don’t know the organisations or their names; we have never spoken with them. We don’t know anything about their work’. In July 2015, an inter-agency group of UN agencies and NGOs launched a nationwide toll-free hotline. The Erbil-based call centre is run by two coordinators and four Iraqi operators, three of whom are female. The operators collect information from agencies every week in order to answer straightforward questions from callers. With more complex queries, the call centre’s coordinators contact the relevant agency or cluster for an answer and get back to the caller within three days.

• Strict access constraints to Syria and dynamic changes on the ground make it very difficult for humanitarian responders to have a clear picture of the situational needs. The UN body for space imagery, UNOSAT, thus aimed to contribute to context understanding using satellite technology. Satellite imagery and analysis were used to assess conflict developments and their impact. The same tools could be used to monitor humanitarian programming in similarly inaccessible situations.

(For more information on these examples, see the SAVE Toolkit: Technologies for monitoring in insecure environments)

Types of technologies 

Several technologies can be used in insecure settings to collect and communicate data from aid programs:

The rising usage of basic mobile phones in volatile environments can make communicating with communities easier. Many organisations are expanding their outreach and feedback systems into accepting messages via SMS, calls or interactive voice response, in which pre-recorded messages are used to provide or gather information.

‘Handhelds’ such as smartphones, tablets or more basic digital devices are becoming cheaper and easier to use. Many aid organisations have found it convenient and cost-effective to use digital devices for data collection instead of traditional paper-based surveys. Aid actors can now tap into a substantial market for digital form creation and data collection applications. Digital data entry linked to electronic databases also facilitates automatic data analysis and visualisation.

Where physical access is constrained, there are a number of aerial technologies to gather information about objects or areas without having a physical presence. This is also referred to as ‘remote sensing’ and can reveal context conditions and other observable changes such as construction, agricultural developments or population movement. Additional remote sensing tools, like radars and infrared, can further augment this. Similarly, devices linked to ‘navigation satellites’ make it possible to identify and trace the location of deliveries or staff and to visualise information on maps.

Radio remains the most popular technology that people use to receive news and updates in many resource-constrained contexts. In many insecure environments, it is the most reliable way to reach communities. It is often understood as a one-way communication tool, but radio programming can also be used for active engagement, involving or supporting communities in creating their own shows.

Finally, where online communication platforms are popular, aid agencies can use them for their monitoring, feedback and accountability efforts. This includes social media networks such as Facebook and Twitter, as well as instant messaging applications like WhatsApp. These tools make it possible to transmit information and messages via online connections – often free of charge.

Important considerations

Take your time

Practitioners are well advised not to rush implementation. Consulting with partners who already use the technology, and developing a thorough understanding of who influences and spreads information in the specific context, is critical for success. Also, factor in time to win the support and trust from colleagues and affected people for the solution, and work with users when inventing, designing and testing tools. Challenges can stem from inherent biases of some technologies – for example, men are more likely to own the household phone, making it difficult for women to use phones anonymously. Concerns can also arise from technical problems or be linked to resistance to change among staff. It is important to proactively address such concerns before they develop into general mistrust and rejection of a particular technology.

Do a proper risk-benefit analysis

There are contexts or situations where certain technologies can do more harm than good. It takes some stamina to resist the urge to be innovative. But experimenting with untested technology on the back of affected people should not be an option. Do not use technology when data collected cannot be adequately protected and is so sensitive that it could put people at risk; when acceptance of a certain tool is very low and using it can create security risks; when the lack of infrastructure makes a project too costly; or when your organisation cannot guarantee long-term implementation.

Watch out for digital risks and privacy concerns

Risks related to digital security are insufficiently understood and addressed, and not just in the humanitarian sector. Even though aid practitioners are becoming more aware that digital data may be intercepted, or that mobile communication can be tracked, many organizations still opt against using encryption to secure their data. Increasing digitisation means increasing dependence on tools, which makes a potential attack on digital systems more harmful. At the same time, intrusion becomes more rewarding when attackers get their hands on greater amounts of information. Furthermore, technology-enabled aid often depends on for-profit actors, including businesses like Google, Dropbox or Facebook. It is important to keep in mind that aid organisations forfeit some control of their data by using third party tools. For a brief overview, see this toolbox for online privacy: https://www.privacytoolbox.gppi.net/obscure-me/

Contribute to improving practice

Because of potential dilemmas and risks of technology use in conflict-affected environments, it is important that humanitarian practitioners continue to aggregate experience with the use of technology in insecure environments. It is important to develop precedence before new actors create technology practice uninformed by humanitarian experience and principles.