52 weeks of BetterEvaluation: Week 23: Tips for delivering negative results

 

It’s a scenario many evaluators dread: the time has come to present your results to the commissioner, and you’ve got bad news. Failing to strike the right balance between forthrightness and diplomacy can mean you either don’t get your message across, or alienate your audience. In our recently published paper, ‘Two sides of the evaluation coin’ the evaluators and commissioners of a network evaluation discussed the difficulty of delivering even neutral findings along with positive statements so that both are heard effectively. 

We recently came across this excellent summary of how to deliver negative evaluation results constructively, produced by Susan Lilley from posts on the American Evaluation Association discussion list EVALTALK. The contributors came up with some great tips, which we’ve added to below.  We’d love to hear your views on the difficult issue of delivering bad news. You can add your comments in the box below this blog.

“1. Use a participatory approach from the start: Engage stakeholders in describing program logic, defining evaluation questions, identifying indicators of success and selecting appropriate data collection methods and tools. When these are defined by stakeholders, evaluation results are more likely to be in line with their expectations.”

This is good practice for most evaluations and useful for a lot more than just delivering bad news; it’s also a crucial strategy for improving the likelihood that evaluations will get used.

“2. Discuss possible negative results in the early contracting and design stages: Encourage clients or stakeholders to articulate their concerns and expectations early on about what the evaluation will reveal, and plan with them about how best to handle these results if they do occur.”

Don’t jump straight into an evaluation assignment without discussing important decision-making processes, quality/ethical standards or the potentially difficult situations that may arise.

 “3. Inform clients immediately and often - a 'no surprises' approach: The worst way for people to learn about negative results is in the evaluation report or in a near-final presentation. As soon as any negative results begin to emerge, gently inform the client through a phone call or a meeting. This approach provides time for people to come to grips with negative findings, to decide how to handle them, and to question the methods or data while there is still time to make adjustments.”

The only exception to this rule might be when your evaluation has uncovered grave failings in the project which key stakeholders are directly responsible for or aware of, or where you fear that stakeholders may move to discredit the evaluation to protect their own interests. In this case, early communication of preliminary results could result in a ‘cover up’.

“4. Build in time for course correction: Recognize from the start that negative findings may occur, and build time into the evaluation plan for clients to initiate action to address them before the evaluation is complete. The final report can then tell the positive story of how a problem was identified and has been corrected.

This will depend on the type and terms of the evaluation -  for independent summative evaluations, there may not be the opportunity for feedback loops between the evaluator and the intervention.

“5. Question the evaluation plan: In cases where evaluation questions, indicators or data collection tools have been imposed on the program, question whether they are appropriate. If not, develop alternative criteria and tools, and tell both stories: how the imposed methods show no progress but locally relevant methods do.

“6. Emphasize the positives: Every initiative will have some positive results, even if they are not very relevant to the funders' priorities. Make sure that your evaluation captures all positive outcomes, and highlight these. Begin and end reports and presentations with the positives, sandwiching the negative findings in the middle.”

However, recent organisational research shows that the positive / negative sandwich is not such a straight forward strategy. For one, negative feedback is not always bad and positive feedback is not always good. It depends on who you are talking to and labelling feedback as positive or negative may not be very helpful. (See this article for more on this).

“7. Tell the truth: Ethically, negative findings must be fully reported. Most of the stakeholders will already be aware of the problems and will appreciate the fact that they have been brought out into the open and can now be addressed.”

A good way of heading off objections to rigorous truth telling is to fall back on professional guidelines, particularly if you are a member of a professional body. It is easier to say ‘the rules won’t let me suppress negative findings’ than ‘I won’t’.

“8. Present results in terms of lessons learned: Identify what is working, what might need tweaking, and what needs to go back to the drawing board.

“9. Provide suggestions for addressing deficiencies: Provide clients with concrete suggestions for addressing the issues, drawing on your own experience and the research literature. Refer to best practices and to how others have successfully handled similar issues. When available, provide contacts who have agreed to speak with them about how they dealt with these issues.

“10. Involve stakeholders in identifying obstacles and ways to overcome them: There are often many good reasons why work has not been carried out as planned or objectives have not been achieved. Use a participatory process such as a force field analysis to engage stakeholders in identifying what internal and external forces were working against them, and describe these in your report. Involve stakeholders in identifying ways to overcome these hindering forces and to strengthen the forces that support their work.”

Have you had any experiences of delivering negative evaluation results? What strategies would you recommend? Share them with us here. 

 

Picture: http://www.flickr.com/photos/chascar/

'52 weeks of BetterEvaluation: Week 23: Tips for delivering negative results' is referenced in: