Balancing innovation and rigor: Guidance on how to thoughtfully integrate AI in evaluation

This guidance note captures learning about integrating artificial intelligence (AI) into evaluation and offers a framework for further exploration.

The resource, jointly developed by the Independent Evaluation Group (IEG) of the World Bank and the Independent Office of Evaluation (IOE) of the International Fund for Agricultural Development (IFAD), establishes clear criteria to judge AI performance and good practices for designing meaningful experiments. These criteria were used to test AI on key tasks such as text classification, summarisation, and synthesis.

"Experimenting with AI in evaluation practice involves thoughtful risk-taking, continuous learning, and adaptation. It is a continuous process of questioning, testing, learning, and refining, analogous to how models like GPT themselves learn during their training phase: through constant trial and error."

This guidance note focuses on defining and adapting evaluation workflows to include AI where it fits best and to build trust through thorough testing.