Search
38 results
Filter search resultsSet-theoretic methods for the social sciences: A guide to qualitative comparative analysis
… This book, by Schneider and Wagemann, provides a comprehensive overview of the basic … causality and applications of Qualitative Comparative Analysis (QCA), the most developed form of set-theoretic … the underpinnings for the vast majority of social science methods and techniques. Most of the well-known and commonly …ResourceForce field analysis
… A force field analysis is used to support the decision making process by … change issue. It allows the user to assess the source and strength of these forces and is particularly useful in … implementation stages of change management "Force field analysis is best carried out in a small group of about six to …MethodRealist analysis of testable hypotheses
… Realist analysis of testable hypotheses tests the program theory by … of ‘what works for whom in what circumstances and in what respects, and how?’. Realist analysis elicits and formalises the theories on what works …MethodLost causal: Debunking myths about causal analysis in philanthropy
… This 2021 paper, updated in 2024, advocates for more causal analysis in philanthropic evaluation - not just describing actions taken and changes observed, but also learning how and why change … sets out and debunks nine common myths: Rigor and causal analysis Myth 1 : RCTs or quasi-experiments are the only ways …ResourceBroadening the range of designs and methods for impact evaluations
… Stern, Nicoletta Stame, John Mayne, Kim Forss, Rick Davies and Barbara Befani for the UK Department for International … describes how theory-based, case-based and participatory methods can be used in impact evaluations. These designs show … designs: Theory of Change, Process tracing, Contribution Analysis, Impact pathways Causal mechanism designs: Realist …ResourceQuasi-experimental methods for impact evaluations
… by Dr Jyotsna Puri for the Asian Development Bank (ADB) and the International Initiative for Impact Evaluation (3ie), demonstrates how the use of quasi-experimental methods can circumvent the challenge of creating a valid … evaluations, using easy-to-understand language, graphics, and specific examples. … Quasi-experimental methods for …ResourceThe rigor of case-based causal analysis: Busting myths through a demonstration
… utilisation of case-based designs for conducting causal analysis and dispelling two misconceptions about their use in the … the methodology employed in the evaluation's causal analysis, highlighting key principles in the analysis design …ResourceUsing qualitative comparative analysis to explore causal links for scaling up investments in renewable energy
… This paper illustrates how qualitative comparative analysis (QCA) was used to identify causal pathways for … scaling renewable energy to meet sustainable development and climate goals. In this resource, QCA was used to validate … information, triangulating findings from other methods and helping to draw conclusions on how best to meet …ResourceProcess tracing and contribution analysis: A combined approach to generative causal inference for impact evaluation
… This article, written by Barbara Befani and John Mayne for the IDS Bulletin (Volume 45 Number 6), outlines how the combined use of contribution analysis (CA) with process tracing (PT) can shift the focus … on causality Generative causality in contribution analysis Contribution analysis Process tracing Using evidence …ResourceMultiple lines and levels of evidence
… Multiple lines and levels of evidence (MLLE) is a systematic approach to … for judging the credibility of the evidence and the causal analysis, often involving an expert cross-disciplinary panel … 'Using the template' Adapting the criteria for MLLE analysis The following, extracted from a table in Young et …MethodMaking rigorous causal claims in a real-life context: Has research contributed to sustainable forest management?
… - the Centre for International Forestry Research (CIFOR) and the Centre de Coopération Internationale en Recherche … an example of a rigorous non-counterfactual causal analysis that describes how different evidence and methods were used together for causal inference without a …ResourceRAPID outcomes assessment
… RAPID outcome assessment (ROA) is a method to assess and map the contribution of a project’s actions on a … tool that can be used in conjunction with other evaluation tools and methods. The method was developed by the RAPID (Research …MethodEvaluation costing
… Evaluation expenses are highly situational and there are no magic formulas for calculating costs. After … Terms of Reference ) which outline the evaluation design, methods, sample sizes and other considerations that have a … billable days. Nevertheless, an estimate of time and fair market price for personnel costs ensures that lump sum …MethodHow to design and manage equity-focused evaluations
… is divided into two parts. It begins by defining equity and its importance and relevance today. It then unpacks the … evaluation framework, evaluation design and appropriate methods to collect data; and how to ensure the evaluation is … Theory-based Equity-focused evaluation B. The bottleneck analysis framework Selecting the appropriate evaluation …ResourceBeneficiary assessment: An approach described
… or one-shot evaluation purposes, most of the ongoing and planned beneficiary assessments are iterative learning … which were found useful by managers in both the design and implementation of activities in all sectors and regions … Impact of Beneficiary Assessments Timing and Cost Specific methods mentioned Interview Themes Conversational Interview …ResourceEvaluation budget matrix
… be costed as individual line items. Evaluation planning and budgeting are intimately connected, developing a budget … matrix may outline costs for planning, data collection, analysis, reporting an evaluation. The cost-per-activity … methods. Excel or bookkeeping software packages are useful tools for developing the budget matrix. The budget matrix may …MethodWhat “extras” do we get with extracurriculars? Technical research considerations
… This research paper uses Instrumental Variable methods to analyse whether "participation in out-of-school … achievement or behavior for elementary school children? [And] If so, are the impacts of participation related to the … & Puma, 2003) Contents Background Methodology The Data Analysis Approach Analytical Options Initial Analyses: The …ResourcePropensity scores
… estimate the difference in outcomes between beneficiaries and non-beneficiaries that is attributable to a particular … have not been randomly assigned to a particular program, and those units which choose or are eligible to participate … scores of all units in the sample, both beneficiaries and non-beneficiaries, are used to create a comparison group …MethodEpisode studies
… to a case study that focuses on a clear policy change and tracks back to assess what impact research had among the … impacts of it forwards over time. This can be useful (see tools on research utilisation and bibliometric techniques), … RAPID episode studies, this was done through a variety of methods: review of the literature interviews with key …MethodProcess tracing
… Process tracing is a case-based and theory-driven method for causal inference that applies … have been observed or measured. This includes identifying and investigating alternative explanations for these changes. … also fit other explanations. They can be used early in an analysis to identify areas that are worth investigating …MethodStatistically controlling for extraneous variables
… on the extraneous variables, as well as the independent and dependent variables for analysis. Evaluators can use a range of statistical … techniques for control including: General Linear Model Analysis of covariance … Statistically controlling for …MethodImpact evaluation in practice
… in the development field. It also provides several tools and approaches for conducting impact evaluations. "The … We provide the reader with a core set of impact evaluation tools—the concepts and options that underpin any impact …ResourceCheck results match a statistical model
… project theory design. Statistical models can be useful tools to predict elements of the program: Cost Time …MethodInstrumental variables
This method is used to estimate the causal effect of variables on an intervention.MethodStatistically created counterfactual
… A statistical model, such as regression analysis, is used to develop an estimate of what would have …MethodLogically constructed counterfactual
… the intervention. Process tracing can support this analysis at each step of the theory of change. … Logical …MethodRandomised control trials for the impact evaluation of development initiatives: a statistician's point of view
… This paper from the Institutional Learning and Change (ILAC) Initiative provides a range of technical and practical reflections on the use of randomised control … for an open mind in the selection of research options and tools." (Barahona, 2010) … Randomised control trials for the …ResourceCracking causality in complex policy contexts
… addresses the challenge of making credible causal claims and discusses experiences from developing the Qualitative … Key features The author questions the lack of interest and investment in qualitative approaches to assess causal … challenges such as confirmation bias, opaque data analysis, credible generalisation, and effective …ResourceMonitoring and evaluation for thinking and working politically
… This article explores the challenges of monitoring and evaluating politically informed and adaptive programmes in the international development … and their own practical experience. They recommend methods that consider generative causality as especially …ResourceCheck intermediate outcomes
… If data are collected about these intermediate outcomes, and they can be linked to final impacts, it is possible to … related to understanding causes (analyze counterfactual, and investigate exceptions) this provides a stronger causal analysis than simply reporting totals for outcomes and …MethodMaking causal claims
… authored by John Mayne for the Institutional Learning and Change (ILAC) Initiative argues the need for a different … Using theories of change, approaches such as contribution analysis can be used to demonstrate that the intervention … made a difference – that it was a contributory cause – and to explain how and why. Content Concepts of causality …ResourceCheck dose-response patterns
… Evaluators can examine the link between dose and response as part of determining whether the program … program theory will identify the key expected responses and outcomes of the program by domain. Develop a …Method