Critical System Heuristics (CSH) provides a framework of questions about a program including what is (and what ought to be) its purpose and its source of legitimacy and who are (and who ought to be) its intended beneficiaries.
CSH, as developed by Werner Ulrich and later elaborated upon in collaboration with Martin Reynolds, is an approach used to surface, elaborate, and critically consider boundary judgments, that is, the ways in which people/groups decide what is relevant to the system of interest (any situation of concern).
CSH is concerned not only with purposive evaluation, where the system or project has a predefined goal and the focus lies in evaluating the means of reaching it, but also more broadly with purposeful evaluation, where both the means and the ends become subjects of inquiry. CSH rests on the foundations of systems thinking and practical philosophy, both of which emphasize the 'infinite richness' of the real world. In this view, understandings of any situation are inherently incomplete, and therefore based on the selective application of knowledge. By systematically questioning the sources of motivation, control, expertise, and legitimation in the system of interest, CSH allows users to make their boundary judgments explicit and defensible. The immediate goal of a CSH evaluation is to elaborate multiple perspectives on a given situation, but the broader aim is to share these perspectives and thereby cut down on actors 'talking past' each other by promoting mutual understanding.
Bases and Background
Systems thinking is a broad field which employs 'systems' as conceptual tools which help to acknowledge and deal with the complexity of the real world (see: Churchman, 1971; Churchman, 1979). Evaluators are constantly confronted with overlapping networks of people, technology, geography, institutions, funding, and ideology which generate complex realities never entirely visible to any single person or from any single viewpoint. Each actor must develop their stances and actions by interpreting the flows of information available to them. Because no two people or viewpoints are exactly the same, this process will inevitably lead to a diversity of conclusions regarding the most appropriate appraisals and the best pathways forwards. Rather than focus on the stances and actions themselves, therefore, CSH zooms in on the process of interpretation. Rather than asking 'what do you see in this situation?' as in most evaluation approaches, CSH inquires more deeply, 'what are the underlying assumptions - values, power structures, knowledge bases, and moral stances - which affect how this situation is perceived by different concerned parties?'. Asking the twelve boundary questions (see: Table 1) fulfills the analytic purpose of CSH, which is to reveal how decisions about what is relevant to this interpretive process (boundary judgments) are being made.
The several strands of practical philosophy which underpin CSH emphasize (a) an orientation towards practical (rather than theoretical) goals, and (b) the role of the researcher as an agent of change, not simply a passive observer. Overlapping with the analytic purpose (of providing more information), therefore, CSH also intends to generate discussion, reflection, and critique. The boundary questions in Table 1 provide a common rubric designed to enable mutual understanding and contructive dialogue. Engaging debate about the issues exposed during the evaluation fulfills a practical purpose by opening up new ways to improve communication and hence the overall wellbeing of those involved.
Finally, by intentionally setting all actors' perspectives on equal footing regardless of their expertise or technical competence (sometimes referred to as a 'Socratic' professional stance), CSH can have an emancipatory purpose, amplifying otherwise unheard voices.
Conversation and learning from an evaluation can be severely hindered by debates that stem from differing boundary judgments and situational framings, but many approaches lack a means of openly and explicitly discussing the underlying sources of these judgments. CSH opens doorways to profound investigation of the different worldviews involved in a project.
The CSH toolbox is composed of twelve 'boundary questions' designed to outline and provoke thought about boundary judgments that determine situational framings. Each question is considered in two modes: an ideal mode (what 'should' be), and a descriptive mode (what 'is'), making twenty-four questions in total.
For systematic boundary critique, each question needs to be answered both in the `is’ and in the `ought’ mode. Differences between `is’ and `ought’ answers point to unresolved boundary issues. There are no definitive answers, in that boundary judgements may always be reconsidered. By means of systematic alteration of boundary judgements, it is possible to unfold the partiality (selectivity) of an assumed system of concern from multiple perspectives, so that both its empirical content (assumptions of fact) and its normative content (value assumptions) can be identified and can be evaluated without any illusion of objectivity.
Sources of Motivation
Sources of Power
Sources of Knowledge
Sources of Legitimation
Source: Ulrich, W. (2000). Reflective practice in the civil society: the contribution of critically systemic thinking. Reflective Practice 1, no. 2: 247-268.
CSH is not a 'prescribed methodology' with step-by-step directions of standard best practices, but instead a framework to encourage reflection. The technique of asking the boundary questions, their order, wording, and the manner in which the answers to them are consolidated will vary. Interested parties can refer to Reynolds (2007) for practitioner guidelines to application based on experience (p. 6-8), and Ulrich and Reynolds (2010) for a 'standard sequence' for beginners (p. 258-259), but the final form will remain eternally open to critique and modification.
Advice for USING this Approach (tips and traps)
CSH offers a unique approach with powerful tools, but its analytic, practical, and emancipatory power relies heavily on the specifics of its application. Discovering and critiquing boundary judgments can be a delicate process, depending on the context. It is important to ensure that participants in the process fully comprehend the end goal of the exercise and thus understand the level of questioning and the types of responses which may emerge. A key factor in developing this understanding is the language and phrasing used to explain CSH concepts. CSH has a significant corpus of jargon and vocabulary which require reformulation and rephrasing in each circumstance. An open, flexible professional culture with high levels of ex-ante 'buy-in' will greatly ease application, though these circumstances may not be where CSH has the most impact or affects the greatest change. Finally, the most significant 'results' of a CSH evaluation (greater awareness of selectivity of claims, enhanced communication, increased tolerance) are generally intangible and inaccessible to direct measurement. Disseminating these results to audiences outside of the project is an exercise of more than factual explanation which includes tactful and persuasive communication.
Impact of CSH Approach on the Conduct of Evaluation
- Mini-Primer of Critical Systems Heuristics: Werner Ulrich's compact summary of CSH
- Critical Systems Heuristics at SystemsWiki (archived link): A systems wiki of CSH
- What is Critical Systems Heuristics? Critical Systems Heuristics summary at Thinkpedia
- Critical Systems Heuristics: A Very, Very Short Introduction: on ConceptLearner Blog.
Overview and Example
- Systems-thinking and equity focused evaluations This chapter in UNICEF publications 'Evaluation for Equitable Development Results' (2011) by Reynolds and Williams gives an indepth overview and example of CSH
Guide and Example
- Critical Systems Heuristics: This chapter from Werner Ulrich and Martin Reynolds provides a detailed introduction to critical systems heuristics and the use of its central tool boundary critique.
- Werner Ulrich's Home Page: This website provides a variety of resources from Werner Ulrich's work in a range of philosophical and research areas with a particular focus on his own work in critical systems thinking and practice or, Critical Systems Heuristics (CSH).
Churchman, C. W. (1971)The Design of Inquiring Systems: Basic Concepts of Systems and Organizations. New York: Delta/Dell Publishing.
Churchman, C. W. (1979) The Systems Approach and its Enemies. New York: Basic Books.
Jeppesen, Sara Lise. (2011) “Exploring an Explicit Use of the Concept of Sustainability in Transport Planning.” Systemic Practice and Action Research 24 (August 26): 133-146. https://link.springer.com/article/10.1007/s11213-010-9180-6 (accessed April 25, 2012).
Reynolds, M (2007). “Evaluation based on critical systems heuristics.” In Using Systems Concepts in Evaluation: An Expert Anthology, edited by Bob Williams and Iraj Imam, 101-122. Point Reyes, CA: EdgePress. Available online at: http://oro.open.ac.uk/3464/
Romero, Roman Vega (2002). “The Relevance of Ulrich’s Critical Systems Heuristic for Social Planning Evaluation.” Cuadernos de Administracion 15, no. 24: 41-71.
Ulrich W (2005). A brief introduction to critical systems heuristics (CSH). ECOSENSUS project web site, Open University, Milton Keynes, UK, 14 October 2005. Available online at http://kmi.open.ac.uk/
Ulrich, W (1983) Critical Heuristics of Social Planning: A New Approach to Practical Philosophy. Bern, Switzerland and Stuttgart, Germany: Haupt. Paperback reprint edition, 1994, Chichester, UK, and New York: Wiley (same pagination)
Ulrich, W (1987). Critical heuristics of social systems design. European Journal of Operational Research 31, no. 3 : 276-283.
Ulrich, Werner, and Reynolds M, (2010) “Critical Systems Heuristics.” In Systems Approaches to Managing Change: A Practical Guide. Edited by Martin Reynolds and Sue Holwell:.243-292.. London: Springer, . https://www.researchgate.net/publication/43642939_Critical_Systems_Heuristics (accessed March 4, 2012).