All Methods Are Wrong. Some Methods Are Useful.

 

Interest in systems concepts is reviving and broadening. However, the sheer size of the field poses a dilemma for both newcomers and those who have long promoted systems concepts. In 2002, the International Institute for General Systems Studies (IIGSS) drew up an inventory of systems approaches and came up with more than 1,200, ranging from the familiar to the downright obscure (see “The Systems Genealogy”). No wonder some people find it difficult to sort through the breadth and diversity of approaches to determine what is useful to them and what is not.

THE SYSTEMS GENEALOGY

THE SYSTEMS GENEALOGY

This is the map of the various systems methodologies, methods, and techniques. Click here to see an expanded version.

Over the years, I’ve found that starting with methods, whether causal loop diagrams or Soft Systems Methodology or Social Network Analysis, often confuses or exasperates novices. Learning one of the above methods is expensive in terms of time, intellectual resources, and effort. Newcomers have enough on their plates without adding yet more things to learn. Furthermore, no single method will equip them with the power of the systems field. Indeed, the statistician George Box is quoted as saying, “All models are wrong. Some models are useful.” If that is so, then should we abandon trying to teach methods to the novice and focus on overarching systems principles?

TEAM TIP

Follow the steps in this article to tailor a systemic approach to suit the needs of your specific challenge.

Yet exploring systems concepts purely at the level of principle, generalization, and especially metaphors such as “machine vs. biological” doesn’t really survive close ontological scrutiny. It has probably been one of the reasons for the field’s reputation in some quarters for vagueness, imprecision, and restating the obvious in obscure ways.

So we have a tension between methods and principles. Without resolving this tension, I believe that we risk having newcomers hesitate or even lose interest in the systems field because of the perceived difficulty and steep learning curve. They will unfortunately miss out on the benefits the systems field offers. The systems field also loses opportunities for cross-fertilization from other fields, for instance, new methodologies in the international development and evaluation fields.

A way around the tension is to acknowledge the existing skills of newcomers and use systems principles as a starting point that can, when necessary, guide them to choose methods that match their skills and situations. The process could go something like this:

  1. Apply some basic principles that underpin the systems field to existing, established practice in the new person’s field of expertise (e.g., geography, evaluation, organizational development).
  2. If operating at the principles level is insufficient for the task in hand, then adopt, in whole or in part, specific methodologies, methods, and techniques from the systems field. For instance, people often find that the CATWOE (Customers, Actors, Transformation Power, Worldview, Owner, Environmental Constraints) mnemonic from Soft Systems Methodology a good way to order their thoughts when addressing key aspects of a problem situation.

These ideas became the organizing framework for Richard Hummelbrunner and my recent book, Systems Concepts in Action: A Practitioner’s Toolkit (Stanford Business Books, 2010). But first we had to deal with the bad news. As we know, there is no consensus about the core principles that underpin the systems field. So where could we start?

Three Core Concepts

For guidance, Richard and I reached back to a meeting the Kellogg Foundation funded some years ago. This meeting brought together practitioners from across a broad spectrum of the systems field, including soft systems, critical systems, complex adaptive systems, viable systems, system dynamics, and activity systems. Our task at that meeting was to explore what we had in common. After a couple of days, something concrete emerged. We came to the conclusion that underpinning all systems methodologies, methods, and techniques were three interwoven concepts: interrelationships, perspectives, and boundaries.

There was nothing startlingly new about this finding. In fact, as Gerald Midgley indicated in his book Systemic Interventions (Klewer, 2000), these concepts can be described in terms of the waves in the development of systems practice over the past 50 years.

Until the late 1960s, the focus of the systems field was very much on interrelationships. In many ways, this timeframe represented the “wiring diagram” phase of thinking systemically, with variations on boxes linked by arrows, often resulting in complicated schemata.

By the early 1970s, it was clear that the concept of interrelationships, while important, was not neutral. The relative importance of particular interrelationships often depended on the different perspectives through which people observed the system. Thus systemic thinking began to include the implications of applying different perspectives to the same situation.

However, by the mid-1980s, it was concluded that perspectives were not neutral either. Perspectives influence what we see to be relevant or not; they determine what is “in” the frame and what lies “outside” it. Whoever defined the dominant perspective controlled the boundary of a systemic inquiry or intervention. Thus, the importance of studying boundaries and critiquing boundary decisions (including those who made them) is the third core concept underpinning a systems approach.

So let’s look at each of these concepts in turn. In particular, we’ll contemplate how they can be expressed in ways that may be applicable outside the systems field and across other fields and disciplines.

Interrelationships

Many newcomers to the systems field are familiar with the idea of interrelationships. How things are connected and with what consequence stems from the earliest thinking about systems. It is also the concept most strongly embedded in the popular imagination. When we talk about the education system or the health system, we imagine a set of objects and processes that are interconnected in some way. The popularity of system dynamics and complex adaptive systems cements the notion of interrelationships as an important systems concept.

However, systemic thinking doesn’t concern itself with just any interrelationships. It focuses on particular aspects of them:

  • Dynamic Aspects — how the interrelationships affect the behavior of a situation over a period of time
  • Non-Linear Aspects — how the scale of effect of interrelationships is apparently unrelated to their scale and is often but not always caused by feedback
  • Sensitivity of Interrelationships to Context — how the same interrelationship in different contexts or areas has varying results, making it unreliable to translate a “best practice” from one situation to another
  • Massively Entangled Interrelationships — how interrelationships can result in simple, complicated, complex, and chaotic behaviors

These ideas are familiar to the experienced systems practitioner but just obscure jargon to the uninitiated. Richard and I needed an alternative way of expressing these general ideas, one that would be easily applied across disciplines and without great technical knowledge of systems concepts, while at the same time avoiding overgeneralization. Our solution was to phrase these as questions implied by thinking systemically about interrelationships:

  • What is the structure of the interrelationships within the situation (e.g., resources, stakes, stakeholders, knowledge)?
  • What are the processes between elements of that structure?
  • What is the nature of the interrelationships (e.g., strong, weak, fast, slow, conflicted, collaborative, direct, indirect)?
  • What are the patterns that emerge from these interrelationships in action, with what consequences and for whom?
  • Why does this matter? To whom? In what context?

These are not definitive questions, but I have found them a good place from which to help people new to systems concepts consider how they can make their existing practice more systemic.

Perspectives

However, a systemic approach is more than describing how boxes and arrows fit together or networks operate. Just looking at interconnections does not make an inquiry or intervention systemic. It is also important to consider how you look at the interrelationships. People will see and interpret them in different ways.

We frequently overlook that fact that our perceptions promote behaviors that affect the way a situation unfolds. What we see as unintended consequences often result from our unwillingness to understand or explore other people’s perceptions. Many times, somebody, somewhere actually intended the result that we consider problematic. Thus we cannot comprehend the dynamics of a situation without identifying and understanding the range of relevant perspectives that lie behind and in front of it.

I find it helpful for newcomers to distinguish between two forms of perspective: stakeholders and stakes. Stakeholders are groups of people or things that have a common role in a situation or intervention (e.g., teachers, consumers, writers). In contrast, stakes relate to individual values and motivations (e.g., wealth, honor, fairness, historical narratives). People belonging to different stakeholder groups may share the same stakes, and any one stakeholder grouping will contain within it several different (perhaps conflicting) stakes. Assessing how people juggle their own conflicting stakes goes a long way to understanding behavior in certain situations.

For instance, in writing this article, I’d be considered an author in stakeholder terms, and you might be considered a reader. I might also be considered a consultant, and you an academic or manager. However, regardless of my stakeholder identity, I have a range of stakes in this article: Knowledge — I want to improve the understanding of systems concepts. Ego — I’d like to earn some recognition for my thoughts. Economic — I’d like to earn some money by selling books or persuading you to hire me. You, the reader, might be reading this for similar reasons, say knowledge and economic (gaining knowledge that you can use to earn money). So, we may be different stakeholders, but some of us will share the same stakes in this small endeavor. To understand the impact of this article in the medium term, an understanding of the stakes at play may be more reliable than an understanding of the stakeholders.

Deliberating on the impact of different stakeholders and stakes gives us an opportunity to reframe issues (see “Perspectives: Stakes, Stakeholders, and Framing”). And different ways of reframing a situation can make apparently insoluble issues of stakes and stakeholders resolvable. For instance, Russ Ackoff once helped a manufacturer reframe what it considered an industrial relations issue into an economic cycle issue. This reframing gave the business space for creating an entirely new product line and in doing so solve (or in Russ’s terms “dissolve) the industrial relations issue.

PERSPECTIVES: STAKES, STAKEHOLDERS, AND FRAMING

PERSPECTIVES: STAKES, STAKEHOLDERS, AND FRAMING

Again, Richard and I sought ways to express these general concepts in terms of questions. We have found that variations on the following queries are helpful for newcomers:

  • Who or what are the key stakeholders within the situation?
  • What are the key stakes?
  • What are the different ways in which the situation can be understood or framed? By whom?
  • How are these different framings going to affect the way in which stakeholders act, especially when things go wrong from their perspective?

Boundaries

Holism, in the sense of including everything, is an impossibility. Every endeavor has to set boundaries — make choices between which relationships to include and which to exclude, which perspectives to honor and which to marginalize. Setting boundaries is not optional.

A boundary differentiates between what is “in” and what is “out,” what is deemed relevant and what is irrelevant, what is important and what is unimportant, what is worthwhile and what is not, who benefits and who is disadvantaged. Boundaries are the places where values get played out and disagreements are highlighted. A lot of power issues get wrapped up in boundaries — just as the person with the magic marker controls what goes on the whiteboard, the person whose perspective dominates a project decides the boundaries.

Accepting that thinking systemically about perspectives and interrelationships involves boundary choices implies that we must take a deliberate and critical approach to boundary identification and selection. Deliberations about whom or what to exclude have considerations both ethical (e.g., an obligation to do no harm) and pragmatic (e.g., the tendency of those excluded to oppose and resist). We have found the following questions a good place to start boundary discussions:

  • Which interrelationships are privileged and which are marginalized? With what effect on whom?
  • What perspectives (i.e., stakeholders, stakes, framings) are privileged and which are marginalized? With what effect on whom?
  • How can we manage the ethical, political, and practical consequences of these decisions, especially those that cause harm or have the potential to cause harm because they exclude an interrelationship or perspective?

The last question of course raises the further question of what kind of harm to whom. Hence the iterative nature of boundary questions; they raise the possibility that you may need to reassess your initial judgments on interrelationships and boundaries (see “Four Key Boundary Decisions”)

FOUR KEY BOUNDARY DECISIONS

FOUR KEY BOUNDARY DECISIONS

Part Two: Some Methods Are Useful

So our newcomers have a dozen questions from which to start their systemic inquiry. While these might be a starting point, we need to provide a blueprint for what to do with them. Because interrelationships, perspectives, and boundary issues are closely interwoven, is the best idea to throw the questions randomly at the situation and see what happens? Or can practitioners pick and thread them in a useful order?

How can practitioners move beyond the “generalization trap” that is inevitable when operating purely at the principle level? And what about all those systems methods that people in the field have spent so much time and energy developing? How can they be a resource to newcomers? How can they mix and match the systems methods to suit different stages of a systemic inquiry?

These are important questions. Below I suggest a heuristic that attempts to address these issues. I also suggest some systems methodologies, methods, and techniques that could be particularly relevant for each stage. The suggestions are examples; the list is not comprehensive. I draw on methodologies Richard and I covered in our book. There are others that could serve equally well. Nor is any approach restricted to the particular step I’ve identified. Consider the suggestions as a proxy for “approaches like these.”

The heuristic has four steps that loop back on each other:

  1. Construct a rich picture of the situation of interest
  2. Frame the situation
  3. Consider the ethical and pragmatic consequences of these framings
  4. Assess the dynamics of the situation

Step 1: Construct a rich picture of the situation of interest

The process of constructing a rich picture of the situation of interest draws from Soft Systems Methodology, although many systems approaches have similar concepts. The following two perspective questions offer a good place to start with constructing the picture:

  • Who or what are the key stakeholders within the situation?
  • What are the key stakes?

The responses will give you information for three interrelationship questions:

  • What is the structure of the interrelationships within the situation (e.g., resources, stakes, stakeholders, knowledge)?
  • What are the processes between elements of that structure?
  • What is the nature of the interrelationships (e.g., strong, weak, fast, slow, conflicted, collaborative, direct, indirect)?

In essence, this first step provides the ingredients for the remaining three steps. This rich picture will be messy and overly detailed, and it will contain multiple perspectives and unidentified boundaries. Because of the risk of getting lost in this noise, at this stage, you might consider drawing on established systems methods.

Possible systems methods, methodologies, and techniques

Soft Systems Methodology (SSM) has useful techniques that enable a deep exploration of perspectives. Because Outcome Mapping focuses on changes in social actors, many people find it brings a good deal of rigor to identifying how they are contributing to an overall result when it is difficult to assign definite causal routes between an intervention and a broader long-term goal. Stafford Beer’s Viable Systems Model (VSM) poses questions that can help you explore information flows through organizations. The various methods that cluster under social network analysis and causal loop diagramming can aid in exploring a system’s core structural components.

Step 2: Frame the situation

Having completed Step One, the task now is to consolidate to possible ways of framing the situation.

  • What are the different ways in which you can understand or frame this situation?

This step marks the first attempt to make sense of the emerging picture and to break the task down into manageable chunks. For instance, if your rich picture is concerned with violence at a rock concert, there will be several ways of framing the situation that considered separately and together may help you address the problem (e.g., entertainment, income generation, cultural identity, drug consumption, marketing, event management) (see “Perspectives: Framing”).

Possible systems methods, methodologies, and techniques

PERSPECTIVES: FRAMING

PERSPECTIVES: FRAMING

If you are concerned with aspects of violence at rock concerts, then framings of the concert as entertainment, income generation, cultural identity, drug consumption can help address that problem.

Soft Systems Methodology (SSM) is a well-established approach largely driven by developing and analyzing different ways of framing a situation. Strategic Area Assessment, a regional planning approach developed in Europe, assesses a situation from a variety of different framings. Cultural-Historical Activity Theory (CHAT), developed initially in Europe and the US, explores the interaction of different motivations within a given situation.

Step 3: Consider the ethical and pragmatic consequences of these framings

Framings imply value judgments about what is relevant and what is to be ignored. These boundary decisions have ethical and pragmatic dimensions that practitioners need to make explicit and deliberate on.

  • Which interrelationships are privileged and which are marginalized? With what effect on whom?
  • What perspectives (i.e., stakes, stakeholders, framings) are privileged and which are marginalized? With what effect on whom?
  • How can we manage the ethical and practical consequences of these boundary choices and decisions, especially those that cause harm or have the potential to cause harm?

In essence, this step challenges you to deliberate on and critique your framing of the situation. In a subsequent iteration, it might force you to reconsider ways of addressing the situation.

Possible systems methods, methodologies, and techniques

This step can benefit greatly from systems approaches based on bringing together opposing or contradictory perspectives to force deep deliberations that enhance creative or innovative thinking. These include Systemic Questioning, Strategic Assumption Surfacing and Testing (SAST), CHAT, SSM, Circular Dialogue, Proposal One-and-a-Half, and Convergent Interviewing.

Critical Systems Heuristics (CSH) takes this dialectical approach even deeper by critiquing how and where we draw boundaries that have significant social and environmental consequences. In particular, it explores the boundaries that surround decisions of purpose, decision making, knowledge, and legitimacy and the tension between the existing situation and different views on how it should develop over time.

Step 4: Assess the dynamics of the situation

Having assessed and critiqued the way in which you understand the situation, the next step is to understand the behavior taking place and to assess the reality of the framings and boundary choices on how the situation actually develops and behaves.

  • How are these different framings and boundary choices going to affect the way in which people act within the situation, especially when things go wrong from their perspective?
  • How will these individual behaviors affect the overall behavior of the situation? With what result and significance?
  • What are the patterns that emerge from these processes? With what consequences for whom? Why does this matter? To whom? In what context?

These last two questions effectively steer you back toward reconsidering Steps 1, 2, and 3; especially Steps 2 and 3. The task is to see if certain negative aspects of the situation can be “swept in” to the system as now defined in a way that continues to serve its positive aspects (which of course is a matter of perspective and boundary setting).

Possible systems methods, methodologies, and techniques

System dynamics is an established approach to exploring how a situation will develop over time due to issues of feedback and delay. The Cynefin framework developed by Cynthia Kurtz and David Snowden and the Containers, Differences and Exchanges (CDE) model developed by Glenda Eoyang have helped many people understand more clearly how to handle the simple, complicated, and complex dynamics present in any given situation. Process Monitoring of Impacts, developed by Richard Hummelbrunner, is a hybrid method that seeks to identify core dynamics within a situation. Assessments of dynamics are heavily dependent on people’s assumptions about the situation and how to address it. SAST, Scenario Technique, and the Rand Corporation’s Assumption-Based Planning all contain questions that can help tease out important assumptions from the unimportant ones.

Part Three: All Methods Are Wrong. Some Methods Are Useful.

By the title of this article, “All Methods Are Wrong. Some Methods Are Useful,” I didn’t intend to criticize method per se or privilege some methods over others. The purpose of the title was to provide a space to addresses the tension between the general and the specific, as well as explore a means by which newcomers can learn to apply ideas that lie at the core of the systems field. I’ve filled that space with an approach that can be used at the level of systems concepts or as a means to help guide newcomers to the potential of specific systems methods. In our book, Richard Hummelbrunner and I take this one step further by proposing additional questions that we think further help people select specific systems methods. We believe that posing questions to identify the utility of particular methods in a systemic inquiry can guide both the newcomer and the more experienced practitioner toward appropriate use. It also advances the idea that no method will be useful in all circumstances and promotes the intelligent and informed use of multiple methodologies and methods in a systemic inquiry.

Bob Williams is a consultant based in New Zealand. He works all over the world, largely promoting the use of systems ideas in the evaluation field. Bob is the coauthor, with Richard Hummelbrunner, of Systems Concepts in Action: A Practitioner’s Toolkit (Stanford Press 2010), a compendium of 20 systems methodologies, methods, and techniques. He’d like to thank Martin Reynolds, Richard Hummelbrunner, Ricardo WilsonGrau, Heather Britt, Irene Guijt, Jan Noga, Gerald Midgley, and Patricia Rogers for their assistance in developing this article.

Go HERE for information on many of the approaches in this article.

NEXT STEPS

Thinking systemically is a matter of capacity development. There is knowledge to be acquired, skills to be gained, learnings to be acquired—and opportunities to be sought for applying all of these. However, it is also critically about capability. Over the years of running systems thinking workshops, I have often heard the comment that the ideas are great, but the organizations people work with would never tolerate the kind of questions systems thinking poses. Under such conditions, the newcomer often asks where he or she should start.

Gerald Midgley suggests that generally speaking the best place to start with is where you are right now. I agree with him. Do the notions of focusing on interrelationships, perspectives, and boundaries help you improve your own understanding of a situation? Will addressing the questions under each of the main headings of this paper help you puzzle your way through a problem or bring constructive light onto an issue? If you have answered “yes” to any of these questions, then you have probably found the best place to start.

Sign up or sign in to bookmark this article.