Can evidence-based and systemic change be combined?
Wisdom is an unattainable ideal This is so because ideals are abstractions, which means that they are only ideal in as far as we can ignore their context. In practice, we can never ignore an ideal’s contingent reality. That’s probably why the Greeks coined the term ‘philosophy’, the love of wisdom. It does mean, however, that ideals – albeit unattainable – can and should be pursued. Although no philosopher perhaps, a systems thinker takes a slightly different angle in that he/she considers wisdom unattainable, because no matter how hard we try, reality will always remain stubbornly complex, way beyond our understanding. This is both frustrating and exciting. With systemic problems abounding, it is also our guaranteed escape from tedium.
Wisdom and the systems approach According to Churchman, ideals give meaning to our objectives and goals, both personally and professionally. Now, practical wisdom – as opposed to the wisdom of academia – can be pursued using the systems approach. In a sense the systems approach IS practical wisdom. The wisdom part of the systems approach is that it is able to design interventions (plans, projects, enterprises) to suit multiple ideals (to some extent). The practical part is that it has an approach to do so. This involves looking at multiple perspectives and their underlying motivations. The systems approach recognizes the relative ineptness of any of the resulting plans, but it thinks there is no better way of achieving wisdom in this world.
Wisdom in the knowledge pyramid… – a.k.a. the DIKW or wisdom hierarchy – is a model for representing the relationships between Data, Information, Knowledge and Wisdom. Ackoff posits understanding as a distinct layer between knowledge and wisdom (see below for my concept map of the model). He refers to understanding as an “appreciation of ‘why'”, and wisdom as “evaluated understanding”. Wisdom is the ability to increase effectiveness. Effectiveness is what justifies action. Justification is the holy grail of the systems approach. Wisdom adds value, which requires the mental function that we call judgment. The ethical and aesthetic values that this implies are inherent to the actor and are unique and personal. As described elsewhere in this blog, judgment is crucial in design, whether it is the design of interventions or the design of art or artifacts.
Systemics or statistics In recent years there has been an avalanche of attempts to develop evidence-based policies or interventions, with Banerjee and Duflo among the best known proponents. It hardly befits me to criticize that approach, not in the least because Duflo has received praise from the most credible and highly revered sources such as Nobel Laureate Amartya Sen. Yet, I feel a great deal of unease with this evidence-based approach. Now why is that?
Opposites by necessity The first point of unease (A) is that it doesn´t fit very well in the above knowledge model: statistics, being data-based, are at the extreme opposite end of wisdom in the knowledge hierarchy. Of course I can imagine statistical designs that compare variants of “wise” policies, but that: (1) would narrow the base for one’s evidence; (2) still necessitates some form of wisdom (whence does it come?); and (3) suggests that the statistician is impregnable to criticism because of the hard, “scientific” evidence he/she produces.
Human medicine derivative The second point (B) is that the evidence-based policy model is derived from a medical model. It all started with the work by Archie Cochrane in the 1970s. Since then health policy research started looking into the possibility of developing the human medical model into a societal (or public) policy and management model. What we forget is that medicine deals with the human body, which is a much more homogeneous system than societal or management systems. Besides, human medicine is an incredibly productive field of science, at least in terms of the number of publications for each of the multitude of medical specialties. There was both a need and a great opportunity for systematic reviews. Health systems, too, are pretty homogeneous. It is the same model that has been exported everywhere. And now this initial homogeneity is guaranteed to persist or return as a result of this new, homogenizing, evidence-based policy making.
Livelihood approaches cannot be top-down The third (C) point is that livelihoods – and therefore poverty alleviation efforts – are not only highly location-specific (i.e. the opposite of homogeneous), but also highly sensitive to the effectiveness of poverty interventions. This is crucial because it implies that a slight failure can have devastating effects on the “beneficiaries”. There seems to be a great risk that in many circumstances the intervention will deteriorate the conditions of the poor, rather than improve them. I doubt that the statistical designs used in evidence-based approaches take this aspect into account (my concern was justified, see Taylor, 2013, below). This is all the more important, because the evidence-based approaches seem to suffer of the problem of top-down approaches generally, namely that they ignore or eliminate local agency, my fourth (D) and final point.
The systems approach differs.… in that it is not statistical at all, although it is not opposed to using statistics to develop good understanding of a problematic situation. Instead it looks at any aspects that seem relevant, whether they are measurable or not. Indeed, a great many important aspects cannot be measured, as is the case when we ponder the question whose knowledge should not be ignored in a design. Evidence-based statistics must by necessity decontextualize design, whereas systems thinking tries to sweep in as much context as possible with a view to make a coherent whole of all design elements by taking into account underlying aspects that are even harder to measure – perish the thought! – such as world views, values, and paradigms. This process of making a coherent whole is also known as ‘unfolding’, so that’s in summary the systems approach: sweeping in and unfolding.
The best of both worlds If Taylor is right in identifying evidence-based policy making and systemic approaches to development as important recent trends, then why not combine the two in order to fill each other’s voids? No matter what else statistics does, it helps us look closely at relationships and may help us avoid assumptions that are clearly untrue. And no matter what else systems thinking does, it can give us a fair idea of the whole picture. It may even help design better systematic reviews or analyse existing data in an alternative way by developing new, more systemic solutions (or lenses of “inquiry”). The systems approach (see Williams & van ‘t Hof, 2014) can also be used for feed-back or evaluation. The systems approach can be particularly good at asking which measures for success to use and many other things. Considering that politicians and managers the world over prefer “silver bullets” over quirky ones – particularly for reasons of upscaling cost-effectively – it might also be possible to develop and test evidence-based framework solutions that leave more room for local, systemic agency. It’s just a thought. They happy, I happy.
Final remarks For a long time I have wanted to write something about evidence-based policy making, so this is it. Brief as it may be. Incidentally, during writing,I came across the paper by Ben Taylor (2013). It shows some parallels with this blog post. Also, the Springfield Working Paper suggests that systemic change is more mainstream in international development than I thought. Interestingly, systemic change seems to be poorly understood and, judging by the cases, not very well implemented. This actually applies not only to the systemic change projects, but to the evidence-based approaches, too. Check for yourself how far project design and implementation practice is removed from what systemic change and evidence-based policy are capable of!
Taylor, B. (2013). Evidence-based policy and systemic change: conflicting trends? (No. 1). Springfield Working Paper Series (p. 28). Durham, UK. Retrieved from http://www.springfieldcentre.com/wp-content/uploads/2013/07/Evidence-Based-Policy-and-Systemic-Change1.pdf
Williams, B., & van ’t Hof, S. (2014). Wicked Solutions: a systems approach to complex problems (v. 1.03., p. p. 97). [Lower Hutt]: Bob Williams. Retrieved from http://gum.co/wicked