Systemic wisdom versus statistic evidence

Can evidence-based and systemic change be combined?

Wisdom is an unattainable ideal   This is so because ideals are abstractions, which means that they are only ideal in as far as we can ignore their context. In practice, we can never ignore an ideal’s contingent reality. That’s probably why the Greeks coined the term ‘philosophy’, the love of wisdom. It does mean, however, that ideals – albeit unattainable – can and should be pursued. Although no philosopher perhaps, a systems thinker takes a slightly different angle in that he/she considers wisdom unattainable, because no matter how hard we try, reality will always remain stubbornly complex, way beyond our understanding. This is both frustrating and exciting. With systemic problems abounding, it is also our guaranteed escape from tedium.

Wisdom and the systems approach   According to Churchman, ideals give meaning to our objectives and goals, both personally and professionally. Now, practical wisdom – as opposed to the wisdom of academia – can be pursued using the systems approach. In a sense the systems approach IS practical wisdom. The wisdom part of the systems approach is that it is able to design interventions (plans, projects, enterprises) to suit multiple ideals (to some extent). The practical part is that it has an approach to do so. This involves looking at multiple perspectives and their underlying motivations. The systems approach recognizes the relative ineptness of any of the resulting plans, but it thinks there is no better way of achieving wisdom in this world.

Knowledge pyramidWisdom in the knowledge pyramid…      – a.k.a. the DIKW or wisdom hierarchy – is a model for representing the relationships between Data, Information, Knowledge and Wisdom. Ackoff posits understanding as a distinct layer between knowledge and wisdom (see below for my concept map of the model). He refers to understanding as an “appreciation of ‘why'”, and wisdom as “evaluated understanding”. Wisdom is the ability to increase effectiveness. Effectiveness is what justifies action. Justification is the holy grail of the systems approach. Wisdom adds value, which requires the mental function that we call judgment. The ethical and aesthetic values that this implies are inherent to the actor and are unique and personal. As described elsewhere in this blog, judgment is crucial in design, whether it is the design of interventions or the design of art or artifacts.

Wisdom hierarchy

Systemics or statistics       In recent years there has been an avalanche of attempts to develop evidence-based policies or interventions, with Banerjee and Duflo among the best known proponents. It hardly befits me to criticize that approach, not in the least because Duflo has received praise from the most credible and highly revered sources such as Nobel Laureate Amartya Sen. Yet, I feel a great deal of unease with this evidence-based approach. Now why is that?

Opposites by necessity     The first point of unease (A) is that it doesn´t fit very well in the above knowledge model: statistics, being data-based, are at the extreme opposite end of wisdom in the knowledge hierarchy. Of course I can imagine statistical designs that compare variants of “wise” policies, but that: (1) would narrow the base for one’s evidence; (2) still necessitates some form of wisdom (whence does it come?); and (3) suggests that the statistician is impregnable to criticism because of the hard, “scientific” evidence he/she produces.

Human medicine derivative     The second point (B) is that the evidence-based policy model is derived from a medical model. It all started with the work by Archie Cochrane in the 1970s. Since then health policy research started looking into the possibility of developing the human medical model into a societal (or public) policy and management model. What we forget is that medicine deals with the human body, which is a much more homogeneous system than societal or management systems. Besides, human medicine is an incredibly productive field of science, at least in terms of the number of publications for each of the multitude of medical specialties. There was both a need and a great opportunity for systematic reviews. Health systems, too, are pretty homogeneous. It is the same model that has been exported everywhere. And now this initial homogeneity is guaranteed to persist or return as a result of this new, homogenizing, evidence-based policy making.

Livelihood approaches cannot be top-down     The third (C) point is that livelihoods – and therefore poverty alleviation efforts – are not only highly location-specific (i.e. the opposite of homogeneous), but also highly sensitive to the effectiveness of poverty interventions. This is crucial because it implies that a slight failure can have devastating effects on the “beneficiaries”. There seems to be a great risk that in many circumstances the intervention will deteriorate the conditions of the poor, rather than improve them. I doubt that the statistical designs used in evidence-based approaches take this aspect into account (my concern was justified, see Taylor, 2013, below). This is all the more important, because the evidence-based approaches seem to suffer of the problem of top-down approaches generally, namely that they ignore or eliminate local agency, my fourth (D) and final point.

The systems approach differs.…       in that it is not statistical at all, although it is not opposed to using statistics to develop good understanding of a problematic situation. Instead it looks at any aspects that seem relevant, whether they are measurable or not. Indeed, a great many important aspects cannot be measured, as is the case when we ponder the question whose knowledge should not be ignored in a design. Evidence-based statistics must by necessity decontextualize design, whereas systems thinking tries to sweep in as much context as possible with a view to make a coherent whole of all design elements by taking into account underlying aspects that are even harder to measure – perish the thought! – such as world views, values, and paradigms. This process of making a coherent whole is also known as ‘unfolding’, so that’s in summary the systems approach: sweeping in and unfolding.

The best of both worlds       If Taylor is right in identifying evidence-based policy making and systemic approaches to development as important recent trends, then why not combine the two in order to fill each other’s voids? No matter what else statistics does, it helps us look closely at relationships and may help us avoid assumptions that are clearly untrue. And no matter what else systems thinking does, it can give us a fair idea of the whole picture. It may even help design better systematic reviews or analyse existing data in an alternative way by developing new, more systemic solutions (or lenses of “inquiry”). The systems approach (see Williams & van ‘t Hof, 2014) can also be used for feed-back or evaluation. The systems approach can be particularly good at asking which measures for success to use and many other things. Considering that politicians and managers the world over prefer “silver bullets” over quirky ones – particularly for reasons of upscaling cost-effectively – it might also be possible to develop and test evidence-based framework solutions that leave more room for local, systemic agency. It’s just a thought. They happy, I happy.

Final remarks      For a long time I have wanted to write something about evidence-based policy making, so this is it. Brief as it may be. Incidentally, during writing,I came across the paper by Ben Taylor (2013). It shows some parallels with this blog post. Also, the Springfield Working Paper suggests that systemic change is more mainstream in international development than I thought. Interestingly, systemic change seems to be poorly understood and, judging by the cases, not very well implemented. This actually applies not only to the systemic change projects, but to the evidence-based approaches, too. Check for yourself how far project design and implementation practice is removed from what systemic change and evidence-based policy are capable of!

Taylor, B. (2013). Evidence-based policy and systemic change: conflicting trends? (No. 1). Springfield Working Paper Series (p. 28). Durham, UK. Retrieved from http://www.springfieldcentre.com/wp-content/uploads/2013/07/Evidence-Based-Policy-and-Systemic-Change1.pdf

Williams, B., & van ’t Hof, S. (2014). Wicked Solutions: a systems approach to complex problems (v. 1.03., p. p. 97). [Lower Hutt]: Bob Williams. Retrieved from http://gum.co/wicked

Advertisements

About Sjon van ’t Hof

Development professional who worked in rural development, tropical agriculture, and irrigation development in Chad, Zambia, Mali, Ghana, Mauritania, Israel, Burkina Faso, Niger, and the Netherlands in capacities ranging from project design and management to information management. Conducted missions to India, China, Kenya, and Bangladesh. Experience in the development and delivery of trainings in irrigation equipment selection, information literacy, Internet searching and database searching. Explores systems thinking in relation to international development, education, and management, with an ever stronger focus on the systems approach of C. West Churchman. Knowledgeable in tropical agriculture, project design and development economics, agricultural mechanization, irrigation, plant pathology, environmental degradation and protection, rural development. Co-authored "Wicked Solutions: a systems approach to complex problems", a book written by Bob Williams and Sjon van 't Hof. It was published in June 2014 and provides a practical way of dealing with wicked problems. Wicked problems are complex, ill-structured, human problem situations. This book will help you design an inquiry and intervention in such messy, wicked situations. It does so by guiding you through the steps and stages of a systemic process that addresses your own wicked problem. For more information, see https://csl4d.wordpress.com/ or http://www.bobwilliams.co.nz/Systems_Resources.html
This entry was posted in General. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s