From Data, with Love

That most hallowed of mental models and glib explanations, the Data-Information-Knowledge-Wisdom hierarchy has taken a bit of a beating this week. It started in an innocent enough way when, in a discussion about knowledge sharing and generation on the KM4Dev listserve, somebody cited the DIKW model as a way of describing how knowledge is generated in organisations. This provoked Dave Snowden into some sharp but illuminating posts (by the way, if you ever get bored and feel like doing some Dave-baiting, get yourself a false identity, sign up to one of the listserves he frequents, and make an enthusiastic post about DIKW, wisdom management, Six Sigma, Ayn Rand or KM certification - or any combination thereof):

“I would reject the DIKW pyramid, aside from the fact it’s just plain wrong, it’s difficult to explain and leads to bad labels. Better to think that KNOWLEDGE is the way we create INFORMATION from DATA. If we share knowledge then we can understand information.”

“Aside from being linked to a particular period of systems thinking approaches, which we are hopefully moving on from, its very culturally specific.  It fails entirely to account of shamanistic knowledge, or the narrative traditions of Sufi philosophy and others.  I could go on, but the you get the point; the DIKW pyramid is a culturally limited and inadequate model which has done more harm than good. The SECI model with its de facto focus on codification comes a close second, as I said the other day it’s the model that launched a thousand failed knowledge management initiatives.  The main problem is its tendency to get people to think of knowledge as a thing rather than as a flow.”

Dave has posted in the past at greater length on DIKW here and here, and so have I.

However, one worried comment from a listserve member that DIKW was a “well-understood idea within the community” struck me, and prompted a further reply from me - because indeed this hierarchy is extremely well entrenched in the KM (and information science) literature. It’s about as sacred as a sacred cow can get. Why? And should that make it immune to attack posts?

Here’s my reply, slightly modified for a wider audience:

image

It’s important to understand the origins of a model to understand what it was designed for. The DIKW model emerged out of the struggles of computer science and information science through the late 1970s and early 1980s to legitimise themselves as strategic disciplines for the enterprise. For the data managers, the struggle was to get their organisations to treat data as a strategic resource, so establishing a relationship to information that fed decisions based on knowledge made a lot of sense. For the information managers the “downwards” link to data gave them a structure to work from, and the “upwards” link to knowledge gave them legitimacy in the eyes of senior management.

So while it had utility for data and information managers, the hierarchy was never designed to accommodate the far more complex world uncovered by knowledge management, and as Dave points out, it completely fails to acknowledge the naturalistic ways that data, information and knowledge interact. For example, it does not reflect the fact that data is a very small subset of repeatable information, abstracted and structured for mechanical processing based on knowledge. Data is the product of a knowledge-driven, purposeful piece of design work. The DIKW model implies the opposite, that knowledge is the product of a series of operations upon data. The model also completely fails to account for the sea of knowledge activity in an enterprise which is never informationalised or structured as data. In the natural world, data is the product of a very small component of knowledge activity.

From the data manager’s point of view, the problem in the enterprise is “we have all of this data sitting around, think of what we could do with it if we could figure out how to squeeze insight out of it”. While this is a legitimate question, the knowledge manager has discovered rather painfully, that you need to go back to the contexts that created the data and the knowledge activities the data supports, in order to figure out how the data can be manipulated for greater advantage. You can’t get there by performing a series of logical transformations on the data to create information, and then another series of operations to create knowledge.

So DIKW is a managerial model intended to explain how data can be leveraged as an enterprise resource. It has no practical value for guiding action beyond the D-I interface where it has limited value, explains almost nothing about knowledge, and its references to wisdom have always been completely without substantive or actionable content.

Why did the hierarchy become received “wisdom” in KM?

(1) It became received wisdom very quickly in computer sciences and information science literature, because it was a legitimising model – such models become entrenched very quickly.

(2) The weaknesses of the model in relation to knowledge and wisdom were never tested in its first decade by which time it had become entrenched in the literature.

(3) Writers for new knowledge management journals in the 1990s – as in any new discipline – suffered from “citation poverty” and so fell back on the received literature and mental models from their parent disciplines, without adequately questioning their applicability in this new context.

(4) If you don’t actually try to do anything based on the model, it serves a quite useful function in proffering a glib explanation of the distinctions between data information and knowledge and makes a pleasing nod at wisdom, so it seems like it has utility.

(5) If you do try to structure your KM work using the model, you get rapid support from the technology side of KM (so the model must be ok), and when you run into problems with ground adoption and usability, it’s easy to chalk this up to human intransigence and change resistance, rather than the poverty of the model as a framing device.

The pyramid form implies that data – at the base – is more abundant than information, which is more abundant than knowledge, which is more abundant again than wisdom, at the very tip. Indeed, from an enterprise perspective that might seem to be the case, but I happen to think it’s mistaken. My own view is that in most cases there’s a lot more knowledge (in and around the people) than information, and even less data. I won’t comment on wisdom.

One constructive way to read the DIKW pyramid is in terms of the VISIBILITY and TANGIBILITY of the different elements, which is a different thing from their presence. From that perspective, the visual representation of the pyramid makes sense. We do see a lot more data, it’s easier to figure out where it is, and what to do with it. Information is less transparent and more complex to audit and map, knowledge is much more opaque, and wisdom auditing (people do actually sell this!) would be either a work of opinion or divination. So if DIKW just made claims about visibility it would have some use.

That may be another reason why the DIKW pyramid has seemed so attractive: the visual form focuses us first on the more manageable (visible and tangible) elements and encourages us to work on those first as foundational elements – providing a visual justification for a quick win bias. The problem is that the knowledge ecosystem is more complex than DIKW allows, and focusing energies and effort on the easier stuff frequently fails to meet the most critical needs. The critical stuff is just not “seen” through a DIKW lens.

It’s interesting to note that some information scientists have recently been reassessing their attachment to their offspring:

image

Anti-DIKW convulsions seem to have their echoes in the noosphere that connects us all. In almost exact synchronicity with the KM4Dev discussion, David Weinberger was posting on “The Problem with the Data-Information-Knowledge-Wisdom Hierarchy” over at the HBR blog (thanks Nancy, and Jaap). He makes very similar points to myself and Dave (although he seems to have borrowed heavily from Nikhil Sharma’s fine paper on the origins of the hierarchy without crediting it – tsk tsk).

I think he’s got it wrong on a minor point: he characterises the hierarchy’s rapid acceptance as a backwards justification of systems that had been built, where I think the emergence of the hierarchy and its acceptance were more of a forwards justification to legitimize data and information management as strategic concerns. But his closing line is nicely put: “Knowledge is more creative, messier, harder won, and far more discontinuous [than the pyramid implies].”

Then I was directed by Simone Staiger to a discussion over at the Agricultural Biodiversity Blog arising out of an example where a youtube video showing trial results from a taro breeding programme in the Dominican Republic was posted on a taro researcher’s Facebook wall with a request for an English translation. He got his translation and found out which of the hybrids were the most promising, along with more information about what was going on with hybrid breeding in Puerto Rico. It’s one of those great, ubiquitous, social media knowledge sharing stories.

The telling part (and the bit that started the debate) was the final line of the post: “Who needs databases?”

Go visit the story to see the full discussion, it’s a very good one. But what struck me was that, in starting a fight over the DIKW-blinkered-view, there are three risks we need to watch out for:

(1) Irrational Prejudice: that we stimulate an “anti-data-mania” along the lines of “structure (data) is totalitarian control and freedom (knowledge/wisdom) is the democratic way”. We’ve seen that already with an anti-expert backlash, and the foolish notion that folksonomies can replace taxonomies. We need data – and structure. They are precious. Data allows us to track and manipulate information about salient features of important things in our world. Like epidemics. Like indicators of weather and climate patterns. Like the efficacy of pharmaceuticals. Like customer behaviours.

(2) Old Mental Models in New Dress: that we “knowledge-people” maintain the prevailing “them and us” attitude towards the “data-people”. I.e. we maintain the same tribal stratification embedded in the DIKW pyramid itself. A consequence is that we treat data just as a servant or instrument of knowledge rather than also as a potential environment for knowledge – one offlist comment that struck me was “one intelligent person can get far more “information” or even “wisdom” from a stream of “data” than any machine” (thanks Jeremy). We run the risk of assuming that data is structure, and so the interfaces with data must also be structured. Jeremy’s comment reminded me that data can also be rich in meaning when we can play in it in less structured ways – though we need to be much more skilled and ethically aware when we do this.

(3) Alienation of Professional Colleagues: that the attack is seen as an attempt to delegitimise data and information management, driving professionals in that field into a defensive mode rather than encouraging the related disciplines to work harder on how we can collaborate constructively with each other. We also need to work on how the different manifestations and artefacts of knowledge can be better represented and supported – in both structured and networked ways.

What does this mean? We need a better conceptual model, certainly, for explaining the elements of data, information and knowledge (and keeping wisdom well out of it), and how they interact. Deeper than that, we need to address the legitimising need of the players in data, information and knowledge management. We need a social model for how the disciplines interact in the service of the enterprise (or the community, or society). If we don’t address this need, we’re just children throwing stones, across lines in a pyramid, at varying levels of abstraction.

image

11 Comments so far

Nancy White

Nice weaving together of many strands. I’m processing some work this week in my mind and I want to pick up a thread in my blog. Now I just need another macchiato!

Posted on February 06, 2010 at 05:02 PM | Comment permalink

nick milton

My current favourite variant is

Data - Information - Knowledge - Understanding - Decision - Action

Posted on February 08, 2010 at 12:43 AM | Comment permalink

Interesting. While I comprehend Dave’s reasons for disliking the DIKW hierarchy, I find it doesn’t bother me in the slightest. Not being professionally involved in data management (other than organising my own thoughts into generally coherent text for clients), I’d derived the hierarchy from first principles. To me it simply summarises the way we seek to understand our environment; I don’t see it as ‘culturally specific’. I suspect that Dave’s viewpoint is itself somewhat ethnocentric on at least two levels.  Everyone, information management professional, shaman or Sufi philosopher, takes a logical approach to understanding their environment; it’s just that the pattern of logic and the logical explanations vary from culture to culture.

Thank you for providing me with a reason to think beyond my work this morning.

Posted on February 09, 2010 at 05:24 PM | Comment permalink

John Tropea

Great comprehensive review and insight thx Patrick.

Here’s my review and insight from a past post
http://libraryclips.blogsome.com/2008/06/26/knowledge-as-interpreter-aspe/

Posted on February 10, 2010 at 10:57 AM | Comment permalink

Patrick Lambe

Here’s a great example of a local community in Haiti using data gathered from the community to analyse what needed to be done to respond appropriately to the earthquake damage. http://www.haitipartners.org/2010/01/analyzing-situation-and-needs/

Posted on February 10, 2010 at 11:28 AM | Comment permalink

Nancy White

Hey, that’s our friend John Engle, who is part of the global Open Space community!

Posted on February 11, 2010 at 08:17 AM | Comment permalink

Patrick Lambe

Yup, I followed a trail from one of your posts… his use of video podcasts is extremely powerful.

Posted on February 11, 2010 at 09:13 AM | Comment permalink

John England

If you leave the “wisdom” bit off the basics of the model are OK for me. The issue is that so often in discussion it seems almost that knowledge is absolute and this is something I do not subscribe to ... let me explain:

If we start from Chaos theory and consider the universe full on data in random, motion. Over time patterns appear and disappear and these patterns form “information”. When that information is brought together or synthesised, in an appropriate manner, we get “knowledge”. This is then ¾ of the model under discussion:

Data (cloud)--> Information (pattern) --> knowledge (synthesis)

However, this approach clearly means that knowledge, by its very nature, is transient. What can be classed as “knowledge” today may not be seen as “knowledge” tomorrow. In this lies the problem with so may knowledge management systems in that they store so call “knowledge” as if it were sacred rather than allow the synthesis of knowledge as appropriate. If that is the accepted interpretation they the model is OK by me (less the “wisdom” bit which I think is a bit pretentious!)

Posted on February 22, 2010 at 03:07 PM | Comment permalink

Stephen Dale

The DIKW model has never really troubled me, but maybe that’s because I’ve traversed the IM to KM chasm as part of my professional life. However, if ever I’ve referred to the model I’ve changed ‘Wisdom’ to ‘Insight’, because that infers a process of understanding (and clearly a link here to Dave Snowden’s sense-making models), whereas Wisdom seems far more static, as if it develops through osmosis.

Posted on February 22, 2010 at 05:04 PM | Comment permalink

An intersting discussion. I have used the model and variants to engage with business colleagues and clients. Most are interested in the outcomes and impact of KM activity so I always included ‘results’. So for Nick’s current favourite variant read Data - Information - Knowledge - Understanding - Decision - Action - RESULTS.

Posted on February 22, 2010 at 06:31 PM | Comment permalink

What seems to be missing from the discussion is any attempt to distinguish between knowledge which has undegone some test of veracity, as against knowledge which is merely proclaimed to represent the truth and may be faith rather than reason based.  Understanding the difference is vital to the KM discourse.

Posted on February 22, 2010 at 10:08 PM | Comment permalink

Page 1 of 1 pages

Commenting is not available in this weblog entry.

Comment Guidelines: Basic XHTML is allowed (<strong>, <em>, <a>) Line breaks and paragraphs are automatically generated. URLs are automatically converted into links.