“I would reject the DIKW pyramid, aside from the fact it’s just plain wrong, it’s difficult to explain and leads to bad labels. Better to think that KNOWLEDGE is the way we create INFORMATION from DATA. If we share knowledge then we can understand information.”

“Aside from being linked to a particular period of systems thinking approaches, which we are hopefully moving on from, its very culturally specific.  It fails entirely to account of shamanistic knowledge, or the narrative traditions of Sufi philosophy and others.  I could go on, but the you get the point; the DIKW pyramid is a culturally limited and inadequate model which has done more harm than good. The SECI model with its de facto focus on codification comes a close second, as I said the other day it’s the model that launched a thousand failed knowledge management initiatives.  The main problem is its tendency to get people to think of knowledge as a thing rather than as a flow.”

Dave has posted in the past at greater length on DIKW here and here, and so have I.

However, one worried comment from a listserve member that DIKW was a “well-understood idea within the community” struck me, and prompted a further reply from me - because indeed this hierarchy is extremely well entrenched in the KM (and information science) literature. It’s about as sacred as a sacred cow can get. Why? And should that make it immune to attack posts?

Here’s my reply, slightly modified for a wider audience:

It’s important to understand the origins of a model to understand what it was designed for. The DIKW model emerged out of the struggles of computer science and information science through the late 1970s and early 1980s to legitimise themselves as strategic disciplines for the enterprise. For the data managers, the struggle was to get their organisations to treat data as a strategic resource, so establishing a relationship to information that fed decisions based on knowledge made a lot of sense. For the information managers the “downwards” link to data gave them a structure to work from, and the “upwards” link to knowledge gave them legitimacy in the eyes of senior management.

So while it had utility for data and information managers, the hierarchy was never designed to accommodate the far more complex world uncovered by knowledge management, and as Dave points out, it completely fails to acknowledge the naturalistic ways that data, information and knowledge interact. For example, it does not reflect the fact that data is a very small subset of repeatable information, abstracted and structured for mechanical processing based on knowledge. Data is the product of a knowledge-driven, purposeful piece of design work. The DIKW model implies the opposite, that knowledge is the product of a series of operations upon data. The model also completely fails to account for the sea of knowledge activity in an enterprise which is never informationalised or structured as data. In the natural world, data is the product of a very small component of knowledge activity.

From the data manager’s point of view, the problem in the enterprise is “we have all of this data sitting around, think of what we could do with it if we could figure out how to squeeze insight out of it”. While this is a legitimate question, the knowledge manager has discovered rather painfully, that you need to go back to the contexts that created the data and the knowledge activities the data supports, in order to figure out how the data can be manipulated for greater advantage. You can’t get there by performing a series of logical transformations on the data to create information, and then another series of operations to create knowledge.

So DIKW is a managerial model intended to explain how data can be leveraged as an enterprise resource. It has no practical value for guiding action beyond the D-I interface where it has limited value, explains almost nothing about knowledge, and its references to wisdom have always been completely without substantive or actionable content.

Why did the hierarchy become received “wisdom” in KM?

(1) It became received wisdom very quickly in computer sciences and information science literature, because it was a legitimising model – such models become entrenched very quickly.

(2) The weaknesses of the model in relation to knowledge and wisdom were never tested in its first decade by which time it had become entrenched in the literature.

(3) Writers for new knowledge management journals in the 1990s – as in any new discipline – suffered from “citation poverty” and so fell back on the received literature and mental models from their parent disciplines, without adequately questioning their applicability in this new context.

(4) If you don’t actually try to do anything based on the model, it serves a quite useful function in proffering a glib explanation of the distinctions between data information and knowledge and makes a pleasing nod at wisdom, so it seems like it has utility.

(5) If you do try to structure your KM work using the model, you get rapid support from the technology side of KM (so the model must be ok), and when you run into problems with ground adoption and usability, it’s easy to chalk this up to human intransigence and change resistance, rather than the poverty of the model as a framing device.

The pyramid form implies that data – at the base – is more abundant than information, which is more abundant than knowledge, which is more abundant again than wisdom, at the very tip. Indeed, from an enterprise perspective that might seem to be the case, but I happen to think it’s mistaken. My own view is that in most cases there’s a lot more knowledge (in and around the people) than information, and even less data. I won’t comment on wisdom.

One constructive way to read the DIKW pyramid is in terms of the VISIBILITY and TANGIBILITY of the different elements, which is a different thing from their presence. From that perspective, the visual representation of the pyramid makes sense. We do see a lot more data, it’s easier to figure out where it is, and what to do with it. Information is less transparent and more complex to audit and map, knowledge is much more opaque, and wisdom auditing (people do actually sell this!) would be either a work of opinion or divination. So if DIKW just made claims about visibility it would have some use.

That may be another reason why the DIKW pyramid has seemed so attractive: the visual form focuses us first on the more manageable (visible and tangible) elements and encourages us to work on those first as foundational elements – providing a visual justification for a quick win bias. The problem is that the knowledge ecosystem is more complex than DIKW allows, and focusing energies and effort on the easier stuff frequently fails to meet the most critical needs. The critical stuff is just not “seen” through a DIKW lens.

It’s interesting to note that some information scientists have recently been reassessing their attachment to their offspring:

• Jennifer Rowley has a good review of how the DIKW model has been used in the literature, and its ambiguities and weaknesses (Journal of Information Science April 2007)
• Martin Fricke has a sustained critique of the hierarchy concluding that it is “unsound and methodologically undesirable” (Journal of Information Science April 2009).
• Nikhil Sharma has a useful overview of the origins of the DIKW hierarchy.

Anti-DIKW convulsions seem to have their echoes in the noosphere that connects us all. In almost exact synchronicity with the KM4Dev discussion, David Weinberger was posting on “The Problem with the Data-Information-Knowledge-Wisdom Hierarchy” over at the HBR blog (thanks Nancy, and Jaap). He makes very similar points to myself and Dave (although he seems to have borrowed heavily from Nikhil Sharma’s fine paper on the origins of the hierarchy without crediting it – tsk tsk).

I think he’s got it wrong on a minor point: he characterises the hierarchy’s rapid acceptance as a backwards justification of systems that had been built, where I think the emergence of the hierarchy and its acceptance were more of a forwards justification to legitimize data and information management as strategic concerns. But his closing line is nicely put: “Knowledge is more creative, messier, harder won, and far more discontinuous [than the pyramid implies].”

Then I was directed by Simone Staiger to a discussion over at the Agricultural Biodiversity Blog arising out of an example where a youtube video showing trial results from a taro breeding programme in the Dominican Republic was posted on a taro researcher’s Facebook wall with a request for an English translation. He got his translation and found out which of the hybrids were the most promising, along with more information about what was going on with hybrid breeding in Puerto Rico. It’s one of those great, ubiquitous, social media knowledge sharing stories.

The telling part (and the bit that started the debate) was the final line of the post: “Who needs databases?”

Go visit the story to see the full discussion, it’s a very good one. But what struck me was that, in starting a fight over the DIKW-blinkered-view, there are three risks we need to watch out for:

(1) Irrational Prejudice: that we stimulate an “anti-data-mania” along the lines of “structure (data) is totalitarian control and freedom (knowledge/wisdom) is the democratic way”. We’ve seen that already with an anti-expert backlash, and the foolish notion that folksonomies can replace taxonomies. We need data – and structure. They are precious. Data allows us to track and manipulate information about salient features of important things in our world. Like epidemics. Like indicators of weather and climate patterns. Like the efficacy of pharmaceuticals. Like customer behaviours.

(2) Old Mental Models in New Dress: that we “knowledge-people” maintain the prevailing “them and us” attitude towards the “data-people”. I.e. we maintain the same tribal stratification embedded in the DIKW pyramid itself. A consequence is that we treat data just as a servant or instrument of knowledge rather than also as a potential environment for knowledge – one offlist comment that struck me was “one intelligent person can get far more “information” or even “wisdom” from a stream of “data” than any machine” (thanks Jeremy). We run the risk of assuming that data is structure, and so the interfaces with data must also be structured. Jeremy’s comment reminded me that data can also be rich in meaning when we can play in it in less structured ways – though we need to be much more skilled and ethically aware when we do this.

(3) Alienation of Professional Colleagues: that the attack is seen as an attempt to delegitimise data and information management, driving professionals in that field into a defensive mode rather than encouraging the related disciplines to work harder on how we can collaborate constructively with each other. We also need to work on how the different manifestations and artefacts of knowledge can be better represented and supported – in both structured and networked ways.

What does this mean? We need a better conceptual model, certainly, for explaining the elements of data, information and knowledge (and keeping wisdom well out of it), and how they interact. Deeper than that, we need to address the legitimising need of the players in data, information and knowledge management. We need a social model for how the disciplines interact in the service of the enterprise (or the community, or society). If we don’t address this need, we’re just children throwing stones, across lines in a pyramid, at varying levels of abstraction.

“It’s a political quagmire. [KM]IA discussions require a horizontal approach to traditionally vertically managed resources. On top of this, business areas tend to feel a strong sense of ownership of the data in their mission-critical applications, and they’re suspicious that any discussions about data usage with “outsiders” could lead to a loss of control.

A very good relationship between IT and the business is a prerequisite for [KM]IA. Overcoming the political difficulties is challenging enough; succeeding when there is a poor track record of communication and trust between IT and the business is even more unlikely.

[KM]IA can look like a boil-the-ocean effort. The data and content mess facing most large organizations is enormous, and any architects who consider getting the enterprise in order quickly recognizes that they will retire before the task can be completed — no matter how young they are.”

Read on in the report for some insightful advice about “street-level-strategy” building to address these challenges – just as good advice for KM as for IA.

There’s one big gap which is not addressed: take a look at the high level view of the enterprise information architecture (shown below) from the report.

What struck me was what was missing: where are the human beings in the framework? “Real” architects never show their models or visualisations without putting in stick figures to show how it works with people in them. Why don’t we? Apparently, this report, unabashedly technical in orientation, has ruffled a few feathers in the more human-oriented IA camps, not least for quoting a reference to them as “Web weenies”.

There’s a reason why user experience folks call themselves information architects, and they’re not going to be expelled from the academy because they don’t fit within a logical array. The parallels with KM sharpen this question for us as well: where does the interface with people’s desires, aspirations, frustrations and needs come into what we do? Where does it fit within our KM frameworks?

Thanks to Nick Berry for highlighting this via the TaxoCop forum.

“2010 might be the year of visualization theory. While our field is certainly an applied one, we still need a much deeper understanding of how it works and how to build better tools. There is some existing work, but much of that is old (Bertin’s work was published in the 1960s, Mackinlay’s almost 25 years ago, Shneiderman’s 13 years ago, Chi’s taxonomy almost ten years ago). The field is progressing and we are developing new tools that do not always fit the old molds. We are also gaining a better understanding of how things work, and we are seeing interesting new concepts from other fields. So an update of our theoretical foundations is really overdue now, and this year will hopefully be when it happens.”

“Since the 2001 anthrax scare in the US, the government here has maintained a list of 80 microbes and toxins that are essentially forbidden to researchers. Now scientists say the list is undermining security rather than strengthening it.

The list is called the Select Agents and Toxins List (SATL), and the microbes on the list are chosen without any input from researchers in a process that is far from transparent. In an article published today in Nature Reviews Microbiology, scientists Arturo Casadevall and David Relman say that the list is hobbling research efforts as well as the nation’s biosecurity. They say that items on the list are almost impossible to get for legitimate research. And in fact, many of the substances are needed for research into vaccines which would protect people from the very bio-attacks the government fears.

Moreover, the scientists take issue with the microbes placed on the list, many of which are chosen based on their taxonomic category. Unfortunately, taxonomy doesn’t always work well with microbes, which can have many different strains of varying toxicity and whose so-called species often overlap. So the list both overreaches and underreaches, missing dangerous strains and including harmless ones.”

Mike Fisher has analysed various social computing tools into Bloom’s taxonomy of learning objectives – ie which tools fit where according to th level or depth of learning you want to achieve.

Olivier Amprimo has a very systematic analysis of how the knowledge organisation requires social computing tools and why, leading to some very practical examples (video of a talk for iKMS).

Ron Young argues that social computing massively enables personal knowledge management, and this in turn puts enormous pressure on organisations to adopt knowledge management practices (video of a talk for iKMS).

“The three principles of transparency, participation, and collaboration form the cornerstone of an open government. Transparency promotes accountability by providing the public with information about what the Government is doing. Participation allows members of the public to contribute ideas and expertise so that their government can make policies with the benefit of information that is widely dispersed in society. Collaboration improves the effectiveness of Government by encouraging partnerships and cooperation within the Federal Government, across levels of government, and between the Government and private institutions.”

They might be principles any knowledge management initiative within an enterprise could sign up to. There’s another development which is interesting: the US Attorney General in 2009 issued new guidelines on how public agencies should interpret the Freedom of Information Act, making it clear that “the default position” with respect to freedom of information is openness.

Too often, I’m working with organisations where the default position (in knowledge sharing) is keep it out of sight unless somebody makes a case for sharing, and then only share with the requestor. I’ve blogged before about how the US Army and State Department are seeking to shift their own default positions. The issue however is that even if senor management adopt the position that knowledge sharing and transparency are good, and come out openly and say that, there are still residual parts of the infrastructure that are designed for the closed way of working. For example, diverse policy documents that are built on the assumption (or actually state that) “all information is confidential” or “information should be shared on a need to know basis”. And from those policies spring an architecture of procedures designed to enshrine and protect that closed way of working. It’s one thing to say, and other things to do.

Which is why I think the OMB directive is worth watching, because it comes with a timeline for action planning and compliance, as well as a dashboard for agencies to report into. Now such high level dashboards can be gamed, but this is a step further than simply expressing a wish for openness and ignoring the trajectory of the current infrastructure. Should we be thinking of our knowledge management plans and strategies more along the lines of an “Open Enterprise Directive”?