Champions of data-quality management

A couple of weeks ago I posted the blog, "Winning the Data Quality Battle" in which I identified the main barriers to DQ and three initiatives to improving it. I promised it was to be the first blog in a series giving advice about winning the data quality battle. I'm sure you've all been waiting with baited breath for my follow-up ;). Due to work commitments it's taken a little bit longer than I'd have liked :(. But, hooray, this week I've actually had a chance to get it down on paper, or bytes. Today I'm going to talk about one of those key initiatives; the DQ manager. So without further ado, ladies and gentlemen, let me present the second in my series about DQ management: "the Champions of Data-Quality Managament".

In our experience, at Semanta, we've found that having a weak DQ manager often leads to the continuation of data-quality problems regardless of the data-quality processes set up within a company. A strong personality and a capacity to wield political power are key aspects for success in any management position but in the case of data-quality this is even more so. Why is this?


Get off my turf!

Let's begin with the politics. The key challenge to improving data-quality in any company is:

  • its continuous character and

  • the fact that it relates to all parts of the enterprise architecture.

That part about relating to all aspects of the enterprise architecture is key here. When a DQ issue occurs in a particular system of the enterprise architecture, its cause very often lies in another system which provides the input data. Because of this, anybody attempting to improve a company's DQ has to tackle problems across departmental borders. We all know how hard it can be coordinating work within a single department. When it's necessary to set up a process across departments...well, I think you can see the problem.

  • Each system is managed by a specific department of the company, cooperating with a specific supplier.

  • To track and remedy such a DQ situation is often a laborious activity. It has a non-trivial budget and usually requires the initiation of a regular project within the company's processes in order to solve it.

  • The rhythm of company projects is normally so slow that often the DQ problems arise faster than their solutions - if the solutions are being created via a standard project.



Solutions, anyone?

The best practice for implementing a solution to these problems is twofold.

Our first recommendation is to introduce a data-quality knowledge management system that's available throughout the company and also to its suppliers. I'll write more about this in a future blog.

Our second recommendation is for all these departments and suppliers to respect and use a single data-quality methodology when implementing changes to their systems. That methodology must be clearly explained and must be consistently implemented by all parts of the company.

Sounds easy enough? Well, here comes the hard part: telling people what to do is as simple as writing a memo. Making sure that they understand and then do it, is a something else altogether.


The buck stops here

Which brings us down to the talents and skills of our DQ champion, the Data-Quality manager. The ultimate responsibility for making all this happen falls on his/her shoulders.

Let's look at that DQ knowledge management system I mentioned above. Such a system should allow the DQ manager to set the rules about DQ checks and monitor how the data-quality is handled in the different parts of the company. It should also provide a clear structure for the different departments when dealing with data-quality (telling them what to document according to which guidelines and what to require from their suppliers etc.). But it's only a tool. And like all tools it requires a good leader to encourage and train people to use it appropriately. So the DQ manager needs to be strong and politically capable. He/she needs good diplomatic skills as well as a strong position in the organisation's hierarchy.  

Training staff and ensuring compliance with the methodology is one thing but the DQ manager also needs to oversee the system and be capable of spotting problems. Beneath the DQ manager there should be a network of data stewards responsible for DQ quality in the different parts of the company's system. However, if there were to be a systematic repetitive DQ issue that a steward didn't pick it up, the DQ manager is ultimately responsible. So he/she needs to be in everyday contact with the current data-quality issues. When there is a systematic repetitive occurrence of a data-quality issue with respect to a data-object (or report) the DQ manager should be capable of introducing a systematic solution. Doing this isn't for shrinking violets. The success or failure often depends on the determination and communication ability of the manager.

Finally there's the budget issue. As we mentioned, the rhythm of company projects is often so slow that DQ problems often arise faster than their solutions - if the solutions are being generated through a normal project. The DQ manager needs to have access to a significant budget (and the authority to use it without any delay) so he/ she can finance on-going DQ improvements regardless of project budgets.



So, as you can see, the DQ manager needs to be some kind of superhuman - or at least demi-human :). Well, I don't think it's that bad! But I do think that companies often underestimate what's required for a successful DQ management process and the power and strength of personality of the DQ manager is the key to not only successfully implementing one, but also communicating to senior management what exactly is required. Companies need to give the DQ manager the authority to use a significant budget without having to go through a slow approval processes. Furthermore, (s)he needs the power to instruct other departments in the company to respect the data-quality methodology which (s)he creates and maintains. So, our DQ manager does need to be strong and politically savvy, but companies need to give them the tools they need to work with.

Read also the first part of this blog series: Winning the Data Quality Battle and the third part Strong data-quality knowledge management system.