PA Consulting Group's Bettina Pickering outlines a policy for improving data quality and accuracy.
In today's increasingly competitive economic environment, companies are under pressure to be more cost-efficient in their use of data and information. In addition, many organisations have failed to achieve the benefits they expected from implementing a range of systems, such as ERP, CRM and SCM.
A significant number of these implementation ‘failures’ can be attributed to lack of data quality and accuracy. According to the Butler Group, poor data ‘hygiene’ is not only the reason why companies fail to realise benefits from system investments, it is also a significant, ongoing and increasing cost to the business.
A recent Data Warehousing Institute study showed that poor data quality costs US businesses an estimated $600 billion a year – and the real cost could be much higher.
Adopting a data hygiene approach can realise large cost savings.
Such an approach has a far wider reach than data quality or data clean-up exercises.
Whereas data quality is comparatively static and usually seen as a one-off cleansing exercise, data hygiene is dynamic, encouraging continuous data cleanliness and discipline. It involves embedding preventative processes in the organisation and its systems, to avoid future data contamination, as well as implementing organisational measures to ensure continuous high-quality data.
In working with a variety of organisations, PA has repeatedly found areas to improve efficiency and reduce costs through a change of approach to data.
For example, a European paper manufacturer reduced its non-productive material stock by 15% a year, through identifying and removing duplicate and redundant material records between its six European plants.
Action
Many organisations realise they cannot afford to ignore the cost of poor data quality, but most are not comprehensively addressing their data issues.
Companies tend to address data contamination either during a system implementation as a one-off data clean-up exercise, by ‘fire-fighting’ data issues as they arise, or through departmental initiatives carried out in isolation.
These highly focused activities, although successful at the time of delivery, do not eradicate the data contamination root causes or data contamination in other related areas.
Examples where poor data quality has caused unnecessary cost and operational inefficiencies include a manufacturer which needed three person-days to carry out spend analysis on just one supplier; an accounts payable clerk and service engineer of an engineering company who took an hour and a half to locate a vendor in the purchasing system; and a European manufacturer that stopped production of a product that made limited contribution at the point of sale, only to discover that the product was highly profitable for its service function.
It is vital therefore that organisations take a disciplined and practical approach to tackling data hygiene. This is supported by the Data Warehousing Institute’s study where 78% of respondents said their organisations needed additional education about the importance of data quality and methods to maintain and improve it.
There are three key components for a successful data hygiene programme:
1. Recognise the symptoms and the areas of data contamination.
The main problem areas depend on the way data is stored and distributed. To eradicate data contamination, an organisation must recognise which of its business issues are caused by contaminated data and identify the root causes and the main contamination areas.
This recognition process is also important in helping the organisation’s decision makers realise that quick-fix fire-fighting exercises will not be enough to remove the issues.
The business issues and the symptoms of underlying data problems vary widely depending on systems landscape, organisational culture and industry. However, all issues caused by contaminated data must be recognised and traced back to their root cause.
2. Prepare a sound business case and gain management support.
Many businesses, despite recognising the significance of data hygiene, find it difficult to quantify the benefits of conducting and embedding a hygiene programme in their organisation.
It can also be both expensive and time-consuming to gather detailed quantitative benefit information, especially as benefits are mostly hidden in other costs or process failures.
In developing the data hygiene business case, it is important to make the issues come alive for the staff who inadvertently cause the problem.
Building the business case around specific examples can also make it compelling for senior management. For example, the purchasing director of a German organisation discovered through the business case that by cleaning the supplier master records, he could view total supplier spend – something he could not achieve even manually before.
To make the business case even more compelling, it is vital that it is owned or sponsored by a board member. Data contamination is an issue usually caused by the business – and to solve it permanently, ‘the business’ has to change its ways of working and should take the lead in the process.
It is essential that IT is closely involved with the project but does not take the leading role, as many businesses do not fully buy into an IT-led data hygiene project.
In building the business case, it is advisable to ensure that quick, medium and long-term wins are identified, to enable the organisation to stagger the benefits over a period of time. This will allow the project to pay for itself, which will guarantee board approval.
The qualitative benefits of data hygiene should also not be forgotten. Usually, senior management consider quantitative benefits in deciding whether a project should be undertaken. But qualitative – that is, non-cash – benefits have a major impact on the cost drivers of an organisation as well as the willingness of employees to support the project.
Such qualitative benefits include:
- Improved management decision making. More accurate and reliable reports can be trusted and used as a sound basis for decision making.
- Increased marketing and cross-selling opportunities. Establishing a ‘single customer view’ in one repository enables organisations to exploit marketing and cross-selling opportunities.
3. Embed data hygiene processes and data ownership in the organisation rather than just implement them.
Many organisations that have completed a data cleansing project often struggle to maintain data quality in the long term – which is the most important part of data hygiene.
Embedding hygiene principles in the organisation, its processes and systems is vital to ensure ongoing data quality.
There are a number of key organisational and process-related factors that are key to continuous data quality; and the most important aspect of embedding something in an organisation is to make people responsible for it.
It is useful to appoint staff to the following three roles:
- A single senior-level business sponsor who has a clear understanding of the value of maintaining data quality.
- A data quality representative for each area or department who acts as a single point of responsibility and source of information on data quality.
- A single business and a single IT owner for each data set, who should be management-level with specific knowledge of the particular data set(s).
The most time-consuming part of a data hygiene project is the largely manual data cleaning. To avoid having to go through such a cleaning exercise again and again, organisations should consider the following:
- Involve users early and on an ongoing basis in developing data cleansing and matching rules, sign-off criteria, design of processes, etc.
- Data-related processes, rules and structures must be defined and rolled out, before any data cleaning takes place.
- Automate or embed data hygiene processes in the IT systems – for example, by using system-supported gatekeeper programs, which check for duplicates before a record is saved to the database.
Organisations should implement workflow software programs to embed data hygiene processes in their IT systems. This supports a decentralised, low-cost and very lean data quality organisation and does not take away the onus from the individual to get data right first time.
Workflow programs, especially if they are part of an existing ERP system, are excellent for providing audit trails, alerts when things go wrong, and metrics and analysis on common mistakes whilst capturing ‘data polluters’.
The best approach is to understand how the pollution occurred, as most users will not pollute on purpose: key reasons are lack of training in data hygiene rules, lack of awareness of the effects of polluted data, time constraints and ineffective search tools. The rewards of data hygiene far outweigh the effort
Conclusion
In the current economic climate and with competition ever increasing, solving data hygiene issues will provide a company with savings going straight to the bottom line.
Meanwhile, legislation is not standing still. US organisations have to comply with the Sarbanes-Oxley Act on corporate accountability, introduced in the wake of the Enron and WorldCom scandals, and the European Banking Industry has to prepare for compliance with the second Basel Capital Accord, designed to improve the safety and soundness of the financial system.
Data hygiene is absolutely essential to enable organisations to comply with these two new pieces of legislation.
A successful data hygiene programme depends on three key components:
- Diagnosing the symptoms correctly.
- Preparing a compelling business case commissioned by the right sponsor.
- Embedding data hygiene processes and measures within the organisation and its IT systems.
Data hygiene is not easy to achieve and may take up to two years, depending on the complexity of the organisation, its culture, its system landscape and the amount of data it holds.
But organisations should understand that the rewards of successfully embedding data hygiene principles right across their organisation will far outweigh the cost and effort required to successfully complete the programme.
Managers can utilise reliable data to make more informed decisions and eliminate some operational inefficiencies, improving the organisation’s long-term performance.
Bettina Pickering is a senior consultant with PA Consulting Group
Tel: 020 7730 9000
Email: bettina.pickering@paconsulting.com
