Resources for Creating a Data Quality Methodology
To help launch your own data quality initiative, included below are a range of data quality frameworks and methodologies that are freely available online and offline.
These resources can help you accelerate your data quality framework by giving you structure and insights from others who have gone before you.
If you know of any other useful resources that are worthy of the guide - please let us know.
List of Data Quality Framework and Data quality Methodology Resources
New Zealand Ministry of Justice Data Quality Framework
Publicly available here on Data Quality Pro, Mandy Mackay, with the support of the NZ MoJ, has kindly released their comprehensive data quality framework.
This covers the data quality lifecycle, the phases and activities of DMAIC, dimensions of data quality, causes and costs of data quality, checklists, assessment process flow and scorecards.
It’s comprehensive, but not excessive, and a good example of how a well structured framework should be constructed.
The framework is built around the well-proven “Define – Measure – Analyse – Improve – Control” cycle and is an excellent example of how a framework should be tailored closely to the needs of the organisation.
CDQM: The “Complete Data Quality Methodology”
This is a framework presented by Carlo Batini/Monica Scannapieco in their book:
Data Quality – Concepts, Methodologies and Techniques.
Essentially this framework consists of 3 core phases:
State reconstruction
Data Quality Assessment
Data Quality Improvement
One of the most compelling aspects of this framework is the focus on business process and organisation services, this helps focus the efforts on the pain areas where costs are highest.
The book itself provides details of several data quality frameworks and a wide array of data quality concepts so is a useful resource when building your own data quality framework.
Data Quality Assessment Framework – Arkady Maydanchik
Data Quality Assessment (by Arkady Maydanchik).
This book provides a complete data quality assessment and measurement framework that you will be hard pressed to find elsewhere in such detail.
Extremely detailed with clear examples and very much focused on practical techniques that can be easily adopted without the need for expensive tools.
Essential reading for the practical data quality consultant and especially any data quality analyst who is looking to establish a framework for data quality assessment as part of a wider data quality assessment and improvement framework or simply on a per-project basis.
CIHI Information Quality Framework from the Canadian Institute for Health Information
There are lots of details on this resource available online at the CIHI Data Quality page or simply download the latest edition using the button below:
The CIHI framework covers the various dimensions of data quality the CIHI use to manage data assets across their various healthcare systems.
There is some good detail provided for each of the different data quality dimension and although obviously focused on the health sector, much of the content is applicable to most industries.
In addition, CIHI provide the following resources that show how seriously this organisation is addressing data quality:
Enterprise Knowledge Management by David Loshin
Enterprise Knowledge Management: The Data Quality Approach
This framework is provided by David Loshin of Knowledge Integrity and covers components such as building the economic framework for data quality, statistical process control, measurement and current state assessment, root-cause analysis, supplier management and how to implement business rules for data quality.
MIKE2.0
MIKE2.0 (Methodology for an Integrated Knowledge Environment): An Open Source initiative launched by Bearing Point that features an entire Enterprise Information Management framework, incorporating data quality.
There are a number of data quality framework components in the methodology and as it’s Open Source you can simply download the content and even build your own Wiki with the new omCollab facility.
An excellent resource, well worth investigating.
It culminates with a detailed 17 step plan for launching your data quality initiative. A very useful resource for shaping your own data quality methodology or framework.
Danette McGilvray – Ten Steps Data Quality Framework
Ten Steps to Quality Data and Trusted Information: This book was written by Danette McGilvray of Granite Falls Consulting Inc., a highly experienced data quality consultant.
Danette details what is in effect a comprehensive data quality framework aimed at organisations who need to understand the logistics of executing well structured data quality initiatives.
There is also a wealth of additional for free download at her supporting website for the book, just scroll down the GFalls website page until you find the free templates.
Here are some shortcuts:
Individual Downloads
Data Quality Dimensions (PDF)
Data Categories (PDF)
Templates
All Excel Downloads (.xls)
All PowerPoint Downloads (.ppt)
We have written a book review of the Ten Steps book in the past and I think it should definitely feature high on your wish list for data quality framework resources as it is thorough, practical and contains excellent supporting resources.
TDQM from MIT
TDQM: This is a joint effort among members of the TDQM Program, MIT Information Quality Program, CITM at UC Berkeley, government agencies such as the U.S. Navy, and industry partners. Activities in the TDQM program have evolved into four key areas:
TDQM stems from the Total Quality Management (TQM) methodology which developed in the manufacturing sector. To learn more about TDQMvisit this link, TDQM is also available in the form of anIQ Certification Programme.
TIQM from Larry English at Information Impact
TIQM (formerly TQdM) – Total Information Quality Management: This data quality framework was developed by Larry English of Information Impact International Inc.
TIQM is a comprehensive information quality framework that is detailed in what has to be one of the most popular data/information quality books of all time: “Improving Data Warehouse and Business Information Quality“.
Information Impact also provide a comprehensive set of free resources at their website. The TIQM methodology is well proven and many organisations have adopted TIQM for their enterprise-wide data quality initiative. TIQM is available as a certification and training program.
Data Quality Framework and Strategy for the New Zealand Ministry of Health
Data Quality Framework and Strategy for the New Zealand Ministry of Health
Karolyn Kerr produces a very useful report that covers particularly well the challenges of creating a data quality framework.
Karolyn provides additional components to the CIHI Data Quality Framework above and highlights the various data quality dimensions and components to be adopted by the New Zealand Ministry of Health.
GS1 Data Quality Framework
GS1 is a leading global organisation dedicated to the design and implementation of global standards and solutions to improve the efficiency and visibility of supply and demand chains globally and across sectors.
This framework has been written with the input from many retailers, manufacturers, industry associations, certification bodies and other organisations in the supply/demand chain sector. As such, it is encouraging to see so many organisations invest time and resources to compile this very thorough framework.
Although geared towards the supply chain industry, this is a good example of how any industry body or trade association could implement a governing framework for improving data quality standards in its member organisations.
The ‘Data Quality Framework Packet’ (zip file) contains a range of assets including presentations, checklists, implementation guides and more: