Poor data quality poses a major risk to every organisation but how do we measure those risks and take appropriate actions?
In this interview we speak with Alexander Borek, Doctoral Researcher and Lead Developer of TIRM at the University of Cambridge.
Alexander and his team have created a comprehensive approach that enables coordinated activities to direct and control an organization with regard to information risk.
The TIRM approach is gaining recognition from many quarters and will be presented as a one hour keynote in March 2012 at the IBM Service Innovation Camp at CeBIT.
Data Quality Pro: Can you give a brief definition for TIRM and your objectives with the methodology?
Alex Borek: In the recent years, we could observe a high rise in the number of software providers for data quality tools and these software tools are becoming increasingly sophisticated to solve technical problems. One of the big open issues is yet how to align your data and information management better with your business processes to create value. In practice, most managers struggle to measure the business impact of data and information quality. This has the consequence that the choice of data and information that should be improved is based on the “gut-feeling” of the project team members rather than be guided by facts and economical justification, when they try to decide where they should concentrate their efforts on to generate the highest benefits for an organisation.
Now, imagine you could have a management process that allows you to measure how data and information quality impacts your business objectives, both financially and non-financially, in all of the core processes of your organisation. You could determine the costs and benefits of a data quality initiative much more accurate and set the right priorities for data quality improvement.
This is exactly what we’ve been working on at the University Cambridge over the past few years.
We developed a process for Total Information Risk Management (TIRM) that enables companies to assess and treat information risks on an organisational-wide scale that arise through poor data and information quality. As decision makers are using a combination of different information resources, TIRM aims at holistically managing risks arising from information resources of all possible types (external and internal, tacit and explicit, structured and unstructured etc.) and sources (e.g. databases, documents and humans). The TIRM process is based on ISO 31000, a widely recognized risk management standard.
Data Quality Pro: Who are the main contributors?
Alex Borek: The essence of TIRM comes from my own PhD project, but the project grew bigger over time since it’s kick off in 2009. The main contributors are my research colleagues at Cambridge University, Dr. Ajith Parlikad, Dr. Philip Woodall and Valeria Klassen of the Distributed Information and Automation Laboratory. Moreover, we work withMartin Oberhofer, a senior information architect at IBM, who helps us to integrate our work with the latest data quality solutions.
We also receive a lot of support from our international research collaborators at KSRI of the Karlsruhe Institute of Technology in Germany, the Strategic Information Management Lab at the University of South Australia and the Business Informatics Group at Dublin City University.
Finally, an important contribution has been made by our case study companies and a number of experienced data quality professionals, who help us to evaluate and fine-tune the TIRM process.
Data Quality Pro: Risk management is obviously widely used in organisations within the financial sector but for people who work outside of banking, what are the benefits of applying a risk management approach to information?
Alex Borek: The basic idea behind TIRM is that the risk management community provides many valuable methods, tools and techniques for measuring the impact of an event on the business, which we have transferred to data and information quality management.
In our research, we figured out the details that need to be considered to do the integration between risk management and data quality successfully. In summary, the TIRM process helps companies to craft effective data and information quality improvement initiatives taking into account economic figures and a detailed analysis of the organisation rather than the pure “gut-feeling” of managers.
We have identified six core benefits of using TIRM for data and information management:
- It gives you abetter understanding of how data and information quality affect your business.
- It enables you to focus your data and information quality initiatives on the “pain points”, where they bring the best business value.
- It provides financial measures to build more sensible business cases for information quality improvement.
- It helps you to improve the alignment of business and IT.
- It protects your business against exposures that arise from poor information quality.
- It increases your organizational business intelligence
Basically, TIRM enables you to manage data and information really for business value.
Data Quality Pro: You’ve carried out a number of implementations of TIRM within companies, can you describe the main steps in that consultative exercise?
Alex Borek: Before starting the actual information risk assessment, the context needs to be established, which includes the examination of the external context (e.g. the legal, societal, competitive environment…) and internal context of the organisation (governance, structures, processes, etc.), but also the risk criteria that should be used. We also have to define the goals and the scope for applying the TIRM process.
The information risk assessment stage is the central part of the TIRM process.
For each of the processes in the defined scope, we run a one day workshops with representatives in the company that know the process very well. For identifying information risks, we examine which information is used in the process and how it is created, processed and accessed. This is followed by a detailed information quality assessment, which can be done both subjectively by the process representatives and objectively using data quality software tools where possible. Then, the direct and intermediate consequences of the identified information quality problems are analysed and the probabilities and the financial impact are measured for each of the consequences whenever they have an impact on business objectives.
We also look at the existing risk controls that are already in place. Information risks are evaluated by comparing the level of risk that has been analysed to the defined risk criteria. We are currently developing a mindmap based tool that supports the data collection process in the information risk assessment stage, an example is shown in the figure below.
Before starting the actual information risk assessment, the context needs to be established, which includes the examination of the external context (e.g. the legal, societal, competitive environment…) and internal context of the organisation (governance, structures, processes, etc.), but also the risk criteria that should be used. We also have to define the goals and the scope for applying the TIRM process.
Finally, we enter the information risk treatment stage, which involves the selection of information quality problems to focus on, the identification of the root causes of these problems, the development of alternative information risk treatment options, a cost benefit risk analysis of these options and then the selection of the best options, which are then implemented and tested for its effectiveness. An example of a risk treatment option at a manufacturing company is shown in the table: Information about the technical product requirements from the customers, which are entered into the ERP system by the sales staff, are often incorrect and/or incomplete.
Modify ERP system so that data users can give feedback if collected information is sufficient and check the correctness already during the data collection stage.
ERP system does not sufficiently support the data collection process
Data collection is not communicated as a top priority by management
|Head of production and head of sales have to make it a requirement for sales staff to fill out the complete checklist with information that is as accurate as possible.|
|Insufficient knowledge of sales staff about which and how data should be entered|
|Special training for sales staff that shows how they can interpret the customer requirements and capture the data better.|
All relevant stakeholders need to be included in the TIRM processes. In particular, the IT management and risk management functions, and the representatives of functions that are examined during the risk assessment stage need to be included. Senior management support is vital for the success of the implementation. The implementation of the TIRM process needs to be regularly monitored and reviewed to improve its performance.
So far, we tested the TIRM process by applying it in a number of companies in the manufacturing and energy industries and refined it each time to get a more effective and robust process. Our next study will be in the transportation industry.
Data Quality Pro: What were the outcomes for the companies you investigated? What kind of issues did you find and how did the company benefit from the process?
Alex Borek: We investigated a large variety of business processes in our studies, which ranged from strategic management, sales & marketing, maintenance, manufacturing operations, planning, product development, logistics to purchasing. We really found a substantial number of information risks in each of the examined business processes, each of them having an observable impact on the business objectives of the company, such as costs, customer satisfaction, compliance, employee satisfaction, operational effectiveness and health & safety. Many of the information have been inaccurate, out-of-date, incomplete, not standardized, difficult to understand or simply not available and companies were not fully aware of these problems.
Let me give you a real example of some information risks we found: At an energy company, plans of the existing electricity network that are in 10-30 percent (15 percent in average) of the cases inaccurate or incomplete, as they are based on historical background plans, which is historical data up to 100 years old. Incorrect and/or incomplete plans of the existing network can lead to safety issues as engineers might work on high voltage cables without knowing it. This could lead 1 to 10 times a year (5 times average) to minor to serious injuries. Another consequence is that the job is calculated using wrong cost estimates due to wrong assumptions following from the network plans. This has a high impact on customer satisfaction, as three times a month customers are requested to pay up to 5000 pounds more than planned (500 pounds in average). Moreover, this creates, on average 3 times a year, additional costs of up to 10000 pounds (2000 pounds in average) for the company. Plus, due to inadequate planning, between 1 and 5 times a month (average 3 times), construction work is delayed by 5 days in average (between 0 and 14), which also impacts customer satisfaction.
In summary, the TIRM process helped our case study partners to understand where poor information quality creates the biggest “pain” and how this affects the business objectives in core processes of their business. They obtain a real measure of the impact of information quality, often expressed in financial terms, and can now focus their resources to improve information quality where it brings the highest value.
Data Quality Pro: What is the feedback from the wider data quality profession? Have people recommended changes or improvements based on your initial surveys?
Alex Borek: We are in contact with a substantial number of data quality experts in academia and business to continuously evaluate and improve the TIRM process. Most of the feedback is very encouraging as it is recognized that TIRM is a very substantial step towards solving one of the most urgent problems in data and information management, which is the measurement and assessment how information quality affects the business. Many experts believe that TIRM can really help companies to better manage data and information quality.
We received also recommendations to further improve the TIRM process. One of the suggestions was to integrate TIRM closer with FMEA. We are currently developing a software tool that translates and exports the output of the TIRM process as a FMEA spreadsheet. An important recommendation was to use more objective data assessment with data profiling tools in addition to our user-based assessments. We will do this in the next studies. Another good comment was that it might be useful to create financial metrics for specific industries, which convinced us to work on this in the future.
Data Quality Pro: Getting into the detail, there is an interesting section on one of the utility case studies. The report states:
“For instance, fatal injuries of engineering staff due to incomplete and inaccurate plans clearly has a very high impact on the organization’s objective “our commitment to our staff is to provide them with safe working conditions”. On the other hand, charging customers too much due to an incorrect bill of material has a medium impact on the objective “to exceed our customers’ expectations”. This, then, leads to the evaluation of the information risks.”
How do you decide that charging customers excessively has a medium impact, if I’m a customer surely that is a high impact? Isn’t this a subjective viewpoint as opposed to qualitative?
Alex Borek: Risk evaluation can be done in two ways.
You can use a list that translates a numeric ranges into a qualitative scale (e.g. . $0-10000 = “Low”, $10000-50000 “Medium”, >$50000=”High”). Often some these lists have been already created by the central risk management function in companies, if not, these lists can be generated by senior management (they should however be standardized).
The other way is to let a group of managers directly interpret the risks by assigning qualitative values, which would then be more subjective. The idea would be that a group has to agree on an interpretation and research shows that such group decisions bring more accurate results. Both ways are commonly used in risk management in organizations today.
Data Quality Pro: How do you arrive at your final list of risks? At the start of stage 4 you show a diagram that lists a series of risks but how do you determine those are the ones to go after? The reason I ask is that in many cases I’ve analysed the data and found hidden risks that the company was completely unaware of.
Does your process account for this voyage of discovery or does it work back from known risks as the starting point?
Alex Borek: Our approach does not start with a predefined list of risks but rather discovers them in the information risk assessment workshops. I fully agree with the experience you made. There are often risks uncovered by the TIRM process that people haven’t thought of before.
Data Quality Pro: How do you discover data quality problems? Using discovery tools or simply via workshops/anecdotes?
Alex Borek: We use a very structured approach to identify and fully characterize data and information quality problems in our workshops (we do not use anecdotes at all). Many of today’s most severe information quality problems cannot be discovered using software tools (such as understandability and representation of data and information) and this assessment needs to be done from a user perspective.
However, we additionally use data profiling and other discovery techniques within the information risk assessment stage of the TIRM process when it is appropriate. Moreover, we have started to work on how to connect the results of today’s data quality assessment solutions to provide automated input into the TIRM process using a software tool that we have programmed to support the facilitation of the process.
Data Quality Pro: Thanks for sharing your time today, I’ll be watching your progress with interest.
Alex Borek: Thank you Dylan.
Contributor Bio – Alexander Borek, UK
Alexander Borek, Postdoctoral Researcher and Lead Developer of TIRM at University of Cambridge
Alexander Borek (Linkedin) is a doctoral researcher and the lead developer of TIRM at the Distributed Information and Automation Laboratory of the University of Cambridge.
He is an expert on how to measure and manage risks that arise from poor data and information quality in industrial organizations and he has published a number of articles in major international conferences.
Before joining Cambridge, Alex studied Information Engineering and Management at the Karlsruhe Institute of Technology in Germany and at Carnegie Mellon in the U.S. He was also a co-founder of an Internet startup and an intern at two international strategy consulting companies.
During all stages of his career, he received a number of awards and scholarships. In March 2012, Alex will be a keynote speaker at the CeBIT Service Innovation Camp organized by IBM, at the ICT industry’s largest international trade fair.
You can contact Alex at firstname.lastname@example.org for any questions about TIRM.