How do you select the right data quality product for your organisation?

Selecting Data Quality Software (Part 1 - The Process)

The data quality software market is relatively young compared to more established sectors such as database software and business intelligence but it is still an extremely large and active marketplace.

If we go back even 5 years there has been a tremendous amount of upheaval and innovation in this sector.

There have been numerous buy-outs and new entrants so it can often be challenging for an organisation, who may be dipping their toes in the choppy waters of data quality management for the first time, to navigate through such a complex terrain to find the right product for their particular data quality challenges.

We therefore need to design a simple series of phases that helps you walk through the software selection process.

These steps are not prescriptive and you may not need every stage of the process but it should prove useful for most of your project needs.

Phase 1: Define your data quality product requirements

• Create a list of business and technical functions to be performed by the tool
• Functions should be described simply, use practical tools like use-cases so that everyone can understand
• Determine what skill levels are required to use the software
• Create a data quality strategy document and associated presentation
• Present the document and presentation to senior sponsors
• Obtain approval
• Present your strategy to staff

Phase 2: Research the data quality product marketplace

• Review product functionality in our video demonstration facility
• Review analyst reports
• Engage with member forums
• Reach out to other organisations who use products
• Visit conferences/events
• Seek independent advice
• Create a shortlist of prospective vendors

Phase 3: Segment and engage data quality product vendors

• Refine your requirements into an ITT (Invitation To Tender) that includes a compliance questionnaire
• Issue ITT to shortlist of vendors
• Assess the initial product compliance scores
• Vet and follow-up any testimonials provided

Phase 4: Implement a pilot assessment to benchmark your shortlisted data quality products

• Create a pilot scenario indicative of your current and future data quality challenges
• Assess the performance of the short-listed vendors
• Obtain independent advice where required
• Select either a clear winner or smaller shortlist for a live trial

Phase 5: Trial data quality products using live data

• Invest in appropriate level of internal training or professional services
• Implement the tool (or multiple tools) on live projects
• Review performance and anecdotal feedback from staff

Phase 6: Create a commercial negotiation process with the data quality vendor

• Confirm final product based on trial results
• Review commercial requirements
• Negotiate on terms

Some useful data quality product selection tips

• Don't define data quality requirements only on current needs: Look at future requirements, predict what your data quality needs will be moving forward. For example, you may only need data profiling at the very start of a data migration project but you will almost certainly need data cleansing capabilities in the later phases. You want to find the tools that are not straining at the seams to cope, flexibility and adaptability should be core requirements.
• Create data quality use-cases using workshops:Your needs must be driven from how the users will use the tool and not how you think the product will perform, get the potential users of the product involved and story-board all the different activities that are required.
• Seek data quality product advice:The fact that you are purchasing a product for the first time probably means you have limited knowledge in this area so get a second opinion on what products will be suitable based on your needs. Analysts can be useful but there are also many forums(ed - to be listed in the next post).Be wary of selecting a consultant or consultancy who only has experience of using one particular product, there is a danger of bias.
• Engage a specialist data quality consultancy before the selection process to help build your requirements: How do you know what your requirements are if you don't have the right technology? Sometimes it is obvious what your product needs are but very often there is a chicken-and-egg scenario whereby you need a tool to identify what problems are impacting your business before you can define the needs of a new tool. In this situation your organisation probably lacks data quality expertise anyway so engage a specialist and use their technology to perform a data quality assessment. This will help you define your current needs and help you create an accurate wish-list in any new purchases you make.
• Create a data quality strategy document for sign-off and a presentation for circulation: You need to foster feedback about your strategy for using the software but a lot of strategy documents can be laborious and too technical. Create a lightweight presentation that gives people the basics so they can engage and provide their feedback on what type of issues they face. Gathering as many needs and functions that the tool must perform is critical.
• Research previous data quality investments in the organisation: One organisation I consulted with, whilst actively researching the market for a name and address standardisation tool, discovered they had already purchased a product in their marketing function. In large organisations there are often many products in circulation. Find them and compare their user experiences against your needs. Having purchased licenses in the past often gives you more bargaining power with a vendor.
• Analyst reports on data quality are a guide not the gospel: Most analysts will spend only several hours critiquing a product, your team may need to use a tool for several years, use analyst reports to help you perform an initial search but don't let them be your final decision-making tool.
• Connect with other organisations and get impartial feedback: There are countless data quality forums (we will list them in the next post) where you can seek advice and opinion from others who have purchased tools, use these social gateways to find others who have made purchases.
• Be rigorous with testimonials: If a vendor provides a testimonial then seek permission to contact that person or organisation. Satisfied customers will generally be happy to convey their experiences Use this as a selection criteria in your process. The more open and honest the testimonial, the better chance you have of making an informed decision. The product may look perfect based on the pilot but how effective is the training and support? Does the product have bugs and glitches your tests are not uncovering? Are there professional services issues? Following up on testimonials and references will uncover these problems.
• Make the pilot tougher than your actual needs: Don't create a simple pilot with a few thousand records and a small amount of issues. Create something to really "separate the wheat from the chaff". Use large volumes of data, indicative of what your future business will look like and create a range of insidious data quality issues to put the tools through their paces. If you make the pilot so challenging that vendors request funding to cover their costs then that is something you should seriously consider. A relatively small investment in the pilot is better than a wasted purchase on a poor product. Get an experienced consultant to help you plan and execute the pilot if you lack the relevant skills, this is the most critical part of the process.
• Weight the performance scores: When assessing the performance of products during the pilot and in the initial ITT some features are nice to have but not essential. Create a weighting system so that your needs are prioritised. For example, some companies may need extensive security features for sensitive customer data, other companies may need extended character sets - weight each accordingly so that your scorecard is prioritised.
• Trial the data quality tools on live data before outright purchase: All vendors should agree a trial period, typically discounted or even free. Even though your pilot should be a good example of how you will use the tool it is still advisable to use the product "in anger" within your business. This will require an investment in training, even with several products if you still have a couple of products still in your shortlist. There are many differences between seemingly similar products such as usability, workflow, reporting, security, interfaces and basic functionality. By using the tools for a period of time by "real" users you will get a far better feel for what is right for your team.
• Step away from the negotiations: By the end of the pilot or trial you may have well developed a great working relationship with the vendor team. It is advisable to use an experienced negotiator to handle the commercials. Outline your key requirements and let them negotiate.
• Ignore the current data quality product licensing model, push for a flexible option that suits your project: If you are performing a data migration for example you may need a data discovery function at the outset, followed by a data quality rules management process, then a cleansing and transformation requirement followed by a monitoring platform for ongoing control. If you can't afford all these components in one hit then explain your project needs and work with vendors who offer the features you need when you need them at a reduced cost. Remember, no vendor will walk away when money is on the table.

In the next post - Selecting Data Quality Software (Part 2 - Process Template) we provide you with:

• Useful templates for segmenting the different types of technology
• Starter list of products and vendors to invite to your ITT
• List of known data quality hang-outs and social outlets to help you find others who have gone through the process
• A sample compliance sheet and some more tips on key activities often omitted from the selection process