A+ A A-

Plan for a successful data migration

Rate this item
(0 votes)
Plan for a successful data migration

Data integration and migration challenges arise every time an organisation moves to a new data system, or wishes to combine multiple data systems, either internally or as a result of a merger. This is a significant undertaking, which, if it fails to meet expectations, inhibits a business’ ability to function effectively. Yet, surveys indicate that more than 80% of all data integration projects fail. There are several reasons for this failure, but the most commonly cited issue is a lack of understanding of the extent and nature of problems that were to be experienced within the data itself. Data quality is critical to the success of any data integration and migration initiative, and organisations leave this aspect to the last minute at their peril.

Planning and data quality go hand in hand when it comes to data integration and migration, and any other big technology project for that matter. However, the moving of data is often an afterthought, since the business challenge revolves around implementing the new system or merging two disparate systems together. Data is also often outside of the scope of most system integrators, and is deemed to be the responsibility of the business once the system has been implemented.


The issue that arises is that projects do not adequately plan for data migration or integration from the start, and it is common for them to get caught out by unexpected data issues as a result. In a typical data integration or migration scenario, during the initial project planning and scoping, data problems remain unidentified. This frequently then requires a rework at a later stage in the project implementation, if issues are picked up during testing for example, which extends project deadlines and leads to unanticipated expenses. In fact, addressing these unplanned data issues can consume up to 70% of the time and budget in a rework later on, which results in costly and unnecessary delays.


The challenge with not planning for data at the outset is that systems are typically designed based on what people think the data should look like, and not what the data is actually like. Issues such as invalid date information, missing codes, incorrectly captured information and duplicate fields may seem like small fry compared to the other challenges of implementing a new system, but the reality is that these data quality problems can have a significant effect on the functionality of the entire system.


Data issues can cause loads to be dropped when the data is transferred to the new system, which means that information will be missing. This can also lead to multiple duplicates being created, which can destroy the integrity of the database, take up unnecessary storage space and impact negatively on system performance. Without information, businesses cannot function, and without ensuring data quality before a migration or integration, organisations cannot guarantee the availability of quality data, severely affecting business performance.


The biggest challenge around data integration and migration is proper planning from the start. This is exacerbated by large data volumes, which can make identifying problems a complex and time-consuming task. The best way of avoiding these issues is to use effective data profiling techniques, which will enable a sound understanding of any data quality issues. It is also advisable to conduct a proper analysis and scope of the implications of the data take-on at an early stage in project implementation, so that data issues can be planned and budgeted for.


Businesses often underestimate the problems that data issues can cause, and the impact of amalgamating data or moving it over to a new system. This aspect is often overlooked in favour of the functionality of the system, but the reality is that data quality affects functionality. Data quality remains one of the major reasons why IT projects fail to be delivered on time and on budget. High quality data lowers integration costs and project risks, increases user acceptance of new systems, and ensures systems meet the requirements of business users, helping to ensure a successful and problem-free project implementation.


For more information on the reasons why data integrations fail, and for strategies and solutions on preventing these failures, download the Data Integration and Migration white paper.

Last modified on Monday, 29 July 2013 12:59
Gary Allemann

Gary Allemann

Gary Allemann is passionate about Information Communication Technology (ICT) and more specifically data quality, data management and data governance. His introduction to the field of ICT was cemented with the completion of a Bachelor of Science degree (Hons) at Rhodes University which saw him enter the workplace in 1993 where he secured a position at First National Bank as a programmer. In 1996, Gary moved on to Dimension Data where he began to be more intimately involved with master data management as a Dimension Data CRM specialist. In 2001, Gary moved onto ILC Lerumo, a company that provides asset management and logistics consulting services to the defence and manufacturing industries, as a sales and marketing manager. In 2003, armed with 10 years’ experience, Gary formed Master Data Management, a business that provides specialist solutions for data governance, data quality, data integration and MDM. Leveraging the international expertise of its vendors including Harte-Hanks Trillium Software and Global Data Excellence, MDM has provided solutions for a range of clients in financial services, government, mining and telecommunications.

Website: www.masterdata.co.za

Social Profiles

Latest from Gary Allemann

Related items

More in this category: « What your IT manager could be doing instead of managing a server room CIOs must consolidate to drive data centre transformation »

Leave a comment

The SA Leader Magazine


In the June issue

Balancing of Business Requirements and Cultural Beliefs in the Workplace

Measuring your worth – productivity in the modern workplace

Turning Terrorists into Angels

Marketing needs to continually re-invent itself if it wants to retain the 'X' Factor


Copyright © 2014 gdmc (Geoffrey Dean Marketing Corporation cc). All rights reserved. Material may not be published or reproduced in any form without prior written permission. Use of this site constitutes acceptance of our Terms & Conditions and Privacy Policy. External links are provided for reference purposes. SALeader.co.za is not responsible for the content of external Internet sites.

Login or Subscribe