No data migration survives first contact with the enemy

Given that data sources may include the following, the chances of your migration (to cloud) working first time round, are very small:

  • packaged on-premise applications (which are increasingly being remaindered by global vendors in favour of cloud versions);
  • legacy or proprietary databases;
  • big data lakes or swamps;
  • the amalgam arising from mergers & acquisitions;
  • data from packaged systems whose data model contains duplications and inconsistencies arising from a software acquisition strategy;
  • migrations from other clouds;
  • storage;
  • or simply spreadsheets that should have been thrown away years ago.

Not every cloud is made equal

Each cloud has its own characteristics and may be informed by:

  • a desire to remainder obsolete systems for efficiency’s sake;
  • technology stack upgrades or changes;
  • changes in technical leadership;
  • proactive pursuit of a common technology platform; or
  • to harmonise data exchange formats or storage mechanisms.

Not every data source should be treated equally

Different data sources require different discovery and preparation:

  • Financial records need to retain integrity and auditability;
  • Customer data needs parsing to achieve and maintain consistency with particular attention to personally identifiable data (PII – GDPR data protection).

If the source data is in a packaged application, this does not necessarily guarantee that it will be meaningfully described or complete; and empty cells can hinder the export process. When extracted to Excel, many reports from a certain packaged application do not come in a clean table format but instead, because of the source data structures, appear with blank cells that have to be logically inferred.

Cloud isn’t a panacea, it’s a discipline

The data models of packaged applications may require you to reverse engineer from the package’s own data dictionary as a necessary precursor to working out how you will downsize those 90,000 tables into the much leaner data models found in the cloud.

Source data may be in remaindered or proprietary silos – requiring an agnostic, broad base of technical skill and expertise to effect discovery, preparation, extraction and migration. Cloud is suited generally to structured data, which leaves the conundrum of what to do with unstructured data which may be as much as 80% of organisational data. Unstructured data may be a crucial part of the digital record and shouldn’t be ported to Cloud without a clear idea of:

  • their volume;
  • scalability implications;
  • performance impact;
  • download costs;
  • ease of exit and exit cost implications.

As organisations become increasingly digital, data migration to cloud or cloud platforms (SaaS, PaaS, DaaS etc) will be an increasingly important task: first in migrating to cloud and subsequently migrating from one cloud to another.