Given that data sources may include the following, the chances of your migration (to cloud) working first time round, are very small: Each cloud has its own characteristics and may be informed by: Different data sources require different discovery and preparation: If the source data is in a packaged application, this does not necessarily guarantee…
If you have massive amounts of data which needs to be available across the extent of your organisation, even your ecosystem, you will need data engineering. Data engineering could be regarded as the antidote to (prevalent) organic (as opposed to purposefully architected) information flows in application-centric organisations. You need to design & deliver optimal circulation…
Data validation tests the accuracy, consistency (of format and standards), quality and integrity of the data (& associated metadata) for onward information engineering, data provisioning and curation. Note that ‘reliable’ data still needs to be validated. Even though this is not its primary purpose, data anomalies revealed by validation techniques can reveal hitherto unnoticed business…
It seems obvious, but migrating to cloud can be a frustrating process if your data is not suitably transformed before you try the ETL process. As data volumes get bigger; formats more varied and growth, exponential, data transformation techniques are vital. Routine transformations ensure harmony of data format and target destination. But more importantly, data…
Poor data quality, taking a reactive, rather than proactive, stance to data errors is a massive own goal. DQ negatively impacts the business so that: Financial data, trades & transactions, are highly regulated and subject to time stamping. Delays and mistakes are not only corrosive, they have much more immediate and damaging implications. The European…
Experience teaches that organisations should consider data discovery services if there is: Since organisations typically have upwards of 80% dark data (data that is unknown to them, and not understood), can you be confident that your organisation is aware of these common data issues? In the age of GDPR companies are expected to know what…
A data audit will help your organisation identify potential: For example, ‘dark’ data: data which is not analysed and from which no value accrues, is now expected to normalise at over 90%+ of unstructured organisational data according to IDC. That means: Dark data may be junk, or it may be a hoard with potential value…