Abstract digital data landscape

akquinet consulting

hyper-PMD

We support complex data migration projects to SAP using our proprietary Data Migration Platform, which enables flexible data cleansing, consolidation, transformation and validation. hyper-PMD helps to manage the migration process in a controlled, repeatable and transparent manner — with full access to data, processing rules, change history and audit logs.

 

You are here:

Factors influencing data migration

The data migration process is an issue that causes organisations many problems right from the project preparation phase. Our experience enables us to identify a list of critical factors that influence the successful execution of a migration.

Determining the scope of the data

Whilst identifying the areas of the company’s operations to be migrated may seem a straightforward task, we must bear in mind that it is only through a detailed analysis of the source systems and the data they contain that we can determine precisely which categories of data, and to what extent, will be subject to migration.

Identifying data sources

The most common problem in migration projects is a lack of knowledge about all data sources. The biggest factor contributing to this is so-called ‘wild IT’, i.e. a situation where some of the data that should ultimately end up in the target system is stored in local, unregistered databases or spreadsheets.

Lack of knowledge of data structures

The consequence of not having a complete understanding of the data sources and scope is the difficulty in technically specifying the source and target data structures. Consequently, developing a migration plan from a technical perspective becomes complex, as the processing algorithms will be subject to change with every alteration to the data structures.

Development of processing algorithms

In large-scale projects, where data from multi-branch organisations, drawn from a wide variety of sources and system types, is subject to migration, designing data processing algorithms to ensure error-free import into the target structures is an extremely complex task. An additional problem is a lack of knowledge regarding the technical specifications of the target system – namely, the import structures and the system’s requirements regarding the quality of the imported data.

Poor data quality

We usually only become aware of sub-optimal data quality during the processing stage, when it transpires that parts of the data are missing, duplicates are present, or it is not possible to transfer the data directly (e.g. data formats or lengths do not match, etc.) In such situations, migration tools must provide functionality that enables the data quality to be brought up to an acceptable standard.

Impact on the organisation’s day-to-day operations

Data migration projects in mission-critical systems must not halt or significantly disrupt the organisation’s operations. It is therefore vital that the migration process does not affect the organisation’s day-to-day operations, and that the production migration itself takes place quickly and without errors.

The list of factors influencing data migration projects is never exhaustive. Unforeseen situations always arise; it is therefore crucial to select a migration environment and processing methodology that allow for a swift and effective response to risks and errors.

 

Data Migration Platform

Drawing on the experience gained from numerous data migration projects to SAP systems, akquinet consulting has developed a proprietary solution to support the data migration process in the areas of data cleansing, consolidation and transformation. Based on the assumption that the greatest challenge in such projects is the lack of comprehensive knowledge regarding data structure and quality, as well as the need for continuous adjustment of processing algorithms, a tool has been developed that allows for flexible customisation of the processing workflow. Each element of this process – namely, source data, control data or processing rules – can be adjusted continuously and repeatedly, thereby enabling the data migration algorithm to be refined in a flexible and secure manner.

Data transformation based on the Data Migration Platform is a service dedicated to organisations planning data migration in an environment where the data processing model is subject to constant modification and where repeated adjustment of data processing algorithms is required.

Data

During the migration process, in addition to the data to be processed, various types of control data are also used, such as mapping data and validation data. At the same time, each data migration project defines a unique data processing model.

A unique feature of the Data Migration Platform is unrestricted access to all data at every stage of processing and from within every module that processes that data.

This means that regardless of the type of data – such as source data, control data, output data or subsequent versions of processed data – it is available within the system without restriction and can be used as any type of input data and in any stream of processed data. It is therefore possible, for example, to retrieve input data, process it in any way, and use it, for instance, as mapping data. Importantly, this data is not subject to any time or process-related restrictions: data retrieved or generated today during the migration of material index data may, six months later, be used as validation or mapping data in the order migration process.  

PMD also allows for flexible access to data in record consolidation or splitting processes. The data version created as a result of data mapping in step three can be used as control data in the consolidation process in the next or a subsequent step. PMD also places no restrictions on data access in terms of the number of data sources. If a processing rule requires simultaneous access to data from an external file, data from another processing stream and user-specified mapping data, this scenario is feasible.

Processing data and processing control data are available to all rules and processing steps without restriction. This allows for the creation of any data migration model.

hyper-PMD diagram: data sources, rule configuration and target system

Data processing -

Iterative processing model

Processing large volumes of data often requires numerous iterations and modifications to the same data. In each successive iteration, based on the experience gained, erroneous data is eliminated, the configuration is adapted to new requirements, and the processing algorithms are adjusted to ensure results with a higher data accuracy rate. The Processing Monitor’s reporting function can be activated at any stage of processing and for any specified data.

 

The processing process

Data processing is based on the use of Data Packages in conjunction with Procedure Packages. Data packages may contain data from multiple sources simultaneously; they can be modified and reused multiple times. Procedure Packages are sets of Rules of various types. It is possible to use standard rules (validation, mapping, bulk operations), which are configurable, as well as custom rules. A custom rule may be based on ABAP code, which allows any processing logic to be programmed. An example of such a rule is the data consolidation module, where processing is delegated to an external report containing processing algorithms tailored to the requirements of a given business environment.

Flowing light waves symbolising data processing

Implementation of any data processing algorithms

The Rule Repository module is used to manage rules, where they can be created and edited. Rules whose operation requires external data (e.g. mapping data) allow the rule model to be preserved, i.e. multiple configuration versions of a single rule. This enables data to be processed multiple times using different configuration data. Repeated processing is made possible by the use of a data variant rollback mechanism, i.e. rolling back the processing to a specified point in time. Once the data has been processed by a rule, the Verification Procedure Package can be run to check the correctness of the processing on an ongoing basis.

Glowing data blocks symbolising the hyper-PMD rule repository

Processing Monitor

The migration of sensitive data always carries the risk of errors. When migrating large volumes of data, it is essential to provide tools that enable the reporting of all events and changes occurring during the migration process.

The Data Migration Platform fully supports the user in this area. The Processing Monitor is a group of background processes which, at every stage of processing, collect information about the operations being carried out and the logs of their execution. Every time a rule is triggered and every time data is modified, this fact is recorded in the processing logs. At the same time, these processes ensure that every version of the processing data and control data is stored. The Processing Monitor also assists the user in preparing subsequent iterations of data processing.

Dashboards on screens symbolising the hyper-PMD processing monitor

Using the data it has collected, the Processing Monitor enables, amongst other things:

  • Enabling change tracking for a specified record
  • Displaying consolidated and split records
  • Displaying the processing history, including for consolidated/split objects, along with their respective histories
  • Displaying the history of changes to a record’s attributes backwards, until the source record is found
  • Displaying log contents in individual modules (by context)
  • Outputting procedure logs (individual, collective, etc.) to screen and to file
Data dashboards on a screen

Reporting

PMD provides a set of reports designed to assist users in working with the system and to support the Audit Console.

The data flow report provides comprehensive information on the flow of data from the data source to the export data

Data preview and comparison report – allows data to be viewed and compared during processing, detailing modified data and providing access to processing logs for the smallest data unit

General logs overview report – an overview of the log of general actions and events, such as data deletion, object creation and object deletion

Qualitative and quantitative report – generates summaries for selected PMD system objects, including data on the quantity and quality of processed records

The Data Migration Platform in practice

The Data Migration Platform was utilised as part of a data migration project carried out for one of our clients.

Migration figures based on statistics generated by the audit features:

The key benefits of PMD

Automated consolidation of suppliers and consolidation of customers

Approximately 400,000 suppliers and customers were processed.
Following consolidation, 230,000 suppliers and customers were loaded into SAP.

An automated process for standardising, cleaning up and consolidating fixed assets

Just under 500,000 fixed assets were processed.
Following processing, 380,000 fixed assets were loaded.

Automated consolidation of material indices

Over 300,000 material indices from 13 branches were processed.
Following processing, just under 26,000 material indices were loaded.

Standardisation of the structure of capital expenditure

Over 80,000 investment projects, comprising just under 600,000 individual items, were processed.

Reducing migration time

The time taken for production migration was halved – depending on the area, from 2–5 days to 1–3 days of production system downtime.

Authorisation and audit log

Guarantee of appropriate authorisation for access to migrated data. Logging of all changes made to data and the ability to trace every step of the cleansing process, including manual data corrections – particularly important for financial data.

FAULT-FREE production migration

All production data was loaded correctly – 100 per cent.

A reduction in the level of staff involvement required

The number of staff dedicated full-time to carrying out the migration on the Client’s side was reduced by 40%, from the planned 40 to 25.
Thanks to the availability of validation tools and reports, the ability to work in parallel and the option for manual data correction, PMD reduced the time and labour required for data analysis by the client’s staff by a further 30 per cent.
Estimated total reduction in the Client’s staff labour costs during the migration: approximately 35%, whilst ensuring the quality, security and transparency of the entire migration process at a level incomparable to the traditional approach to migration based on cleaning data in files.

The Data Migration Platform provides unique functionalities that are not available in other data processing products.
At the same time, as a platform open to changes, it allows for easy extension of the available functionalities. The solution, based on Rules as independently programmable modules, allows all required transformations to be designed and executed within the Platform, without the need for additional processing in external environments.

In practice, this enables comprehensive reporting of data changes throughout the entire processing workflow (e.g. for audit purposes), which is not possible where partial processing takes place outside the migration environment.
A thorough understanding of the environment also allows changes or new functionalities to be introduced in consultation with the client.
Where necessary, standard solutions can be modified to fully tailor the tool to specific requirements.

PMD is a tool designed to support data cleansing, consolidation and transformation in data migration projects to S/4HANA systems as part of greenfield implementations of these systems.

Very easy to reach

We look forward to hearing from you!

Title*
Please make a selection.
Please make a selection.
Please make a selection.
This is a required field - please complete it.
This is a required field - please complete it.
This is a required field - please complete it.
This is a required field - please complete it.
This is a required field - please complete it.

By completing this form, you give your informed consent to the processing of your personal data for the purpose of responding to your enquiry. You may withdraw your consent granted to akquinet consulting sp. z o.o. at any time; however, such withdrawal will not affect the lawfulness of the processing carried out prior to the withdrawal.

Additionally, if you would like to receive information about current promotions and products from akquinet consulting sp. z o.o., please tick the channel(s) through which we may contact you (optional):

Consent is required for registration; please tick the box.
Consent is required for registration; please tick the box.

akquinet consulting sp. z o.o. processes your personal data in order to respond to an enquiry sent via our contact form and/or to send marketing emails if you have given your consent. You have the right to access, rectify, delete, restrict the processing of, and to transfer your data. You also have the right to lodge a complaint with the President of the Personal Data Protection Office. Further information on how we process your personal data can be found in the Information Notice.