Data Migration 101: Working in the Cloud with Dynamics 365 ERP technology

Data Migration for any ERP (Enterprise Relationship Management) implementation is a daunting subject. Years, potentially decades, of data must be analyzed, sorted, and reconstructed for the new ERP system at hand, Dynamics 365 for Supply Chain Management and Dynamics 365 Finance.

Dynamics 365 introduces new challenges for data migration, however, Microsoft’s formal shift to the cloud means new best practices.

Looking to our past, Dynamics AX 2012 is an on-premise ERP system, and its core tools have proven reliable. These tools include:

• Data Import Export Framework (DIXF)
• Application Object Tree Data Dictionary
• SQL Database Access

DIXF is the tool in AX 2012 for the actual data movement, but any Dynamics professional who has been involved with AX 2012 Data Migration will cite use of the AOT’s Data Dictionary and/or directly checking data in the SQL Database. These latter two tools are “lenses” giving administrators the ability to directly search, inspect, test, and analyze data in system tables. During initial loads in DIXF, errors WILL happen. Working with the AOT’s Data Dictionary and running basic SQL database searches greatly simplifies troubleshooting those errors.

With the move to the cloud, Data Migration’s mindset is now different. The production environment lives in the cloud which is controlled by Azure. We must now use and trust our new tools for data migration, which include:

• Data Management Framework (DMF)
• Data Entities

In Dynamics 365, DMF is the spiritual successor to AX 2012 DIXF, functionally serving the same purpose. Two other changes have also occurred:

• The production SQL database is no longer controlled by the IT administrator and is controlled directly by Azure Administrators.
• AOT access is no longer granted to system administrators in Production, and is instead available in development environments.

In Dynamics 365 Supply Chain Management and Dynamics 365 Finance, the new Data Migration focus is 100% on Data Entities. I intentionally left out Data Entities when discussing AX 2012. They exist in AX 2012 installs, but typically 1 or 2 technically-minded persons per project focus on the data entities, relying on SME’s from elsewhere in the project to provide and validate data for the system. End Users SME’s and Consultant SME’s typically use either the Data Dictionary or SQL queries to analyze and compose records for the full load.

While many, including myself, miss those tools we had in an on-premise solution, our approach in Dynamics 365 has a new focus, and all users working in migration must learn it, which brings us to Data Entities.

Data Entities now serve as THE data migration tool for both the technically minded consultant and the SME’s (who we must remember is almost certainly new to the product and learning the features of the system).

Rather than have the end-users work and think in terms of the SQL table which is rather technical, Microsoft has deliberately shifted the focus to entities for simplicity? so what is an entity? In short, it is a vessel that will accept a data import (typically from Excel or CSV files but other options are possible) and deposit the imported data onto a staging table within the entity. That staging table will then automatically be validated and exported to the SQL table in the D365 Installation.

For instance, we have an entity named “Approved Vendor List by Products” found in the DMF workspace in Dynamics 365.

This entity will migrate data to a table called PurchProductApprovedVendor.

To a new user, this table name means nothing, to a seasoned Dynamics professional, we know this is the SQL table that stores which approved vendors are approved on what date range for individual products.

And this is where entities serve their purpose. Rather than have the user think in terms of the table, we have an entity, named to describe what is being imported.

Sounds simple enough, right? Well? yes and no. Data entities do streamline data imports, but they have their own details which must be learned, and I will cover that in a future post. The intricacies of entities can be a lot for a new end-user to take in. Also, as of the time of this blog post, Dynamics 365 2680 entities.

As we said at the very beginning of this post, that’s a daunting list to sort through. Microsoft has provided “Data Templates” which group together hundreds of entities and gives vision to the appropriate order to follow with data migration. But I can tell you from experience the Data Templates are more or less “guidelines” rather than an actual road map. No two projects will ever have the same data migration requirements, and professional help is required to determine appropriate scope for data migration.

Get across the finish line with expert consultancy

This is where AlfaPeople’s experience comes into play and gets clients to the finish line efficiently and with confidence. AlfaPeople has fully mapped the default list of entities provided by Microsoft and simplified the process so SME’s can learn fast and digest the data migration process easily.

Our tools that we bring to the table include:

• A fully spelled out list of data entities in the exact order from beginning to end for a successful migration
• A clear-cut method for determining which entities are needed for the project and which ones are not, giving a clear scope to what data migration will require
• Designated locations for custom entities in the migration process which be needed as projects typically require at least a few custom table add-ins

Contact us today to speak to an AlfaPeople consultant!

Learn more about Microsoft Dynamics 365 here.