Handling your ERP data migration and transfers

Modern business systems hold an enormous amount of data so, when transiting to a new system, important decisions include how much of that data can, and should, be transferred, and how that should be done.

Companies essentially have two choices when loading data to a new system; manual entry or electronic transfer. Both have advantages and both have disadvantages.

Manual entry

Advantages of manual data transfer

Nobody wants to spend hours, days or weeks manually entering data on obsolete products, customers who haven't ordered in years and suppliers who may even have gone out of business, so the thought of doing so is a pretty good inducement to cleanse data and not just mindlessly copy it to a new system. A significant advantage is that the procedures and processes developed for the task can be kept in place to help ensure that cleansed data remains clean. Another consideration is that a lot of legacy data usually needs to be augmented anyway because new systems bring new possibilities, and that usually means new fields to populate.

Make sure you don't miss a step during your ERP implementation by using our free implementation guide

For some companies, entering outstanding sales orders and purchase orders can be a major task. However, the good news is that only orders due for delivery after go-live need to be considered and, if it is necessary to enter orders to the legacy system during the transition (e.g. to enable credit-checking and MRP to continue to work), such data can frequently be entered at a summary level only.

Finally; one advantage of having staff enter orders prior to go-live is that doing so is excellent practice and will get them up to speed before actual ERP go-live; when the pressure of them to perform within a new system will be at its greatest.

Disadvantages of manual data transfer

Inevitably staff will make errors when keying large amounts of data; so all re-keyed data will have to be carefully checked and great care will be needed to ensure that changes to data in the legacy system, such as new customers and changes to bill of material, are reflected in both systems during the transition period.

Some companies try to get around the problem by bringing in temporary staff to handle the task but this can be problematic because such staff will be entering data that they don't fully understand so, again, all of their work will have to be carefully scrutinized; perhaps even at the individual record level.

An additional problem is that the size of the task may be such that it needs to be started many weeks, or perhaps months, ahead of go-live. But, in the intervening time and, in the worst case, even at final system testing, changes to planned system configuration may be found to be necessary or advantageous, and some of these changes may mean that data needs to be reformatted after it has been painfully and expensively keyed.

Electronic transfer

Advantages of electronic transfer

Following on from the previous section, one very big advantage of transferring data electronically is that minimal demands will be placed on front-line staff, who will only be required to check some records to ensure that the transferred data is both correct and complete. There may still be demands placed on IT staff to write transfer scripts but, when very large volumes of data are being transferred, this is usually justifiable. And because only the process needs to be checked in detail, and not individual records, it can be expected that data accuracy will be good. Completeness of data transfer can be verified by comparing record counts in both systems (e.g. the number of customers in the customer master file) or by running reports (such as stock valuation reports).

Additionally, the transfer can be tested, and re-tested, until it is perfect and the actual live data transfer can be left until the last minute in order to ensure that is a complete replication of data in the old system; thus allowing system changes up until the last minute without worrying about keeping the two systems in sync during data load.

Moving from QuickBooks to ERP? Check out our complete guide on migrating your data from QuickBooks to ERP

Lastly, when data is being uploaded from, e.g. spreadsheets, it is easy to use mass updates to introduce new data fields or to set default values.

Disadvantages of electronic transfer

The big disadvantage in transferring data electronically (assuming the cost is proportionate to the value) is that moving data electronically is generally a task that only the IT department can perform because it requires specialist skills. So the task of moving data cleanly into the new system is seen as IT's responsibility; but some people will interpret that as meaning that IT are responsible for the data being clean. If the data in the old system is inaccurate, then the process of transferring it is not going to cleanse it and it is generally unrealistic to expect IT to know what data is accurate and complete and which is not.

So, which should you choose?

When very large files have to be transferred, and little or no augmentation or reformatting of data is required, there is a very good case for moving the data electronically. But when data volumes are low, or when the data has to be extensively manipulated before being entered into the new system, then there is little or no advantage in writing bespoke routines for the transfer; especially as expertise will be required in both systems and the vendors of the old system, who will understand its data structures, may not want to be over-helpful in aiding customers move away.

As is normal in ERP, there is a pronounced gray area between these two extremes; so choices must be made on a case-by-case basis. Major factors will be how easy it is to extract data from the legacy system and whether standard routines are available in the new one to facilitate bulk data upload. Companies are likely to find that a mix of the two methods (manual and electronic) is best. So, if they have thousands of customers but only dozens of products, moving the former electronically but re-keying the latter may be the best approach.

Whatever they decide, they should realize that data cleansing beforehand is absolutely essential and that it is never too early to start it. Processing bad data in a good new system is not going to produce the results that they should be looking for and demanding.