ERP BI - Testing

Testing the output of business intelligence can be any combination of tedious, confusing, and difficult – but in the end, also rewarding.

The easiest form of ERP BI testing is when there is a fixed “right” answer in the core ERP system, such as “last month’s revenue”. Almost all of the financial numbers that appear on the income statement and balance sheet fit into this category. Those numbers just need to be routinely reconciled to make certain they are, and remain, in synch.

It is a little more difficult to test and verify output that has been a traditional business metric, but does not have an easily-retrievable counterpart in ERP. In that case, you are evaluating the output in terms of common sense and historical norms. An example might be your net unfilled order position. If there is a big difference between what used to be, and what is now, you must break the number down to its next level of granularity. Normally you will find that you have either (a) changed the scope of what you are measuring or (b) changed the definition of what you are measuring. Only after you have eliminated those two should you suspect any technical problem with the way the data is constructed or assembled. For instance, a change in scope might be that traditionally the unfilled order report could not include any recent acquisitions made, but now that everyone is on ERP, it is a bigger number than people are used to. An example of a definition change might be that you have decided not to measure an order unless it has cleared credit, a capability you did not have in legacy.

Some Assumptions are Inevitable

It is very difficult to test “first-time-we’ve-ever-seen” information. In this case, you can only pass the output through a “is it possible for this to be the right answer?” filter by constructing a common sense estimate of what a reasonable answer could be. If the output passes that filter, then you have to assume the output is correct. These are the metrics which will sometimes embarrass you; by the time you find a problem with the output, you realize that you have been looking at and reacting to faulty information all along.

Finally, be prepared for the pressure of testing primarily in the live system. It would be nice to be able to verify BI output in your ERP test system – and you should certainly prove out all possible technical connections, data definitions, and equations in test – but you will never have sufficient volume of sustained data to prove everything out in the test system.

There is nothing as satisfying in information management as being able to achieve new insight into the way your business, markets, or customers behave. The more you depend on math, and the less on anecdotal evidence, the better your business decisions will be. Testing sufficiently to develop a world class ERP BI system is a difficult journey, but the destination is well worth while.