A recent uproar in economic circles was caused by an Excel spreadsheet error that puts into doubt the results of a paper by economists Carmen Reinhart and Kenneth Rogoff related to the relationship between debt-to-GDP ratio and economic growth. In their paper, Reinhart and Rogoff argued that countries with debt-to-GDP ratio greater than 90% are more likely to have slower growth than other countries.
This could have been a mere academic discussion if these results had not been used by politicians and governments to justify extreme austerity measures to control national debt. In his New York Times op-ed piece entitled “The Excel Depression” Paul Krugman notes that “in this age of information, math error can lead to disaster!”
The use of Excel is widespread in business as well since it often provides the flexibility – not found in transaction and other business software – to analyze data. However, the risk in Excel models is that they are not rigorously tested and may contain errors that are hard to detect.
One way around this is to use specialized software where it exists. For instance in operations there are network design, inventory optimization, pricing, routing and many other off-the-shelf decision support systems that have been rigorously tested over many projects and a long time period. Even in this case, there is still a need to make sure that the model developed correctly represents the supply chain or business environment.
All of this implies that models developed with Excel or other (off-the-shelf) software to make business decisions require a methodology to systematically validate the input, assumptions and results. At OPS Rules, our methodology involves building the baseline model which represents the current business environment. We then compare model results to the details of the business.
But, this is not enough! As many of you know, the baseline is not capable of always finding all errors. Therefore, OPS Rules complements the baseline with an analysis of many scenarios. If the model results for a specific scenario are not consistent with the intuition about your business, you need to understand where the discrepancy is coming from. Sometimes, the model suggests a new and surprising insight that would have been hard or impossible to obtain without the model. But sometimes, it is an indication of a problem with the data, the assumptions or the model. Any such discrepancy needs to be understood before continuing with the model.
This also implies the need to involve various subject matter experts who can check the results based on their experience and knowledge of the domain as well as familiarity with the input parameters. Put differently, an expert in analytics may not be able to detect surprising deviations in the results. It is the combination of operations expertise and analytics know-how that enables effective validation and eventually transformation of your business!
To see an example of this process in an end-to-end inventory optimization project with PepsiCo, I recommend our upcoming webinar. To register, click the image below.
Written by David Simchi-Levi, Chairman of OPS Rules Management Consultants