Sales leaders who manage sales performance management (SPM) environments must routinely test their systems. However, they often overlook the critical need for developing test cases. In my years of experience reviewing SPM test approaches, one of the most common mistakes I see is the focus on parallel testing in lieu of proper test cases.

First, let me explain what I mean by parallel testing: An organization takes one or more months of data (historic or live production data), runs it through both systems (old and new), and reconciles the results to see if the new system is performing correctly. While parallel testing does provide some helpful feedback, this is not a thorough testing strategy and will only provide a false sense of security if done in isolation. Here’s why:

  • Parallel testing will not guarantee that all scenarios are tested. At best, parallel testing might cover 80-90 percent of the scenarios, but outliers are frequently missed and even small gaps can make a big difference. Many of the missed outliers are complex scenarios with special business rules that should be tested, but have not naturally occurred in the chosen data set. The only way to ensure full test coverage is to create a traceability matrix. The traceability matrix will map all system requirements to their corresponding test scenarios. Note: There will usually be multiple scenarios for each requirement, and in an ideal world every possible scenario would be covered in at least one test case.
  • Parallel testing almost always leads to redundant or excessive testing. Let’s say an organization has 1,000 payees that are paid incentives based on their year-to-date sales achievement. There are five different tiers of achievement that yield five different payout results. Now let’s say the payee population’s achievement results follow a standard bell curve, with approximately 70 percent of payees achieving the middle-tier payout. This means the organization is inadvertently testing the same test scenario 700 times, when only one test should suffice.
  • Parallel testing will uncover data issues, but might not put appropriate controls in place. In the process of reconciling hundreds of payees, an organization will usually discover missing or incorrect data that leads to incorrect payments. Although records can be fixed, how can organizations prevent similar errors from occurring in production? Careful review and analysis of source data should be part of the overall test plan. Appropriate controls must also be put in place in upstream system or pre-processing engines to alert users when there are inconsistencies or errors.

A Better Alternative: Developing Test Cases

Test cases, or scenario-based testing, provide a granular testing methodology by accounting for all conditions and rules that factor into a calculation. For instance, in an insurance company there are several types of insurance policies and commission rates that vary based on the type of insurance sold, the age of the policyholder and several other variables. By developing test cases for every possible scenario, companies can accurately and consistently calculate commissions for every salesperson regardless of the variables involved. In addition, the organization can:

  • Drastically reduce errors and delays in the compensation process.
  • Facilitate the debugging process by quickly isolating trouble spots, which can help increase accuracy and shorten testing time.
  • Establish a team of experts who become highly experienced and familiar with the SPM system through the testing process.

In short, while parallel testing can be a helpful exercise, it tends to take a “shotgun” approach that throws a ton of data at the system hoping to hit certain scenarios. It’s best performed only as a final step in the testing process. Detailed test cases, on the other hand, provide a more targeted “rifle” approach that ensures every requirement and scenario is tested.

Of course, not all organizations have the resources or time to develop and execute detailed test cases. The good news is, some consultant firms have highly experienced teams in place who can perform these tests to ensure the SPM system is as accurate, efficient and reliable as possible. But regardless of whether you maintain the testing process on site or outsource to skilled experts, running thorough test cases will spare you many headaches and wasted dollars down the road.

Do you have questions or ideas concerning the SPM testing process? Please let us know in the comments below!