Modelers and analysts often show model validation and results using Decile Reports (reports in which a population is sorted by model score and then split into 10 equal-sized groups of data). A Quintile Report is a variation of the Decile Report, where data is broken out into 5 equal groups. Decile and Quintile Reports are useful means of viewing model results; a good model demonstrates that key metrics are better for highest scored records.
In our hypothetical views that follow, a population of 100,000 customers is ranked from highest to lowest model score, based on the ‘likelihood of buying.’ The A group represents our best customers, and the E group represents the worst.
The Quintile Model Results Report below displays how well a model actually performed, by comparing customers who bought to how each customer was predicted to behave by the model. In a random sample, we would see 2,000 buyers from each quintile (10,000 buyers divided by 5 quintiles), but since our model is performing well, we see higher purchase rates for higher ranked customers.
Following the Pareto (80-20) Principle , a majority of the activity is in the top 20%, and very little at the bottom. A more useful view to focus attention where the activity occurs is to further break out the top quintile and condense the bottom quintiles into a pyramid.
The resulting Pyramid Model Results Report:
We can more clearly see what’s happening with our most important and influential customers. The very top 1% of ranked customers contribute 8% of all buyers and 39% of spend. The 8% buyers may not sound high, but it is coming from a fraction of the total population, the very best 1,000 customers. Looking at it from a horizontal perspective, 80% of the top 1,000 of ‘likely to buy’ customers did indeed become buyers. The pyramid allows us to visually see the value of focusing at the top and aligning marketing spend and touches based on how our customers are tiered.