Pushing the Envelope: How Custom AI turned Neighbourhood Mail into a Profit Machine

Every marketer must make a decision as to how much to spend on each channel of communication available to them.  In our experience, budget goes to…and stays with…channels that consistently deliver measurable results at a par or better than any other channel available.  But many marketing teams today are overworked and understaffed.  They need channels that are easy to plan and buy, delivering those results with less risk and less effort than competing alternatives. 

Digital platforms have done an excellent job at making it easy to plan and buy.  (delivering profitable results?  not so clear).  This post will focus on how a channel that has been with us for a very long time…Neighbourhood mail…was re-invented by adding custom AI tools, to become a profit machine. 

We will look at a case involving a telecommunications company, selling Home Phone, Internet Service, Mobile and Bundles (combinations of services at attractive prices).  The company’s competitive advantage is price; they offer services at a 20-30% discount to national major brands at comparable quality. 

Before using Neighbourhood Mail, the company relied on digital campaigns, especially search, to sign up new customers.  While the success rate of such work was acceptable, volumes were not. Neighbourhood mail was considered as a channel that could deliver reach to all serviceable households in their footprint, and sign up customers at an attractive cost. 

Often Neighbourhood Mail is targeted by demography or region, and used as a media to reach a lot of households.  But for this case, such an approach yields poor results; more precision in targeting was needed to generate a good result. 

To support this effort, Custometrics built a set of models, each of which predicted, for a given time period, the activation rate (number of new customers divided by pieces mailed) for a given postal walk (the unit of NM delivery) for a given product. 

In building these models, we found that many factors affected activation rate: 

  • How many times we had sent mail about a given product to that postal area, and how recently (or how many times we had sent mail about another product) 
  • Impact of other channels used in the same areas at the same time 
  • Demography; including income, family structure, and age distribution 
  • The baseline of demand for a given product 
  • Pricing vs competitive alternatives 
  • The penetration of current customers in that area, and the trend; was new customer acquisition outpacing churn, growing penetration?  How quickly?  Or was it the reverse; we are having trouble keeping customers in that area and replacing them when they go 

The difference in predicted activation rate, and therefore cost-per-activation, was large; top deciles were often at 4x as high an activation rate as the 5th decile; and the bottom deciles typically delivered hardly any activations at all.  (note; the measurement of activations took into account multi-channel effects, isolating the impact of Neighbourhood Mail for better decision-making).  Model accuracy rates, from pre-campaign scoring to post-campaign evaluation, are consistently in the 92-95% range. 

The cost of delivering a given service depended on its footprint.  This meant that since we wanted to calculate the predicted profit of mailing a given postal area, we needed to know the cost of delivery in that area. 

We found that churn was not uniform in all areas; there was large variation with some neighbourhoods consistently churning at higher rates.  We needed to take this into account as well, since the value of a new customer from those areas would, all other things being equal, be less than in areas that showed lower churn risk. 

For each postal walk, we had 7 scores; one for each product/footprint combination. For a given postal walk, the score for one month would be different depending on whether we mailed the previous month or not. 

To avoid this level of complexity from overwhelming decision makers, we added our custom optimization algorithm, tasked to find the optimal distribution of mail: 

  • For each postal walk… 
  • …for each of the three months of the campaign… 
  • …for each product… 

This meant some postal walks could get Product A for months 1 and 2, and Product B for month 3.  Or a given walk might not get any mail at all, if the model predicted the costs would exceed the profits of the predicted number of new activations. 

Interestingly, about 1/3 of new activations came from existing customers, something the model picked up by taking penetration momentum into account.  So in this case, Neighbourhood Mail became not only a profitable new customer acquisition channel, but a good cross-sell channel as well. 

For planning purposes, we created an ROI curve: 

Each point on the curve was the result of the model doing an optimal allocation of available budget across waves, time, and product.  All the work of taking full advantage of the model’s learning and accuracy was done for the planners by the software. 

Since there is no free lunch, the curve flattens as more pieces are added.  This reflects what we know about all advertising; after a given point, additional touches add nothing to business outcomes.   

The ROI curve enables a planner to increase spend with confidence of the outcome (risk can be calculated explicitly for each scenario).  It also demonstrates the overall impact of this channel; for a mailing of 2.5 million pieces, for example, a profit after all variable costs and campaign fixed costs are paid of over $4.5 million is generated. 

We have been running this system for this client for many years; with results in line with model expectations each time.  Predicted vs actual is carefully tracked by decile so we know the model continues to track today’s market conditions. 

Taken together, the reach of Neighbourhood Mail plus the decision-making impact of Custometrics’ predictive and optimization models has created a profit machine…high performance, with consistent results, at low risk and that is easy to plan for.   

The Free Money Effect and COVID Recovery

(Or the optimization bonus, or the budget multiplier…these blog post titles require a lot of thought…)

Imagine this scene: the CMO goes to see the CFO. “This economy is tough…I need more budget to hit my KPI target.”

The conversation does not go well.  Not only does the CFO not want to approve more budget, they want to CUT budget, to “protect the bottom line”.

If this, or something like this, has happened to you (or a friend) then chances are you are not using a technology that I would like to talk about today…predictive optimization.

Predictive optimization combines two analytical tools; predictive modeling and customized optimization algorithms.

The idea is to capture the dynamics of your business with an accurate predictive model, and then use that model to drive simulations in which we vary spend allocations (by channel, time and market, typically, although other factors can be simulated as well).  One of those allocations is identified by the algorithm as optimal; that is, the highest KPI effect for the money spent over the time period being planned. (we will come back to that point, below).

The key to making it work: model accuracy.

Aim to have your model explain a very high percentage of variation in past behaviour.  Most of our implementations use models that are over 90% accurate when we compare predicted to actual on recent time periods (yes, including the post-COVID period; see our last blog post here)

Integrated marcom campaigns behave in complex ways; we want our models to capture these effects.  But the price we pay for that accuracy is to have a complex model.  Think about the dynamics it must account for:

  • Diminishing return effects overall and by channel
    • One of the few “laws” of marcom planning is the diminishing return effect.  As you increase spend, there comes a point where each additional dollar has less and less an effect.  In our experience, this applies not just to the overall spend in the campaign, but to individual channels as well.  A planner needs to know where each channel is on its curve for the spend proposed.
  • Channel multipliers
    • We ideally want the channels we pick to reinforce each other’s impact in the market.  Generally, the more channels we have, the better the effect, but to a limit.  What we absolutely must avoid is cannibalization; the effect of spend in one channel diminishing the contribution of another channel.
  • Time
    • Marcom has an effect over a period of time, as we all know.  What is less well known is how that period varies by channel and by creative within channel.  Brand ads tend to have an impact over a longer period of time; promotion/call to action shorter.  Most digital forms have very short effect periods while other channels impact over a longer period
  • Local geographic effects
    • Retailers and other location based marketers must take into account the impact distance to store has
  • Creative
    • What is the right balance of brand vs promotion?  Budget allocation across different creative executions?

With all this (and more) going on, I just don’t see how planners can put together a campaign that takes full advantage of these effects without tools like predictive optimization. 

Further, to give guidance to decision makers, it is important to be able to produce credible, accurate forecasts of KPIs that will result from a given plan.  That is a big advantage of using a technology like predictive optimization.  It is one of the aspects of the technology that is most important in bringing the CFO onside.  Once they see the accuracy of the model, it is easier to trust it to help with decisions like these.

In our experience, this approach can lift KPIs by double digit amounts, while keeping budget constant.  Or, it can offset some or all of the effect of a budget cut. Or, it can make a persuasive case for a budget increase.

More impact on the same budget?  That’s the free money effect.  We think predictive optimization has a key role to play in helping brands recover in this tough COVID economy.

COVID Recovery Analytics

Resilience; n.; defn: the capacity to recover quickly from difficulties (Oxford Dictionary)

About 4 months ago now (though it seems much, much longer) our worlds were turned upside down by first the pandemic and then the lockdown used to deal with it.  Only now, here in Canada, are we seeing what appears to be the light at end of the tunnel (or is that an oncoming train??).  But many regions around the world are not there yet, and may not be there for many months.  And even as our case volume falls here, there is the constant awareness that a second wave may be in our future.  It does appear that until an effective cure or vaccine is found, we must find a way to live with COVID-19.

The impact on business has been dramatically bad; I won’t repeat many of the stats I am sure you have heard all too often. Many brands have put advertising on hold; hunkered down and waiting for the storm to pass.  In the meantime, though, these companies are bleeding business and piling up losses.  Some will not make it to the other side.  Some will have to open up and learn to function in this COVID economy, because there is no alternative.

(That said, some companies are in sectors that have benefited from these unusual times.  Online entertainment, ecommerce, products related to cleanliness and personal protection; these have all seen healthy sales volumes, if not increases vs the Before Times.)

How will we navigate this new normal?  How can we safely guide sales and brand-building marketing, while controlling risk? 

In our corner of the marketing world…advanced analytics…the answer lies with technology that we have been developing and perfecting for many years before COVID-19 showed up. 

Let’s start with predictive models.  This technology has been used by most marketers, although in my experience, some use it in a very tactical way, to, for example, help them build lists of customers for a campaign.  Predictive modeling is capable of much more.  It is possible to build models that predict near term (up to 6 months) outcomes with a high degree of accuracy and granularity (deciles, markets, segments, channels, etc)….certainly high enough accuracy that we can make more confident marketing decisions with them.  Such models are generally holistic in data inputs…covering a range of factors that drive results, both those under a marketer’s control and those that are not.

We have built models that incorporate the nearly 4 months of datapoints we have seen since lockdown.  We were pleased to see those models are as accurate…or even more accurate…than similar models on the same business from the Before Times.  That is very encouraging. We are talking accuracy rates…the difference between predicted and actual sales, by day or week, in the range of 93%.  That is more than enough to enable us to design high performance, low risk marketing campaigns.

But what if the virus flares up again?  To answer this, we have incorporated COVID case volume data by day into our models.  The data is allowing us to see strong differences between regions across Canada (an effect we would expect to see in the US, or other countries).  It makes sense in looking at the case volumetrics why some areas are still struggling with re-opening, while others have a much healthier outlook.  More importantly, as we move through the summer and into the fall/winter, when the concern is for the virus to make a comeback, we will have the data coming in, daily, to act as an early warning signal for our clients.

The second technology that enables a safe restart is optimization.  At Custometrics, we build custom optimization algorithms that take full advantage of the data and predictive models, to manage complex allocation problems…how much to spend by channel, market, time, tactic, etc.  Further, these optimization models also produce accurate forecasts for different planned scenarios.  This forward looking approach, allowing us to test various campaign designs for both effect and risk, gives clients more confidence to spend.

Finally, a critical component to recovery analytics is testing.  Testing expands our knowledge of marketing effects and builds the accuracy of our predictive models.  It also gives feedback to our decision makers that has immediate effect on the next campaign cycle. 

Bottom line; the combination of communication-smart, COVID-aware predictive models, custom optimization and smart testing will help us cope with the most challenging market we have seen in our lifetimes.  There is much to be cautious about, but also developments here that we can take courage from.


Istock photo credit:  frankpeters