It is important for marketers to track the impact of campaigns in terms of how each one impacts the predicted future value (FV) of its recipients. By tracking the effectiveness of campaigns in terms of FV, marketers can take a long-term view of the value generated by campaigns, rather than just focusing on short-term metrics, such as short-term spending.
When a campaign is effective, the FV of the campaign’s test group customers should be higher following the campaign, as compared with the FV of the control group customers (a random selection of similar customers who did not actually receive the campaign).
This is true even for customer segments with future values expected to decline, such as Risk of Churn customers: even as their FV declines as a group, the marketer can take action to slow the decline and “rescue” some of the group’s members.
How Optimove Measures Campaign Impact on Future Value
The Optimove software applies statistical testing to the differences in future value between the test and control groups, before and after each campaign. Future value determinations are based on customer migrations among different micro-segments: as customers move to higher-value micro-segments, their predicted future value increases accordingly.
Optimove selects members of the control group to represent the entire campaign’s Target Group by first dividing the Target Group into its various micro-segments. The software then randomly chooses several customers from each micro-segment to be in the control group. The number of customers Optimove selects for the control group from each micro-segment is proportional to the size of the micro-segment compared to the entire Target Group. As a result of this approach, the future value prediction of both the test and control groups before the campaign is equal.
Of course, this statistical test will not apply to campaigns run without a control group.
What the Results Mean
An increase in future value indicates the relative FV uplift generated by the campaign, beyond what would likely have happened had the campaign not been run at all. As mentioned, this is calculated by comparing the predicted FV of the test and control groups at the end of the campaign’s measurement duration.
The statistical significance of the results is determined by analyzing the differences between the future value predictions of the test and control groups at the end of the campaign duration, regardless of the overall trend (upward or downward). For example, a Target Group consisting of Risk of Churn customers will have a negative trend, since these are customers expected to churn.
Having said that, examining the difference between the future value predictions for the test and control groups at the end of the campaign will be a strong indicator of the campaign's effectiveness.