The Campaign Analysis page summarizes the results of all campaign types available in Optimove, including A/B and A/B/n Test Campaigns, Multi-Template Campaigns, Multi-Channel Campaigns, the AI Offer Decisioning Agent (formerly Self-Optimizing Campaigns), and the AI Journey Decisioning Agent (formerly Self-Optimizing Journeys).

The page contains five main sections:

Campaign uplift and execution details
Long-Term Impact
Campaign Summary area – review your control and test group performance.
Campaign Analysis by Domain – drill down into specific domains/verticals.
Channel Metrics – see channel performance and engagement.

To analyze a specific campaign:

In Mission Control, on the three dots on the left of the campaign you want to analyze
Select 'Go to Campaign Analysis

Campaign Analysis Overview Video

Video Transcript

Welcome to this overview of the campaign analysis report. This report contains 5 main sections:

The financial result of the campaign, along with its execution details
Long Term Impact
Campaign Summary area
Campaign Analysis by Domain
Channel Metrics

Now let’s take a closer look at each section.

At the top here, we see a drop-down list box from which you can filter the report by a pre-defined attribute. In this case, for a specific Region. This campaign was executed on December 29th with a 1 day duration. You can select a different analysis date range here.

The series analysis button adapts to report multiple occurrences of the same campaign. This is also applicable to a/b and a/b/n campaigns.

Here you may enter any remarks you want to save with this report for later viewing.

This drop-down list shows the different metrics available with which to analyze the campaign’s performance. The options in this list are customized for each deployment as appropriate.

In this example, we’re looking at how the campaign performed in terms of net revenue.

“Increase in net revenue” indicates the uplift in net revenue generated by the test group versus the control group. If we change it to another metric, such as number of orders, we will see the uplift in terms of number of orders. Note that this selection changes everything below in the report.

If this figure is displayed in gray, it indicates that the results are not statistically significant.

Increase per customer is the total uplift divided by the total number of customers involved in this campaign (test group only).

This is the target group name. In this example – Active High/ Mid Tier. Details about the recurrence will appear if relevant and also the duration of the campaign.

Here we can see the execution details: This is the name of the marketing action, Promotion (if selected), the channel, the template details and preview, and control group proportion.

Let’s take a look at the next section.

In some cases where the campaign led to a positive future value impact, a Long-term impact analysis will be displayed, showing test vs. control future value increase, before and after the campaign. In this example, pre-campaign both groups had a similar FV, and after the campaign we see that both the test and control groups had an increase in their FV, but the test vs. control had an increase of almost 20%.

The Campaign Summary area presents the performance results for this campaign, for the test and control groups separately.

Group size – under test group we see the number of customers who received the campaign. Under control group we see the number of customers who received no campaigns at all during this campaign’s duration.
The existence of a control group allows us to determine how the campaign performed as compared to a similar group of customers who did not receive the campaign. This comparison reflects what probably would have happened if the campaign had not been executed at all.
# of customers who made an order shows the number of customers who made an order during the selected period. Changing the selected metric here will also change the terminology in this chart.
% of customers who made an order shows the percentage of customers who made an order from among all the customers in each group.
Average net revenue is the average order amount of the customers in each group who made an order during the selected period.

In this example, 9252 customers received the offer, 468 of them made an order resulting in an 5.1% response rate. The average order made by customers in this group was $87 during the campaign day.

The Gaussian bell indicates whether the results are statistically significant. Green indicates that the results are significant and gray shows the opposite. Rolling over the icon provides additional details about the differences between the test and control groups.

The campaign uplift presented at the top of the page is based on the difference in the response rates and average net revenue between the test and the control groups. Read more about thishere.

On the right, we see the list of customer IDs of those customers who made an order during the analysis period, along with some key metrics regarding their activity. To see the list of control group customers who made a made an order, simply click this number. The currently selected metric is always shown in the first column in this table, and the table is sorted by this column in descending order.

Clicking an individual customer ID will take you to the customer 360 report.

To further analyze these results, you can click here to export a more detailed list to CSV. The exported list contains both test group and control group customers along with a variety of additional metrics, aggregated for the measurement period defined by the “analysis Range” selected above.

In the Analysis by Region section, you can see a breakdown of the campaign analysis results by the different Regions.

In this last section, key metrics for the campaign are shown. These metrics are supplied by supported third-party systems (email, SMS, etc.) Optipush or Optimail like in the example here. Clicking the number will lead you to the Customer Explorer with the full list of delivered, opened, clicked or unsubscribed customers.

The yellow lightbulb at the bottom right of this page indicates that you have specific AI Insights Agent (formerly Optibot) recommendations for this campaign. In this example, the Focus your Target Group card indicates that 14% of the campaign target group generated most of the campaign revenue. AI Insights Agent recommends splitting the group into 2. Keep the same action for the 14% “High, all around Orders” and try something else for the remaining 86% of the target group.

This concludes our overview of the Campaign Analysis report. If you have any questions, please feel free to contact your CSM for further explanation.

Series Analysis

Toggle Series Analysis "ON" to measure the combined incremental impact of all the campaign's occurrences during a given timeframe, as if it were a single execution. By default, aggregated data from all campaigns executed within the past three months is included under the Analysis Range; however, you can edit the date range to expand the analysis.

"Increase" calculation in Series Analysis: When analyzing multiple occurrences in Series Analysis, the Increase is calculated by summing up all of the customers targeted in all occurrences. Meaning, customers targeted twice or more will be counted multiple times if they responded to various campaign occurrences.

Excluding Outliers

An outlier is a data point that differs significantly from other data points in a campaign’s results. Outliers can skew the campaign analysis. Therefore, to avoid ambiguity, you can exclude outliers from the data set.

An outlier in a campaign is not a predefined rule. It will always be calculated relative to the campaign’s group.

Toggle the “Exclude Outliers” button on or off to exclude or include campaign outliers.

Upon toggling to “on”, outliers will be removed from the entire analysis, except for the Channel Metrics section.

Note: For campaigns/series of campaigns with fewer than 20 customers or more than 500,000 customers, we are unable to run the outliers algorithm. Additionally, for campaigns with zero responders, this option will not be available.

How does Optimove define an outlier?

Optimove utilizes a statistical algorithm to identify outliers for each campaign. Below, we will explain the process in technical terms.

Optimove uses three standard deviations to detect outliers, with an adaptation to the KPI type.

Specifically, since many of the KPIs in effect have a right-skewed distribution (resembling a log-normal distribution), we also apply the three standard deviations rule to the distribution of the Log-transformed values. If the distribution is indeed right-skewed and positive, Optimove will only look for outliers in the right tail of the distribution.

Example use case

In the first screenshot (with “Exclude Outliers” OFF), the campaign shows a large increase in "Total Order Amount" of $29.1K (or $0.03 increase per customer).

Screenshot 2025-11-11 at 12.43.34.png

Let’s assume that after reviewing the deposit amounts, we suspect this campaign was influenced by an outlier skewing our data.

To remove this outlier, we toggle the “Exclude Outliers” button ON. After doing so (as seen in the second screenshot), we can see that Optimove classified 1 Responder as an outlier. Toggling the switch removed this user from the results, and the increase in Total Order Amount dropped significantly to $6,794 (or $0.007 increase per customer), completely changing our ability to learn from the experiment.

When should you toggle “Exclude Outliers” ON?

As a rule of thumb, if, when analyzing the customer list’s responses, some customer responses differ significantly from the average response rate, that is a good indication that you should toggle “Exclude Outliers” ON.

After integrating your bonus systems, Optimove can present the number of customers who claimed a promotion in the Campaign Summary area.

How to enable promo code tracking

Please note: this procedure must be coordinated with your Customer Success Manager. As part of the ETL process, a new table/file named ‘Promotion Redemption’ should be sent to us, so we will receive the bonus information redeemed on a given day.

The file should consist of the following columns:

[PromotionId] [int/alphanumeric] NOT NULL
[CustomerId] NOT NULL
[RedemptionDate] [date] NOT NULL
If the date provided is not within the campaign’s date range, it will not be added to the campaign analysis page.
[Amount] [numeric](18, 2)
If this data is not available, please set it to be empty. (18,2) – this is the format of this value. 18 is the maximum length of characters before the decimal point and two after.

Note:

The number of clients (who claimed the bonus) presented in the campaign analysis is on a customer level.
If you wish to update the redemption data historically, reach out to your Customer Success Manager to coordinate.

How often are the metrics in this report updated?

	Description	When
“Increase In” Estimation	The “Increase In” figure provides an estimate of the additional revenue (or other selected metrics, such as Purchases, Activity Days, or Wagers) that a campaign generated.	Daily
Campaign Summary	Summary of campaign statistics and response. Along with a breakdown of the customers who have responded to the campaign.	Daily
Email Engagement Metrics	Sent: The number of unique customers in the selected Target Group minus ineligible (i.e., no email address) and dropped customers. Delivered: The total number of unique customers to which an email was successfully delivered (i.e., received by the recipient’s mail server). Opened: The number of unique customers who opened the email. Clicked: The number of unique customers who clicked on any link in the email. Unsubscribed: The number of unique customers who unsubscribed (opted out) from the email. Spam Report: The number of unique customers who actively marked the email as spam (or dragged it from their inbox to their junk folder). Dropped: The number of unique customers removed from email eligibility due to a previous bounce, spam, or unsubscribe. Bounce: The number of unique customers whose email address bounced (invalid email address or account not accepting email).	Real-time
Mobile App Push Engagement Metrics	Sent: The number of unique customers in the selected Target Group. Delivered: The total number of unique customers to which the mobile push was successfully delivered. Opened: The number of unique customers who opened the mobile push. An Open is counted when the customer clicks or swipes to open the app. Dismissing the notification but opening the app separately will not count as an Open. Bounced: The number of unique customers whose mobile push notifications bounced. Bounces are measured by those that are bounced by the Apple Push Notification Service (APNS) or by Firebase Cloud Messaging (FCM). Bounces are outside of Optimove's control.	Populated throughout time as we receive callback data
Mobile App In-App Engagement Metrics	Sent: The number of unique customers in the selected Target Group. Delivered: The total number of unique customers to which the in-app message was successfully delivered. Opened: The number of unique customers who viewed the in-app message	Populated throughout time as we receive callback data
Web Push Notification Engagement Metrics	Sent: The number of unique customers in the selected Target Group. Delivered: The total number of unique customers to which the web push notification was successfully delivered. Opened: The number of unique customers who viewed the web push notification.	Populated throughout time as we receive callback data
Web Page Pop-Up Engagement Metrics	Displayed: The number of customers for whom the web page pop-up was rendered. Each time the system successfully displays the pop-up to a customer, it’s counted as one.	Real-time
SMS Engagement Metrics	Sent: The number of unique customers in the selected Target Group Delivered: The total number of unique customers to which an SMS campaign was successfully delivered Clicked: The total number of unique customers who click on a link in your SMS campaign. Bounced: The number of unique customers whose SMS message bounced, as in the phone’s carrier rejected the message. Dropped: The number of unique customers who have not received the SMS message, as it was dropped before it reaches the recipient server, either to protect the sender's reputation or when a subscriber has already unsubscribed or bounced previously.	Populated throughout time as we receive callback data
3rd Party Engagement Metrics	Sent: The number of unique customers in the selected Target Group minus ineligible (i.e., no email address) and dropped customers. Delivered: The total number of unique customers to which an email was successfully delivered (received by the recipient’s mail server). Opened: The number of unique customers who opened the email. Clicked: The number of unique customers who clicked on any link in the email. Unsubscribed: The number of unique customers who unsubscribed (opted out) from receiving emails	Dependent on third-party