👉 Configuring Sequential Testing Alerts

  • Updated

📖 Definition

Sequential Testing Alerts offers a sensitivity threshold to detect underperforming variations early and trigger alerts or pause experiments. It enables you to make data-driven decisions swiftly and optimize user experiences effectively. Sequential Testing saves time and resources by stopping experiments that are unlikely to yield positive results.

 

⚙️ Configuration

To enable Sequential Testing alerts on an experiment, apply the following steps:

  1. Access the AB Test configuration
  2. At the bottom of the first step, the Main information step, click on the Advanced Options
  3. You can then configure 2 options: 
    1. The type of automation
      You can either receive an email if your campaign is failing, a kind of “Warning, you’re experiment is going right into the wall” or directly pause the campaign if it’s failing.
    2. The sensitivity threshold you want to experiment on

      You have 3 different level of sensitivity here:

      • High - It’s the safest solution. You’ll be more likely to receive alerts but you’re sure that losing experiments will be detected sooner.
      • Low - It’s the more advanced setup. You’ll receive less alerts, but your losing experiments will take longer to be detected.
      • Balanced - It’s the half-way between the 2 previous detection levels.

You can learn more about detection level on the Sequential Testing Alert article.

Also, more information about Sequential Testing in the FAQ article

 

Good to know 💡

You can only configure this setup if you selected a Primary Goal at the previous step!