A/B Testing

Split a flow's traffic between versions, measure which one converts better, and roll out the winner.

Beta

This feature may not be switched on for your account yet. Contact the Swapt team if you'd like it enabled.

The A/B Test step splits your visitors between two or more versions of a flow — for example, a page offering free shipping versus one offering 20% off — and measures which one actually converts better. When there's a clear winner, you can send all traffic to it with one click, or let Swapt do it for you.

An A/B Test looks like a Random Split on the canvas, but it also keeps score. Use a Random Split when you only want to spread traffic around; use an A/B Test when you want to know which path wins.

How it works

Each variant is its own path out of the A/B Test step. Build whatever you want to compare after each one.
One variant is the control — what visitors get today. Every other variant is measured against it, and if the test ends without a clear winner, the control is what stays live.
Each visitor is assigned a variant once and keeps it on every return visit, so the result reflects the variant rather than who happened to come back.
Until you set the test live, every visitor sees the control and nothing is measured.

Set up a test

In the Flow Builder, add an A/B Test step from the Popular or Flow Control group, and click it to open its settings.

Under Experiment, give the test a Name and, optionally, a Hypothesis — what you expect to happen, written down before you see the result.

Under What counts as a win, pick a Goal:

Visitor completes the flow
Visitor reaches a node — then choose the Conversion node that counts as a win
Visitor submits a form
Order created

Set Minimum visitors per variant. No winner is reported until every variant has at least this many visitors.

Under Variants, name each one and set how much Traffic it gets. You start with Control and Variant B at 50% each; click Add Variant for more. The percentages must add up to exactly 100%, and one variant must be marked Control (use Set as control to change which).

Build each variant's path on the canvas.

Click Set live at the top of the settings to start measuring.

Click Preview next to any variant to walk through the flow down that variant's path. Preview visits aren't counted, so check as often as you like.

Can this test work?

The Can this test work? card helps you size your test before you start. Enter the Smallest improvement worth detecting (as a percentage), and Swapt uses your flow's recent traffic to estimate how many visitors each variant needs and roughly how long that will take.

This is an estimate, not a setting. To hold the test open that long, enter the number in Minimum visitors per variant. If the estimate is longer than you're willing to wait, try a bolder variant rather than a longer test.

While the test is live

The status bar at the top of the settings shows Draft, Live, or Finished.

While a test is Live, its goal, sample size, and variants are locked, so the result describes one test rather than several.
To change anything, click Stop test. The settings become editable again.
After a test is Finished, it's kept as a record. Clicking Start a new test begins a fresh test with the current settings.

Read the results

On the Flows page, open the flow's Actions menu (⋯), choose Node Analytics, and open the Experiments tab. Each test shows its goal, its status, and:

A verdict at the top:
- Not enough data yet — at least one variant hasn't reached your minimum.
- No clear winner yet — one variant is ahead, but not by enough to call it (Swapt needs a 95% probability).
- A winner, with the probability it's the better option and its estimated lift over the control.
The traffic split — how visitors were actually divided.
A table of Visitors, Conversions, Rate, 95% range, Lift, and Beats control for each variant. When orders are involved you'll also see Revenue and revenue Per visitor.
A chart of how each variant's results built up over time.

Click Export CSV to download the results.

Swapt warns you if something could make a result misleading — for example Traffic split doesn't match the configuration (visitors aren't landing in the proportions you set, often because a variant's page is broken or slow) or Settings changed during this run. Treat the numbers with caution until you've looked into it.

End a test

When a variant is leading, the Experiments tab gives you two buttons:

End test — [variant] wins records the winner but leaves your traffic split as it is.
End and send all traffic to [variant] records the winner and sends every visitor down that path from now on.

If you end a test before every variant reaches the minimum, Swapt reminds you that you're deciding on incomplete evidence.

Changed your mind? Click Undo the rollout to put the previous traffic split back. The result and its winner are kept either way.

Roll out the winner automatically

In the test's settings, the Roll the winner out automatically card has a checkbox: Send all traffic to the winner without asking me. When it's on, Swapt sends all traffic to the winner only when:

Every variant has reached the minimum visitors per variant,
One variant has pulled ahead clearly enough to be called the winner, and
Nothing undercuts the result — the traffic split matches your settings and the settings didn't change mid-run.

You're notified when it happens, and you can undo it.

Use Give up after (21 days by default) to set a deadline. If the conditions above aren't all met in time, the test ends and the control stays live — running out of time isn't evidence that any variant is better.

Tests with automatic rollout on show an auto-rollout badge on the Experiments tab.