v0.8 — A/B test variants

A/B testing for nudge copy, available on Growth and Scale plans.

Release date: 1 March 2026

What's new

A/B testing for nudge copy

Growth and Scale plan merchants can now create multiple variants of any nudge message and run a statistically valid A/B test to find what works best for their audience.

What you can test

Any nudge type (chat bubble, exit modal, toast, sticky bar) can have up to 3 message variants. The variant a visitor sees is assigned randomly on session start and stays consistent for that session.

Traffic split

Traffic can be split evenly (50/50 for 2 variants, 33/33/33 for 3 variants) or you can use a custom weighted split — for example, 70/30 to show a new variant to a smaller portion of visitors before fully rolling it out.

Results

Test results are visible in Dashboard → Analytics → A/B Tests after a minimum of 100 impressions per variant. The dashboard shows:

Impressions and conversion rate per variant
Absolute and relative lift
Statistical significance (expressed as a confidence level)

Winner selection is manual. When you are satisfied with the results, go to the test and click Apply winner to make that variant the default for all visitors.

How to set up an A/B test

Go to Dashboard → your site → A/B Tests.
Click New test.
Select the nudge you want to test.
Enter your variant messages.
Set the traffic split.
Click Start test.

Tests can be paused or stopped at any time without losing accumulated data.

Breaking changes

None. Existing nudges continue to work exactly as before. A/B testing is an additive feature — nudges without a configured test are unaffected.

Notes

A/B testing requires a Growth or Scale plan. Starter plan merchants can view this section of the dashboard but cannot create new tests.
Running multiple tests on the same nudge simultaneously is not supported. Run one test at a time per nudge.
Test data is retained for 12 months after the test is completed or stopped.