Runner
Sign in

Free toolAds

Did version B really win?

Type how many people saw each version and how many converted. See each conversion rate, the lift, and whether the gap is bigger than chance would usually make.

Runs in your browser. Nothing you type is sent to Runner.

Version A converts
4%
Version B converts
5.5%
Lift of B over A
37.6%
p-value
0.0145
  • Significant at 95% confidence: passes

A gap this large would appear by chance less than 1.4 times in 100.

How it works

  1. Type the visitors and conversions for version A, your current one.

  2. Type the same for version B, the change you tested.

  3. Read the lift and the p-value. Under 0.05 is the usual bar for calling a winner.

Want this done for you, every week?

Runner gives growth a whole team: it finds the leads, sends each one your introduction, and drafts the posts and ads. You set the number it works toward.

Build my team

Questions about the A/B Test Calculator

What test does this run?

A two-sided two-proportion z-test, the standard test for comparing two conversion rates. It asks how often a gap this large would appear if the two versions converted equally well.

What does the p-value mean?

It is that chance: a p-value of 0.03 means a gap at least this big would turn up about 3 times in 100 between two identical versions. Below 0.05 is the usual line for calling it significant at 95% confidence.

Can I stop the test as soon as it is significant?

Deciding the sample size first and waiting for it is safer. Checking every day and stopping at the first significant result makes false winners much more likely.

More free tools like this

See every free tool