An experiment plan template forces the decisions that most A/B tests skip: what exactly you expect to happen, which single metric decides the winner, how many visitors you need, and when you are allowed to stop. Writing those down before launch is the cheapest protection against reading noise as a result. The template below is one page per test. Copy it into your docs tool, or download the template as CSV for a test backlog in a spreadsheet.
It is designed for B2B traffic levels, where a single test can take weeks. If you have not planned a test before, read our conversion rate optimisation guide first for what is worth testing at low traffic.
The A/B Test Plan Template
Fill in every row before the test goes live. A blank row is a decision you will end up making after seeing the data, which is exactly what the plan is meant to prevent.
| Section | Field | What to write |
|---|---|---|
| 1. Context | Test name and ID | Short, searchable name, for example "DEMO-07 five-field form" |
| Owner | One person who launches, monitors and reports the test | |
| Evidence behind it | The data, recording or customer quote that made you pick this test | |
| 2. Hypothesis | Change | Exactly what differs between control and variant |
| Expected effect | Which metric moves, in which direction | |
| Reason | Why you think it will happen, in one sentence about buyer behavior | |
| 3. Metrics | Primary metric | The one metric that decides the result, with its exact event definition |
| Guardrail metrics | Metrics that must not get worse (lead quality, bounce rate, page speed) | |
| Secondary metrics | Interesting but not decisive (scroll depth, clicks to pricing) | |
| 4. Audience | Page(s) | URLs included in the test |
| Who is included | All visitors, or a segment such as paid traffic or a company size band | |
| Who is excluded | Existing customers, internal traffic, bots | |
| Traffic split | Usually 50/50 for two arms | |
| 5. Statistics | Baseline conversion rate | Last 4 to 8 weeks for this page and this metric |
| Minimum detectable effect | Smallest relative lift worth acting on | |
| Confidence and power | Usually 95% and 80% | |
| Sample size per variant | From the A/B test calculator | |
| Planned duration | Sample size divided by weekly traffic, rounded up to whole weeks | |
| 6. Rules | Stop rule | When you will read the result (sample reached, never earlier) |
| Kill rule | What would make you stop early (broken variant, guardrail drop) | |
| Decision rule | What you ship if the result is a win, a loss or inconclusive | |
| 7. Results | Dates run | Start and end date |
| Result per arm | Visitors, conversions, conversion rate | |
| Significance and interval | p-value and confidence interval of the difference | |
| Decision and learning | What you shipped and what it taught you about buyers |
How to Fill It In
- Start from evidence, not ideas. Section 1 asks why this test exists. Good sources: a funnel step with an unusual drop, session recordings, sales call notes, a CRO audit checklist finding. "The CEO wants a new hero" is not evidence.
- Write a falsifiable hypothesis. "If we cut the demo form from nine fields to five, demo submissions per visitor will rise, because buyers abandon when asked for budget before they have seen the product." A hypothesis without a reason teaches you nothing when it fails.
- Pick one primary metric. If you pick three, one of them will probably move by chance and you will want to call it a win.
- Size the test before you build it. Plug the baseline and the minimum detectable effect into the calculator. If the test needs more than a quarter, change the plan: test a bigger change, use a more frequent metric or pick a higher-traffic page. Our guide to A/B test sample size explains the trade-offs.
- Agree the decision rule in advance. Decide what "inconclusive" means for you. Many B2B teams ship the simpler variant when results are flat, which is a fine rule if it is written down before launch.
Illustrative Filled-In Plan
Illustrative numbers for a fictional B2B software company.
| Field | Entry |
|---|---|
| Test name | DEMO-07 five-field demo form |
| Evidence | Form analytics show most abandonment at the "Budget" field; sales confirms they do not use it on the first call |
| Hypothesis | Cutting the form from nine to five fields raises demo submissions per visitor because buyers will not share budget before seeing the product |
| Primary metric | Demo form submissions divided by unique visitors to /demo |
| Guardrails | Share of submissions accepted by sales as qualified must not drop by more than a quarter |
| Audience | All visitors to /demo, excluding logged-in customers and internal IPs; 50/50 split |
| Baseline | 8% of /demo visitors submit |
| Minimum detectable effect | 20% relative (8.0% to 9.6%) |
| Sample size | About 4,900 visitors per variant at 95% confidence and 80% power |
| Duration | /demo gets about 1,500 visitors a week, so about 9,800 visitors in total takes 7 weeks |
| Stop rule | Read result after 7 full weeks or once both arms pass 4,900 visitors, whichever is later |
| Kill rule | Stop if the variant form fails to submit or qualified rate falls by half in the first two weeks |
| Decision rule | Win: ship five fields. Loss: keep nine. Inconclusive: ship five fields, since it is simpler and enrichment fills the gap |
The Test Backlog
The CSV download also includes a backlog sheet so you can rank tests before writing full plans. One row per idea:
| ID | Page | Idea | Evidence | Impact (1-3) | Confidence (1-3) | Ease (1-3) | Score | Weeks needed | Status |
|---|---|---|---|---|---|---|---|---|---|
| DEMO-07 | /demo | Five-field form | Field drop-off | 3 | 2 | 3 | 8 | 7 | Planned |
| HOME-03 | / | Industry-specific hero | ICP visitors bounce more | 3 | 2 | 2 | 7 | 10 | Idea |
Ideas that score high but need more than 12 weeks are better candidates for a holdout comparison or a personalization rule than for a classic A/B test. See A/B testing vs multivariate testing for when each method fits.
Frequently Asked Questions
What should an A/B test plan include?
A hypothesis with a reason, one primary metric with an exact definition, guardrail metrics, the audience and split, baseline rate, minimum detectable effect, sample size, duration, and the stop, kill and decision rules. Add a results section so the plan becomes the record of the test.
What is the difference between an experiment plan and a test brief?
The terms are used interchangeably. Some teams use "brief" for the design and copy request sent to a designer, and "plan" for the statistical setup. This template covers both, with the creative change described in the hypothesis section.
How long should a B2B A/B test run?
Until it reaches the sample size you calculated, and for at least one full week. Many B2B pages need four to twelve weeks. If the math says longer than a quarter, change the test rather than stopping early.
Should I write a plan for every small test?
Yes, though for a button-copy test the plan can be five lines. The stop rule and the primary metric are the two parts you should never skip, because they are what keep you from calling a random swing a win.