A/B test brief for landing page personalization
An A/B test brief is a short document written before a test that states what will change, for whom, which metric decides, how big a sample is needed and when the test stops. Writing it first prevents moving the goalposts after the results come in.
When to use it: Use it every time you test a personalized page against the original, or two personalization approaches against each other.
The brief
- Test name
- [2026-10 Google Ads competitor headline]
- Owner and reviewers
- Who runs it, who signs off on the result.
- Hypothesis
- Because [observation], changing [element] for [audience] will increase [metric], measured by [method].
- Audience
- Which traffic enters the test: channel, campaign, UTM values, new vs. returning.
- Control
- The original page, unchanged.
- Variant(s)
- What changes: headline, supporting copy, CTA. Paste the exact copy or the instruction given to the tool.
- Traffic split
- For example 50/50, or 90/10 if the control is a holdout.
- Primary metric
- One metric that decides: demo requests, trial signups, qualified pipeline.
- Guardrail metrics
- Metrics that must not get worse: bounce rate, lead quality, unsubscribe rate.
- Minimum detectable effect and sample size
- Use a sample size calculator with your baseline conversion rate. Write the number down before launch.
- Confidence level
- For example 95%.
- Stopping rule
- Stop when the sample size is reached, or after [n] weeks, whichever comes first. No peeking-based early stops.
- Decision
- What you will do if the variant wins, loses or is inconclusive.
- Result and learning
- Filled in after the test: numbers, decision, what to try next.
Automate this in GetIntent
GetIntent Experiments (Pro plan and above) keep a share of visitors on your unchanged original page (10% by default) and split the rest between personalization variants, each with its own AI instructions. You set the holdout, weights, UTM targeting, minimum sample size and confidence level; results show the lift against the original page per conversion goal with a confidence interval, and each variant's improvement over the control, from a two-proportion z-test. Optionally, traffic shifts to the best variant with Thompson sampling while the holdout keeps its share, and daily results export to CSV.