Why it matters
A/B testing replaces opinions with evidence. Instead of arguing over whether the button should be green or the headline shorter, you split traffic at random, send half to the original (the control) and half to the change (the variant), and see which gets more of the action you care about. Because both run at the same time, season, ad spend and day of the week affect them equally. Google Ads offers the same logic as campaign experiments.
Done properly, it turns small gains into compounding ones. Done badly, it turns noise into confident wrong decisions.
How to run one
- Pick one metric, tied to money: bookings, purchases, qualified leads, not clicks.
- Change one thing, or one clear idea. If three things change you will not know which one worked.
- Estimate the sample first. Use a sample size calculator with your current rate and the smallest lift worth detecting.
- Split randomly and run full weeks, so weekday and weekend behaviour both count.
- Decide the end in advance, and do not stop the moment the numbers look good.
When not to test
If a page gets 600 visitors a month and converts 3 in 100, a test would take many months to detect anything but a huge change. A school or clinic with low traffic gains more from fixing obvious problems, using customer calls and session recordings, and then comparing before and after.
Common mistakes
- Peeking and stopping early, which makes false winners.
- Testing trivial changes on low traffic.
- Ignoring segments: a win on desktop can be a loss on phones.
- Not recording the result, so the same test runs again next year.
BridgeE runs this as part of conversion rate optimisation, and says plainly when the traffic is too low to test.