A/B Testing: What to Test and When It's Worth It

A/B testing occupies an odd position in small business marketing. It is universally recommended, widely attempted, and frequently useless — not because the method is flawed but because it needs a volume of traffic most local businesses do not have on any single page.

The result is a specific and expensive failure: a test runs for nine days, one version is ahead, the business adopts it confidently, and the difference was random variation. That is worse than not testing, because it produces false certainty.

Be honest about whether you have the volume

The uncomfortable rule is that with low traffic, a difference between two versions is usually noise. A page with forty visitors a week cannot distinguish a genuine improvement from an ordinary fluctuation, however long you stare at the numbers.

This does not mean giving up on improvement. It means testing where you actually have volume — often an email list or an ad account rather than a page — and making changes elsewhere on judgement, openly, rather than dressing a guess up as measurement.

Test things that change the offer

When you can test, test the elements substantial enough to produce a detectable difference. The headline, the main call to action, the length of the form, the offer itself — these change what is being asked of the visitor and can move behaviour enough to register.

• Headline and the promise it makes — the largest single lever on most pages.
• Form length, which reliably changes completion rates.
• The specific ask: call versus book versus request a quote.
• Whether a price or range appears at all, which changes both volume and quality.

Button colours and small wording changes are the classic examples in testing folklore and the worst use of a small budget, because the effects are too small to detect at any volume a local business will reach.

If a change is small enough that you need statistics to notice it, you do not have enough traffic to find it.

Run it across a full business cycle

Behaviour differs between weekdays and weekends, between paydays and the days before them. A test that runs Tuesday to Friday has measured a slice of the week rather than the audience.

At least a couple of weeks is a sensible minimum for most local businesses, and the discipline that matters more than the duration is deciding the end date before starting. Stopping the moment one version pulls ahead is the single most common way false results get adopted, because at some point during any test one version is always ahead.

One variable at a time

Testing several changes at once is tempting when each test takes weeks. The cost is that you learn the combination performed better without learning which part did it — and next time you will not know what to keep.

Sequential single-variable tests are slower and far more informative. For a business running perhaps a handful of tests a year, understanding each result properly matters more than getting through more of them.

What to do when you cannot test

Most improvements available to a local business do not require a test to justify. A page that loads faster, a form with fewer fields, a clearer price, an answered phone — these are known improvements, and measuring them formally is a way of delaying them.

Reserve testing for genuine uncertainty where the volume exists to resolve it. Elsewhere, apply judgement and watch enquiries over a longer period.

The page-side improvements are a website matter, the volume for testing usually sits in paid advertising or email marketing. To work out where you have enough traffic to learn something, ask for a proposal.