Conversion rate optimisation: what it actually means

Most conversion rate optimisation gets sold as a subscription: a fixed number of tests a month, a dashboard of uplift percentages, and very little explanation of what a test actually needs before it is worth running. The buyer usually finds out months in that their site never had the traffic to make any of those results trustworthy, and that the metric everyone was celebrating can rise or fall for reasons that have nothing to do with the website at all.

Conversion rate optimisation (CRO, the practice of increasing the share of site visitors who complete a wanted action, usually a purchase or a lead) is a measurement and research discipline, not a synonym for split-testing button colours or page copy. Conversion rate itself (the number of conversions divided by the number of sessions) moves whenever the mix of traffic moves, so a rising rate on its own is not proof that anything on the site improved. This article covers the arithmetic behind conversion rate and where it breaks, what genuinely counts as a conversion, why most UK sites do not have the traffic to run a statistically valid A/B test in any sane timeframe, what CRO looks like without that volume, how it sits alongside SEO and paid spend, and when to leave testing alone.

This is written for the marketing lead or founder being pitched a CRO programme, or already paying for one, before more money changes hands. It is an explainer, not a testing playbook: it tells you what the discipline actually is and whether your site is in a position to benefit from it, not how to run any specific test.

What CRO actually is, and why testing is the smallest part of it

Conversion rate optimisation is the discipline of increasing the proportion of visitors who complete an action you care about, using evidence rather than opinion. That evidence comes from three places: analytics (what people actually do on the site), qualitative research (session recordings, on-site surveys, structured usability testing, anything that shows why they do it), and controlled experimentation (testing a specific change against a control to see whether it moves the outcome).

Testing is the part that gets marketed, and in the accounts we run it is usually the smallest part of the discipline in terms of hours worked, not the largest. It also depends most heavily on something a site either has or does not have: enough traffic to tell a real result apart from noise. The analytics and research work has to happen first, because it tells you which change is worth testing at all, and because a test built on bad data will confidently produce the wrong answer. Most of the CRO conversations I have start from the opposite assumption, that testing is the main event.

The arithmetic, and the two different reasons it moves

Conversion rate is conversions divided by sessions, expressed as a percentage. A thousand sessions and twenty purchases is a 2% conversion rate. Nothing about that number tells you whether it is good, because it is a ratio, and a ratio can move for two entirely different reasons that look identical on a dashboard.

The first reason is real: the site genuinely converts a larger share of the same kind of visitor than it did before, because a form got shorter, a price got clearer, or a checkout step got removed. The second reason is not: the mix of visitors changed, so the same site is now being measured against a different, more or less qualified, audience. Both move the percentage by exactly the same amount. The dashboard cannot tell you which one happened. You have to go and check.

Why a rising conversion rate is not proof of anything

Different traffic sources convert at wildly different rates for reasons that have nothing to do with the site. Someone who typed your brand name into Google, or clicked a link in an email they signed up for, is closer to buying than someone who saw a cold prospecting ad forty seconds ago. Blend those audiences into one site-wide conversion rate and the number tells you more about your media mix than your website.

The failure mode. A team pauses a broad prospecting campaign that had been bringing in cheap, low-intent clicks. Overall conversion rate climbs the following month, because the visitors left behind are more qualified, not because anything on the site changed. Total conversions and total revenue can fall in the same month the celebrated metric goes up. I have seen this land in a board deck as a CRO win in the exact month the paid channel that had been driving that volume got its budget cut. Nobody had lied. Nobody had checked the traffic mix either.

The fix is not complicated: read conversion rate by channel and by landing page, never as one blended site-wide number, and check what happened to session volume and channel mix before crediting or blaming anything else.

What counts as a conversion, and the soft-conversion trap

A hard conversion maps directly to revenue: a completed purchase, a qualified lead, a booked call, a phone call that turns into a sale. A soft conversion, sometimes called a micro-conversion, is a step along the way that shows intent without proving it: an email signup, an item added to a wishlist, a basket that never reaches checkout.

Soft conversions are genuinely useful diagnostically. They show where interest exists without showing where it is being lost. The trap is reporting: hard conversions are difficult to move and take time to show up, so when a testing programme needs something to report this month, the metric quietly drifts toward whichever one is moving. A test that lifts add-to-basket rate by a healthy margin and does nothing to completed purchases has not improved the business. Before believing any CRO uplift claim, including your own, ask which conversion it was measured against.

Why most sites don't have the traffic to test validly

A valid A/B test needs three inputs before it starts: your baseline conversion rate, the smallest lift worth detecting (the minimum detectable effect), and the confidence level you are testing to, conventionally 95%. Feed those into any significance calculator and it tells you the sample size the test needs before the result means anything.

Nielsen Norman Group ran the actual numbers on an ecommerce example: detecting a genuine 10% lift in sales needed a sample in the region of 14,000 users; detecting a 2% lift needed closer to 340,000. Their own testing primer puts the general case plainly: a valid test often needs thousands of users interacting with the product before it reaches statistical significance. That is the volume a single page, on a single site, needs to move through a single variant, not the volume the whole site gets in a month.

In the CRO conversations we have, the pattern shows up before anyone runs a calculator: if a site's monthly conversions fit in a spreadsheet you can read without scrolling, a formal test on one page element needs months to say anything with confidence, and by month three the catalogue, the price, or the season has usually already changed under it. Calling the test early on a result that looks promising after two weeks does not fix that. It just means the winning variant was picked by noise.

What CRO looks like without that volume

Most sites are in this position, and CRO does not stop being useful there, it just stops being about split testing. What still works without traffic is qualitative research, funnel analysis, structured usability testing and heuristic review, used together.

Qualitative research: watching real sessions and reading on-site survey answers shows where people hesitate or give up, which a conversion rate never explains on its own.

Funnel analysis in your existing analytics: step-by-step drop-off through checkout or a lead form usually shows one or two points where most of the loss happens, long before you would have enough volume to test a fix.

Structured usability testing with a handful of real users: around five sessions, watched rather than guided, tends to surface the obvious friction that a split test would take months to prove statistically. Clients often want to skip this stage because it feels less rigorous than a test with a percentage attached, but it usually finds the same problem a test would take three months to confirm.

Heuristic review against known friction patterns: checkout step count, form length, page speed, trust signals near the payment step, mobile tap targets. None of this needs a confidence interval. It needs someone who has looked at enough sites to recognise the pattern.

None of that produces a percentage uplift with statistical backing. What it produces is a reasoned, evidence-led change that you ship because everything above points the same way, not because a calculator declared it significant. On low-traffic sites that is the honest version of CRO. Treat anyone promising a monthly test cadence and a monthly uplift number on a site that does not have the volume to support either with real suspicion.

How CRO sits alongside SEO and paid spend

Acquisition and conversion are the same funnel, measured from opposite ends. Sending more traffic, whether earned through SEO or bought through paid search, into a site that loses a predictable share of visitors at the same point every month does not fix that loss; it just pays to refill it every month. A genuinely improved conversion rate compounds every channel's return without a penny of extra spend, which is the real commercial case for doing CRO work at all.

The reverse also matters, and it is the point the traffic-mix section makes from a different angle: because channel mix moves conversion rate, CRO reporting and acquisition reporting have to sit in the same conversation. I see this constantly: the two teams rarely look at the same chart at the same time, so a conversion rate chart read in isolation from what happened to traffic and channel mix that month reads as a website story when it might just be a media story.

When CRO is the wrong tool right now

Three situations mean a formal CRO programme should wait, whatever an agency's pitch deck says.

The tracking cannot be trusted yet. If analytics double-counts conversions, misses a channel, or cannot separate real purchases from test orders, no test result built on top of it means anything. The first thing I check before agreeing that a CRO programme is the right next step is not the funnel, it is whether the numbers underneath it are real. On 18 Carati, the jewellery client whose Magento 2.4 store I run SEO, tracking and paid media for, GA4 was recording purchases with no revenue attached while the store was genuinely selling, because the ecommerce events were arriving without their value payload. That looked, from a distance, exactly like a conversion problem, and the client knew something was wrong well before we arrived; where to put their hands was the missing part, and once the right layer was opened the cause took an afternoon to find. Getting it right meant specifying a rebuilt GA4 setup, six events and four consent signals end to end, before conversion rate meant anything on that account. More often than the industry likes to admit, what gets diagnosed as a conversion problem turns out, on inspection, to be a measurement problem, and testing on top of unverified numbers just produces confident, wrong answers faster. What a CRO audit should cover walks through that check in the order I run it.

The traffic is not there yet. The maths in the sample size section does not change because a deadline is close. If a significance calculator says a test needs months to reach the volumes involved, that is not a reason to test anyway and call it early, it is a reason to do the qualitative work above instead.

The real constraint sits upstream of the site. Wrong audience, wrong price for that audience, a genuine product-market mismatch: no landing page change I have seen fixes any of those, and a testing programme aimed at the wrong layer of the problem burns budget proving that the page was never the issue.

If none of those three apply, and the numbers can be trusted, CRO is worth commissioning properly, and it should start with the measurement layer before a single test gets designed. That is the order our CRO work follows.