Step One Life · AU · Aug 2026 · 2 tests
The product page ran on an iFrame “Islands” architecture — slow and fragile. The rebuild in native Shopify Liquid was gated behind two sequential A/B validations (Men’s, then Women’s) with a maintain-or-improve bar. Both migrations deployed with zero commercial regression — and the tests caught a +39.9% desktop “win” that was pure instrumentation artefact.
CVR, Men’s Liquid PDP (nothing significantly worse anywhere)
RPV, Men’s — climbing all test
Women’s SKU index post-migration — friction closed from ~85
categories migrated behind A/B validation, both shipped
Contents
Step One's product pages ran on an "Islands" architecture — iFrames embedded into the PDP controlling the main functionality. It worked, but it was slow and fragile to iterate on. The decision was made to rebuild the PDP in native Shopify Liquid: same design, faster infrastructure.
The risk is the classic re-platform one: a rebuild that changes no visible design can still quietly break conversion, and you only find out after the revenue is gone. So the migration was gated behind A/B tests — the old Islands PDP against the new Liquid build, run as non-inferiority validations before each category was allowed to migrate.
Two sequential tests on the AU storefront: Men's PDPs first, then Women's. The question each test had to answer was not "is the new page better?" but "is anything meaningfully worse?" — a maintain-or-improve bar across conversion rate, revenue per visitor, add-to-cart and clickthrough, cut by device, visitor type, channel and SKU category.
One risk was flagged at setup: the Liquid build potentially sends fewer pageview events when switching between products, which could show up as a clickthrough decline that isn't a real engagement loss.
| Metric | Change |
|---|---|
| Visitors | ~56k total |
| Conversion rate | +5.1% |
| Revenue per visitor | +7.4% |
| Average order value | +1.9% |
| Add-to-cart rate | +1.8% |
| Clickthrough rate | −3.6% significant |
Nothing commercially meaningful was significantly worse anywhere. CVR and RPV climbed in probability as volume grew rather than fading. The one confirmed negative — CTR −3.6% — tracked exactly with the flagged event-counting risk and never dragged add-to-cart or conversion at any point. Desktop was the cleanest win (+19.8% CVR, significant). The Men's migration deployed.
One genuine worry did surface: Women's items under-indexed in the variant (index ~85) and stayed there across three data pulls as the sample tripled — pointing to friction at the boundary where sessions crossed between the new Liquid Men's pages and the still-Islands Women's pages.
| Metric | Change |
|---|---|
| Visitors | ~77k total |
| Conversion rate | +3.3% |
| Revenue per visitor | +8.8% |
| Average order value | +5.4% |
| Add-to-cart rate | +0.5% |
| Clickthrough rate | −0.5% (flat) |
The safety case was met on every metric that reflects real behaviour, and the CTR penalty from the Men's test did not appear at all. Most importantly, the Women's under-index from the first test closed from ~85 to 94.8 — no longer distinguishable from parity — exactly what the cross-architecture friction hypothesis predicted once both categories shared one build. The Women's migration deployed.
The Women's test produced a desktop CVR read of +39.9% at very high confidence — and it is not a finding. The two arms received significantly different device mixes despite balanced totals, an imbalance extremely unlikely under random assignment. A 40% desktop lift from a same-design rebuild is not a credible product effect; the per-device read was invalid, and the recommendation was to treat all per-device splits on the testing framework as unreliable until audited — including re-checking the Men's test, whose cleanest win had been a desktop number.
The same discipline applied to the headline positives: RPV +8.8% rested entirely on basket size, with no proposed mechanism by which a like-for-like rebuild makes people buy 5% more units. The instruction that went with deployment was explicit — do not carry that number into any forecast.
A re-platform test is a non-inferiority test dressed up as an A/B test. The bar is "nothing worse," and the deliverable is permission to deploy, not a win to celebrate.
Direction is not a mechanism. Numbers that can't be explained — an AOV lift with no cause, a desktop win on a skewed sample — get flagged as artefacts, not banked as wins.
The most valuable finding was cross-test. The Women's under-index only made sense because the first test's data was kept and re-read against the second. Running the two migrations as a programme, not as isolated tests, is what caught and then resolved the friction signal.
Sources: Shoplift experiment reports AU_Theme_CVR_260804 (Men's, 4–11 Aug 2026) and AU_Theme_CVR_260818 (Women's, 18–24 Aug 2026), Step One AU storefront. All figures traceable to the named experiment IDs.