Step One Life · US · Jul 2026 · 2 iterations
Quick-buy on the home page was supposed to shorten the path to purchase. It cut clickthroughs significantly and conversion with it — so the component was rethought around the diagnosed SKU-mix problem and tested again. Same result. Stopped in 4 days and abandoned, because significance wouldn’t have changed the decision.
CVR, iteration 1
CVR, iteration 2 — after the diagnosed fix
clickthrough, statistically significant in both
Boxer Brief unit index — the tile steered buyers off the hero SKU
Contents
If a quick-buy component is added to the home page and key landing pages — letting shoppers select a core product and check out without visiting the PDP — conversion rate will increase because the path to purchase gets shorter.
It is a plausible idea, and it is exactly the kind of plausible idea that loses money when shipped on instinct. It was tested. Then, after the component was rethought around the learnings, it was tested again.
4 days, ~2.7k visitors, US home page:
| Metric | Change |
|---|---|
| Conversion rate | −17.2% |
| Revenue per visitor | −14.8% |
| Clickthrough rate | −12.2% significant |
| Add-to-cart rate | −8.0% |
| Average order value | +2.9% |
Not formally significant on the primary metric, but the leading indicator was: the component significantly cut clicks to product pages, and conversion followed it down. The SKU mix told the sharper story — Boxer Brief, the core product, lost around nine points of unit share (index 88). The component's fixed product set was steering buyers away from the hero SKU toward secondary lines. Decision: do not roll out; investigate the product-selection logic before any re-test.
The second version attacked the diagnosed problem directly: a quick-buy tile explicitly featuring the core styles, placed on the home page and the men's underwear page.
4 days, ~2.7k visitors:
| Metric | Change |
|---|---|
| Conversion rate | −18.1% |
| Revenue per visitor | −15.7% |
| Clickthrough rate | −12.4% significant |
| Add-to-cart rate | −8.6% |
| Average order value | +2.9% |
The same shape, almost the same numbers. Boxer Brief under-indexed again (index 87.6) even with the tile built to promote it. Returning visitors — the programme's leading risk indicator — were hit hardest on every metric in both tests.
Test 2 was stopped at 4 days rather than run the ~13 more days needed for formal significance. That is a deliberate economic call, and it is the point of this case study: when every device, every visitor type and the largest channel point the same way, and the one significant leading indicator agrees, the remaining question is not "is it significant?" but "what would significance change?" Nothing. Two more weeks of sending half the traffic to a worse experience costs real revenue to buy a p-value that wouldn't alter the decision.
The concept was abandoned — not extended, not "re-tested with a different button colour."
A shorter path is not automatically a better path. The PDP was doing work the quick-buy tile couldn't: range, social proof, pack architecture. Removing it removed the selling.
Diagnose before iterating. The second test existed because the first one's SKU mix named a specific mechanism. When the mechanism-level fix reproduced the same result, the concept itself was falsified — that's when you stop.
Kill criteria are part of the test design. Returning visitors degrading first, in both iterations, is the pattern this programme treats as the earliest reliable stop signal.
Sources: Shoplift experiment reports US_HP_CVR_260720 (20–22 Jul 2026) and US_HP_CVR_260724 (24 Jul – 3 Aug 2026), Step One US storefront. All figures traceable to the named experiment IDs.