Shopify Compatibility Checker A/B Test: Why a Desktop Winner Lowered Mobile Revenue per Visitor by 14.3%

TL;DR

A European car diagnostics brand sells an OBD2 scanner alongside a paid app that unlocks hidden car features, advanced diagnostics and service tools. What a shopper can change depends on the exact make, model and year of their car, so the compatibility checker is where many journeys start.

In June 2026 we tested a new version of the checker. It won on desktop, with +58.25% revenue per visitor and +46.6% conversion rate.

As part of our Shopify A/B testing programme for the store, we then tested a mobile iteration against the original mobile page. It lowered mobile revenue per visitor by 14.3%.

The challenge: the store's biggest entry page sold almost nothing

The compatibility checker is the store's largest entry page. In the 90 days we analysed for our June 2026 audit, it received around 251,000 sessions and converted at 0.04%.

The page answered the shopper's first question, but it did not lead them to the product once the answer was yes.

The research: what the checker needed to do

In our Shopify CRO research, analytics showed how the checker performed, while surveys, reviews and a walkthrough of the store showed why.

• ✓Compatibility: 24 survey mentions named support for their exact car as the deciding factor.
• ✓Compatibility doubt: 19 negative signals came from shoppers who couldn't confirm a feature worked on their car before buying.
• ✓Device split: mobile converted at 0.19% against 0.12% on desktop, which pointed to friction specific to desktop.

The hypothesis: bring the desktop winner to mobile

If we bring the desktop-winning version of the compatibility checker to mobile, mobile shoppers will move from a positive result to the product more often, increasing revenue per visitor.

The variation: the desktop checker on mobile

The variation brought the desktop-winning version of the compatibility checker to mobile. The test measured the combined effect of these changes and does not isolate any single element.

The results: -14.3% revenue per visitor on mobile

We ran the test in Intelligems, comparing the original mobile page with the variation.

The loss came from conversion: 15.4% fewer mobile visitors completed a purchase, while order value rose slightly.

What this test taught us about testing across devices

Devices on this store do not behave alike.

Mobile already converted better than desktop, so the friction the desktop variant fixed was not there on mobile. Our reading is that the same idea changed a mobile journey that was already working.

The desktop result also needed care. Its sample was small, so we judged it on 45 days of data and on the funnel steps: add-to-cart rate rose from 1.73% to 2.84%, and begin-checkout rate from 0.67% to 0.94%.

Find out what your mobile shoppers need from your key pages

We plan, build and run A/B tests for Shopify stores, grounded in customer research rather than guesswork. Get in touch to find out what to test next.