Our Shopify CRO Process: From Research to Tested Results

A successful CRO programme starts with understanding why customers hesitate, leave or buy. It turns those insights into prioritised changes, measures their effect and uses the results to decide what to do next.

At craftberry, we have built and optimised Shopify stores since 2015. Our in-house team brings together CRO strategists, UX designers, developers and QA specialists to research opportunities, run experiments and implement validated improvements.

In this guide, we explain the eight steps in our Shopify CRO process, when to start testing and how we use research and revenue data to make decisions. We also share results from real A/B tests across our client programmes.

Key takeaways

• ✓A CRO process is a repeatable cycle of measurement, research, prioritisation, testing and implementation.
• ✓A CRO strategy sets the business goals and priorities. The process turns them into actions and measurable decisions.
• ✓We generally look for around 1,000 monthly orders before recommending a continuous A/B testing programme. Lower-volume stores can still benefit from research and an audit.
• ✓We assess revenue per visitor alongside conversion rate and average order value to understand the commercial impact of a change.
• ✓Across six programmes, we concluded 68 tests from January to October 2026. Product pages, navigation and collection pages accounted for 43 of them.
• ✓Results from one store are evidence for that store. They are a starting point for a hypothesis, not a guarantee that the same change will work elsewhere.

What is a CRO process?

A CRO process is a repeatable method for improving how a website turns visitors into customers and revenue. It involves checking measurement, researching customer behaviour, identifying opportunities, prioritising changes, testing where traffic allows, and using the results to guide the next decisions.

A CRO strategy sets the direction: which business goals, customer journeys and opportunities matter most.

A CRO process puts that strategy into practice. It defines how the team finds problems, decides what to test and evaluates the outcome.

For example, a strategy might focus on increasing revenue from product pages. The process then helps the team identify what prevents customers from buying, choose a change to test and measure whether it improves the result.

Our Shopify CRO process follows eight steps:

1. 1Validate tracking and establish a baseline.
2. 2Research customer behaviour and identify friction.
3. 3Turn research findings into hypotheses.
4. 4Prioritise opportunities.
5. 5Design, develop and quality-check the changes.
6. 6Run controlled tests when traffic allows.
7. 7Analyse conversion and revenue metrics.
8. 8Implement the appropriate changes and document the learning.

The process is a loop. Each result informs the next round of research and testing.

When should a Shopify store start CRO?

A store can start researching conversion opportunities at any stage.

The right time to begin A/B testing depends on whether it has reliable tracking, enough traffic and enough conversions to produce useful results within a practical timeframe.

After a new store launch or migration, we generally allow six to eight weeks for the initial baseline to settle before starting a testing programme. That gives the team time to check tracking and understand how customers use the new experience. The right timing depends on the store, its traffic and any changes happening after launch.

For continuous A/B testing, we generally look for around 1,000 orders per month as an initial qualification point. The relevant volume is the traffic and conversions reaching the page or element being tested.

For example, a store might receive 1,000 orders overall but have a low-traffic landing page. A test on that page may still take too long to provide a useful answer.

For stores with less traffic, we typically recommend a CRO audit and prioritised improvements first. The team can fix clear usability or measurement issues, monitor performance, and revisit testing as the store grows.

Step 1: Validate tracking and establish a baseline

Before researching opportunities or launching tests, we check whether the store’s data is reliable enough to support decisions.

We review the measurement setup across Shopify Analytics, GA4, and the testing platform. We also check key events and purchase tracking on desktop and mobile.

Different platforms can report different numbers because they use different attribution methods, definitions, and tracking rules. The goal is to understand material discrepancies and fix issues that could distort the analysis.

Common problems include:

• ✓Missing or duplicated purchase events.
• ✓Tracking affected by a theme or checkout change.
• ✓Old app scripts sending duplicate events.
• ✓Important steps in the customer journey that are not measured.
• ✓Behaviour tools collecting too few sessions for the questions the team wants to answer.

Once the data is dependable, we establish a baseline for the relevant pages and metrics.

That gives the team a reference point for research and helps identify where further investigation is worthwhile.

Step 2: Research where customers hesitate and why

A CRO audit combines quantitative and qualitative research to identify where customers experience friction and what might be causing it.

Analytics can show where visitors leave. Session recordings can show what they do before leaving. Reviews, surveys and support conversations can help explain why they hesitate.

No single source tells the whole story. We look for patterns across several sources before deciding what to investigate further.

We start with a broad review of the customer journey, then investigate the areas with the strongest combination of friction and business potential.

Our research tool, Signalveritas, helps surface opportunities across store data. The CRO team then reviews those findings alongside customer research and business context.

From craftberry A/B tests:

For a car diagnostics brand, research across more than 500 post-purchase survey responses and reviews showed that 29 buyers first heard about the product in a car forum or social group. It was the second most cited reason to buy, yet the store’s video section showed YouTube channel content rather than other drivers.

We replaced those videos with short clips filmed by customers using the product in their own cars. Nothing else on the page changed. Revenue per visitor increased by 19.5% and conversion rate by 13.7%.

Step 3: Turn findings into testable hypotheses

A finding becomes valuable when the team can turn it into a clear decision.

A hypothesis connects four things:

1. 1The problem observed in the research.
2. 2The evidence behind it.
3. 3The change the team proposes.
4. 4The outcome the team expects to measure.

A useful format is:

Because we observed [evidence of a problem], we believe [change] will help [specific customers] do [desired action], measured by [primary metric].

For example:

Because product reviews appear low on the page and many visitors do not reach them, we believe moving relevant reviews closer to the top will help customers evaluate the product sooner, measured by revenue per visitor.

From craftberry A/B tests:

This was the hypothesis behind one of our tests for an electronics brand.

It already had reviews from creators its customers recognised, but they appeared at the end of the landing page, where many visitors did not reach them.

We moved the reviews directly below the hero and refreshed the thumbnails so the creators were easier to recognise. After four weeks, conversion rate rose by 21.01% and revenue per visitor by 21.63%.

The opportunity was not a lack of reviews. It was whether customers saw them early enough to use them.

This makes the test easier to design and interpret. If the variation does not improve the result, the team can examine whether the proposed solution, its placement or the underlying assumption needs further investigation.

A hypothesis does not need to be complicated. It needs to explain why the change is worth testing and what the team expects to learn.

Step 4: Prioritise the CRO roadmap

Most audits uncover more opportunities than a team can test at once. Prioritisation helps decide which ideas deserve traffic, design time and development resources first.

We use research evidence alongside factors such as:

• ✓The amount and quality of traffic exposed to the change.
• ✓The importance of the page or journey to revenue.
• ✓The strength of the evidence behind the hypothesis.
• ✓How noticeable the change will be to customers.
• ✓The effort required to design, build and QA it.
• ✓Any dependencies or risks.

One framework we use is PXL, developed by CXL. It replaces broad estimates with more specific questions about evidence, visibility, reach and implementation. For a simpler first ranking, a team may use ICE: impact, confidence and ease.

A high score does not mean a test will win. It means the idea has a stronger case for being tested earlier.

From craftberry A/B tests:

A smart home brand offered a device individually, in a two-pack and in a four-pack, but gave customers little guidance on how to compare them.

Adding "Most Popular" to the two-pack and "Best Value" to the four-pack scored well on the factors above: customers would notice it immediately, and it needed no new design, no price change and almost no development.

With prices and layout unchanged, the test increased conversion rate by 11% and revenue per visitor by 8%.

Some issues do not need an experiment. If a broken button prevents customers from adding a product to their cart, the priority is to fix it.

Testing is more useful when the team needs to compare reasonable alternatives and does not already know which one will perform better.

Step 5: Design, develop and QA each change

Once a hypothesis is prioritised, the team turns it into an experience that can be tested reliably.

At craftberry, the CRO strategist defines the hypothesis and requirements. A UX designer creates the variation, a Shopify developer builds it, and QA checks the experience before launch. The same team can also implement a successful variation in the live theme.

QA covers the details that might affect the result:

• ✓Does the variation work across relevant devices and browsers?
• ✓Are the correct products, prices and offers displayed?
• ✓Do buttons, links and forms work?
• ✓Does the test affect page speed or introduce visual flicker?
• ✓Are visitors assigned to the intended experience?
• ✓Are the agreed metrics being recorded?

A broken variation can lose because of a technical issue rather than the idea itself. That makes QA part of the research process: it protects the validity of what the team learns.

Step 6: Run a controlled A/B test

An A/B test compares the existing experience, called the control, with a changed version, called the variant. Eligible visitors are assigned to the versions, and the team compares their outcomes.

Before launch, we define:

• ✓The eligible audience and traffic split.
• ✓The primary success metric and relevant secondary metrics.
• ✓The expected test duration and sample requirements.
• ✓The conditions for stopping or evaluating the test.
• ✓Any campaigns, promotions or other changes that could affect interpretation.

For Shopify A/B testing, we use Intelligems to measure metrics including conversion rate, average order value and revenue per visitor.

We typically aim for 95% statistical significance and run tests across at least two full business cycles, often two weeks. For lower-volume situations, we may use a 90% threshold where that trade-off is agreed in advance.

Neither two weeks nor a significance threshold on its own guarantees a reliable decision. The required sample depends on the baseline, traffic, expected effect and testing method. Some tests need more time; others may end without a conclusive result.

We also monitor for technical problems and material risk during a test. Any early-stop decision needs to account for the testing platform’s method and the rules agreed before launch. Stopping as soon as a variant looks positive can produce misleading conclusions.

From craftberry A/B tests:

When we redesigned the mobile menu for a car diagnostics brand, we limited the test to mobile visitors, because that was where the original menu hid the products behind three text links. The variant showed product cards and a compatibility check as soon as the menu opened, with the same destinations as before. Among mobile visitors, revenue per visitor rose by 11.8% and conversion rate by 9.7%.

Step 7: Analyse conversion rate and revenue per visitor

Conversion rate tells you what share of visitors completed a purchase. Revenue per visitor, or RPV, shows how much revenue the store generated per visitor.

The metrics answer different questions:

A change can increase conversion rate while reducing average order value enough to lower revenue per visitor. The opposite can also happen: fewer people buy, but a higher order value increases revenue per visitor.

That is why we review the metrics together and generally use RPV as the primary commercial measure for our tests. Where the business has the data available, we also consider margin and profit: more revenue does not automatically mean more profit.

From craftberry A/B tests:

For a car diagnostics brand, a dedicated landing page for a scanner-and-app bundle increased conversion rate by 20.6%.

Average order value fell slightly, by 1.6%, so revenue per visitor rose by 18.7%.

The test was still a clear win, but reporting conversion rate alone would have overstated the commercial gain.

Step 8: Implement the decision and document the learning

A test can produce three broad outcomes:

• ✓A winning variant: the evidence supports implementing the change against the agreed criteria.
• ✓A losing variant: the evidence suggests the change performed worse on the metric being evaluated.
• ✓An inconclusive result: the test did not provide enough evidence to make a confident choice.

For a winning test, our developers build the change into the live Shopify theme and QA the implementation. We then remove temporary test code where appropriate and monitor the live experience.

For a losing or inconclusive test, we document what happened and decide whether to revise the idea, investigate further or move on. A loss does not always prove the original customer problem was wrong. The execution, placement, offer or audience may have affected the outcome.

Each test record should include the hypothesis, research, audience, dates, control, variant, metrics and decision. Over time, that record helps the team avoid repeating the same work and build a clearer picture of what matters to the store’s customers.

Four of our strongest recent Shopify A/B tests

These four tests come from two stores in our 2026 programmes: a European car diagnostics brand and a European sleep therapy equipment retailer.

Each example shows how we identified an opportunity, tested a change and measured the result. Customer research also helped us understand the findings and inform further work.

The figures are results from individual tests, so they should not be treated as expected outcomes for every store.

Subscription landing page: explain what the price buys

A car diagnostics brand sells an OBD2 scanner and a paid app subscription that unlocks the features customers buy it for.

The subscription was sold through a standard product template designed for a physical product. It showed the price, a plan selector and an add-to-cart button, with little room to explain what the plan included or how renewal worked.

We tested a dedicated landing page that explained the subscription before asking customers to choose a plan. The variant increased revenue per visitor by 42.4% and conversion rate by 55.0%.

Later, research across more than 500 survey responses and reviews confirmed why the change mattered: 36 negative signals came from customers who had not realised the app required a paid plan.

Homepage trust signals: show the reasons customers already gave

For one retailer, reviews praised expert support 35 times and fast dispatch 26 times. Another 11 support chats included questions about whether customers could pay later.

Yet the homepage hero promoted a single product, while delivery, returns and support information appeared further down the page, after category tiles and product carousels.

We moved the Trustpilot rating and the store’s service promises above the fold. Over 30 days, revenue per visitor increased by 35.4%, conversion rate by 7.0% and average order value by 26.5%.

Most of the revenue gain came from the increase in average order value. The result showed why we assess RPV alongside conversion rate: the larger commercial impact did not come from conversion alone.

Rewards button: test the words before the incentive

A car diagnostics store displayed a floating rewards button on every page, making its label one of the most visible pieces of copy on the site.

The original label named the programme but did not communicate the benefit or connect it to the paid Pro plan sold across the store.

We changed only the wording to “Unlock PRO.” Revenue per visitor increased by 31.2%, conversion rate by 23.4% and average order value by 6.4%.

Because the wording was the only change, the test isolated its effect. It became one of the fastest and cleanest experiments in the programme.

Product cards: help customers compare on the collection page

At a sleep therapy store, 96 out of every 100 visitors who viewed a product did not add it to the cart.

Similar masks had near-identical names and photos, customers asked about machine compatibility in 13 support chats, and sale prices looked almost the same as full prices.

We redesigned the product cards to show the percentage saving, a compatibility line, feature tags and the customer rating. Over 19 days, revenue per visitor increased by 28.2% and conversion rate by 20.6%.

The new cards made it easier for customers to compare products and understand key differences before opening a product page.

Where our 2026 CRO tests ran

From January to October 2026, we concluded 68 A/B tests across six Shopify programmes. Of those, 43 tests focused on product pages, navigation or collection pages.

Product discovery was a frequent focus across these programmes: customers need to find a relevant product, understand it and feel confident in their choice before they reach the cart.

That does not mean every Shopify store should begin with the same pages. A store’s traffic, catalogue, customer questions and existing experience determine where research should start.

Methodology: craftberry’s 2026 test log covered six client programmes and was updated on 22 October 2026. The count includes concluded winning, losing, neutral and inconclusive tests; it excludes tests still running, planned tests and changes implemented without a test. Each test was assigned to one page area. Percentages are rounded, so they may not total 100%.

Explore more control-and-variant examples in our Shopify A/B test library.

What does a 90-day CRO roadmap look like?

The first 90 days establish the measurement, research and workflow needed for an ongoing programme. The exact timing depends on the store’s tracking, traffic, technical setup and the complexity of the first tests.

Some tests may not reach a useful conclusion within the first 90 days. Others may identify technical issues that need to be addressed before a programme can proceed.

Once the programme is established, a typical retainer can include three to five tests per month, depending on traffic, complexity and the number of opportunities that can be tested without interference.

When is A/B testing not the right first step?

A/B testing is useful when a store has enough relevant traffic and conversions to compare alternatives in a reasonable timeframe. It is less useful when the foundation is not ready.

We may recommend starting elsewhere when:

• ✓Tracking is unreliable. The team needs dependable measurement before interpreting test results.
• ✓The tested page has too little traffic. Even a high-volume store can have individual pages that cannot support a practical test.
• ✓A critical issue is already clear. Broken functionality should be fixed rather than left in place for an experiment.
• ✓The store has just launched or migrated. The team may need time to establish a stable baseline and resolve launch issues.
• ✓The test would take too long to inform a decision. The resources may be better spent on research, implementation and monitoring.

In those cases, a Shopify CRO audit can identify the highest-priority issues and help the business decide when testing becomes viable.

Turn customer insights into a CRO roadmap

If your Shopify store has established traffic but you are unsure where to focus your next improvements, a CRO audit can help identify the friction worth addressing first.

Our in-house team combines research, UX, development and testing to turn those findings into a practical roadmap.

Talk to our CRO team