craftberry
Contact Us

Shopify A/B Testing Agency

craftberry is a research-led Shopify A/B testing agency. We study your customers and store data to find what stops people buying, then design, build and run the AB tests in-house.

Every test is measured on revenue per visitor, and every testing program is backed by a performance guarantee.

  • Shopify Premier Partner
  • 30+ in-house specialists
  • 800+ Shopify projects since 2015
  • Intelligems testing partner
  • 4.8 on Clutch from 28 verified reviews

By sending this form you agree that we process your details to answer your enquiry, as described in our privacy policy.

OnVivoPepperTakomoHonestShellyDoctor MIMidiJoseraGlowforgeBultexLucovitaal

Our testing programs in numbers

Ongoing testing programs

5

Concluded tests

50

Winning tests

30

Win rate

60%

What clients say about working with us

4.8 on Clutch from 28 verified reviews.

View All
Clutch
“All of our deliverables were provided on time unless otherwise communicated, and their responses were prompt.”
Richard KrolikowskiE-Commerce Consultant, Midi Health
Clutch
“They don't feel like an agency, but rather part of a team that is just as invested in our company's success as we are.”
Michael SilvaVP E-Commerce & DTC Marketing, Honest Baby Clothing
Clutch
“They recognized that our business was unique and offered bespoke solutions rather than one-size-fits-all features.”
Svetoslav TodorovCEO, DoctorSwitz
Blurred blue and peach gradient

What to expect from our Shopify A/B testing service

01

Who we test for

Shopify and Shopify Plus brands with $2M+ annual revenue and 1,000+ orders a month. We check traffic and conversion rate on the pages we'd test. Lower-traffic stores start with an audit from our Shopify CRO agency team.

02

How we measure success

Our main metric is revenue per visitor: total revenue divided by visitors, so conversion and order value show in one number. We also report conversion rate, AOV and, with cost data, profit per visitor.

03

Testing pace

Retainers are built around 3 to 5 A/B tests a month. We schedule them around your traffic, campaigns and development capacity, and each test runs for at least two full business cycles.

Our Shopify A/B testing guarantee

4x

Our target return in added revenue (a target, not part of the guarantee)

10%

Typical revenue-per-visitor lift we guarantee, agreed with you before the first test

90 days

From your first test launch

What we guarantee on every testing program

Before the first test, we agree your baseline revenue per visitor and a guaranteed lift, typically 10%. Intelligems measures it against a control group running at the same time, so seasonality doesn't skew the result.

If we haven't reached it 90 days after the first launch, we keep testing without a retainer fee until we do.

The guarantee requires stable traffic, winning variants shipped on schedule and a retainer of at least 90 days.

Discuss your testing program

Shopify brands we've worked with

View All
On
Midi
Honest
Doctor MI
Glowforge
Shelly
Bang Cookies
Vivo
Pepper
Takomo
Josera
Bultex
Lucovitaal
Tomorrowlabs
Carista mobile mega menu A/B test: text-link control versus the winning product-card menu, with +11.79% revenue per user and +9.74% conversion rate

How our A/B testing differs from a typical program

Brands that have tested before often tell us the same story: the tests were small, the results were hard to read, and the winners never made it into the theme. These are the parts we run differently.

Test ideas from your own customers

We start from your own customers: about 70% of our testing time goes into research, and every hypothesis links to the evidence behind it.

See how Signalveritas works

How we run a Shopify A/B test

Reconcile the tracking

We reconcile orders and revenue across GA4, Shopify Analytics and Intelligems, so results can be trusted before anything launches.

The team behind every Shopify A/B test

Abstract craftberry brand gradient background
craftberry

One team for the whole testing cycle.

Shopify Premier Partner

Where our Shopify A/B tests won in late 2025 and 2026

Winning tests from our late 2025 and 2026 programs, grouped by store area. Lifts measured in Intelligems against the control.

Store area

Winning tests

Revenue per visitor lift

Collections and product finders

7

+3% to +25%

Product pages

6

+5% to +25%

Navigation and mobile menus

4

+4% to +14%

Homepage

3

+17% to +35%

Landing pages

2

+19% to +42%

Offers and loyalty

2

+8% to +31%

Cart

1

+7%

Powered by Signalveritas, our in-house research tool

Every test we run starts from evidence Signalveritas finds across your reviews, session recordings, analytics and surveys.

Shopify's native A/B testing vs. working with an agency

Shopify's built-in split testing is called Rollouts. It runs Experiments from the admin on the Grow plan or higher and is a low-cost way to start AB testing simple theme and offer changes.

Rollouts can test changes to:

ThemeCheckoutAccounts pagesDiscountsCatalogs

Rollouts tests whatever your team picks. Our tests start from customer research, and if Rollouts can handle an idea, we'll say so.

Liquid templates can't be changed in a rollout, so variants like a new product finder or cart drawer need our developers.

We run them in Intelligems and read the results on profit per visitor.

The developers who coded the test build the winner into your live theme.

Our Shopify A/B testing stack

Testing

Intelligems logo, craftberry's A/B testing partner

Our testing platform and partner, for code-level, price, shipping and offer tests, with revenue and profit per visitor for each variant.

Research

Signalveritas

Our in-house research tool, combining up to eight sources per store, from reviews and surveys to analytics.

Behavior

Microsoft Clarity · Hotjar · Mouseflow

Heatmaps and recordings in the tool your store already uses. We watch them during research and while a variant is live.

Analytics · GA4 & Shopify Analytics

Google Analytics 4 logo

Our cross-check on every result, by device, channel and customer type, reconciled with Intelligems.

Frequently asked questions about Shopify A/B testing

We help brands grow through connected strategy, technology, creative, and marketing.

Discuss your testing program
An AB testing agency researches why shoppers don't buy, then runs controlled experiments that compare your current store with a changed version. Our in-house team handles everything from research to implementation and measures each test on revenue per visitor.
Yes. Shopify Rollouts, under Markets in the admin, runs Experiments on theme, checkout, discount and catalog changes on the Grow plan or higher. Liquid templates can't be changed inside a rollout, so code-level tests need a dedicated tool.
It depends on the test. Rollouts covers simple theme and offer changes, while dedicated tools add code-level variants, price and shipping tests and deeper reporting. We use our testing partner Intelligems, which reports revenue and profit per visitor.
Yes. With Intelligems we test prices, shipping thresholds, bundles, upsells, discounts and gifts. We read the results on profit per visitor, so a lower price that lifts conversion can't hide a loss in margin.
In ecommerce, they usually mean the same thing: traffic is split between the current version and one or more variants. A test with several variants at once is an A/B/n test, and we run both.
We usually work with stores taking 1,000+ orders a month. Before testing, we check traffic and conversion on the pages involved and the size of the effect we need to detect. For lower-traffic stores, an audit with direct implementation is usually more useful.
At least two full business cycles, usually two weeks or more, depending on the sample the test needs. If the evidence doesn't support a clear conclusion, we report the result as inconclusive.
Start where your own research shows shoppers drop off. In our recent programs, most winners came from collections and product finders (7 of 25) and product pages (6 of 25).
Testing tools and variant code can affect speed, so our developers and QA team check loading and key journeys before launch, then monitor the store while tests run.

Ready to discuss your testing program?

Share your Shopify store and monthly order volume. We'll reply within one business day on whether an A/B testing program fits your store and what the next steps could be.

By sending this form you agree that we process your details to answer your enquiry, as described in our privacy policy.

OnVivoPepperTakomoHonestShellyDoctor MIMidiJoseraGlowforgeBultexLucovitaal