How it works

From the test you create to the order that counts

Every test goes through six steps: created, published, decided, collected, matched to Shopify's orders and read. Here is what happens at each one and where it runs.

Book a call Developer docs

One test, six steps

Where each step runs

Createcro.growthdept.ai
Publishcro.growthdept.ai and your Shopify store
DecideThe shopper's browser, your Next.js middleware or a Cloudflare Worker
Collectcro.growthdept.ai
Matchcro.growthdept.ai, reading Shopify
Readcro.growthdept.ai

At a glance

Three things to know first

Your store decides

The group is picked from set-up already in the page, or by your own server, before anything paints. No server round trip.

Shopify keeps the score

Purchases count only when Shopify's own order record backs them.

Nothing is called early

Sample ratio and data-quality checks run before a winner is shown.

Step by step

The six steps of a test

Step 1CreateYou create a test in the app, the assistant drafts one, or an agent creates a draft through the API. Starting it needs a live runtime and a test that passes validation.cro.growthdept.ai
Step 2PublishThe publisher builds the set-up (running and QA tests, live allocation weights), records its revision, and writes it to your store and to the config endpoint.cro.growthdept.ai and your Shopify store
Step 3DecideThe runtime picks each visitor's group from a salted hash, keeps it in the _cro cookie and shows the group before the first paint.The shopper's browser, your Next.js middleware or a Cloudflare Worker
Step 4CollectExposures and page views go to the collector; the pixel sends add to cart, checkout and purchase events with the visitor's groups.cro.growthdept.ai
Step 5MatchThe order sync reads your Shopify orders every five minutes. An order counts for a group only when Shopify's record carries the visitor's groups or confirms a pixel purchase.cro.growthdept.ai, reading Shopify
Step 6ReadResults are worked out from raw per-visitor rows, with corrected comparisons, the chance to beat control, sequential boundaries and quality gates.cro.growthdept.ai

Orders

Shopify's record decides what counts

The visitor's groups travel into the cart as the _cro attribute and stay on the order.
Only real, uncancelled, non-test orders are confirmed.
Refunds and returned items come off product revenue.

Product shot beats the current hero

Data looks sound · 92/100

Revenue per visitor up 8.4% (95% interval +2.1% to +14.9%). Chance to beat the current hero: 97%.

Chance to beat control97%

QA check

Passed

Published set-up reached the store
App embed on in the live theme
Group B's section selector found
Redirect destination answers 200
Order sync ran in the last 5 minutes

The parts

What runs where

In your store

A theme app embed or a snippet in the theme, a web pixel for cart, checkout and purchase events, and Shopify extensions for offers, prices, shipping, checkout and post-purchase tests.

On the platform

The app, API, collector and config endpoint at cro.growthdept.ai, a job queue for order sync, allocation and schedules, and a Postgres database with row-level security.

About shoppers

No names, emails, phones, addresses or IP addresses are stored. A post-purchase greeting reads the first name when the offer shows and keeps nothing. Customer ids are hashed with your brand's key, and message recipients are kept as salted hashes.

FAQ

Common questions

Does a test slow the page down?

On a Shopify theme the decision runs inline, from set-up already in the page, before the first paint. With no running test the embed renders nothing.

What happens if the script fails to load?

Shoppers keep seeing the control, and that view is not counted. Default CSS shows only the control until the runtime marks the page ready.

Do redirects break my ad attribution?

Split URL redirects keep the whole query string, including UTM tags and click ids. The original referrer is saved in the session and as a cart attribute.

How do I check a test before shoppers see it?

Make it QA only and open the store with ?cro_qa=1, or force a group with ?cro=<test>-<group>. QA visitors are marked and left out of the results.

Can one visitor be in several tests?

Yes, unless the tests share an exclusion group. Each test buckets the visitor separately from the same salted hash.

The full breakdown

Every test type, side by side

Test type	What changes	Where it is decided	Shopify needs
Split URL and templates	Which address, or which template, a visitor gets	Browser, middleware or Worker	Any plan
Content	Which sections show on the same address	Browser before first paint, or server	Any plan
Theme	Which theme renders the visitor's next pages	Browser, through Shopify's theme preview	Any plan
Offer	Discounts, gifts, bundles and upsells	Shopify discount Function	Shopify Plus
Price	The price of chosen variants	Price swap before paint, Shopify cart transform	Shopify Plus
Shipping	Which shipping rate shows at checkout	Shopify delivery customization	A custom app on Shopify Plus
Checkout	Blocks inside checkout	Shopify checkout UI extension	Shopify Plus
Post-purchase	A one-click offer after payment	Shopify post-purchase extension	Shopify's approval of the app (coming)
Feature flag	A value your theme or app code reads	Browser, middleware or Worker	Any plan
Multivariate	Every combination of several changes	Browser before first paint, or server	Any plan
Email and SMS	Which version of a message a recipient gets	Your sender, through the API	Any plan

Find out what is worth testing on your store.

Book a 45-minute call with the Growth Dept team. We look at your store and your traffic, and tell you which tests to run first.

Book a call