@shopkit/ab

Section-level A/B testing for the storefront edge.

Answers one question per request: does this URL belong to a running experiment, and if so, which arm is this visitor in?

Everything else — authoring variants, setting the split, promoting a winner — happens in the visual editor and the admin dashboard.

Constraints this package is built to

It runs in Next.js middleware, on the request path, for every merchant — including the ones who never use A/B. So:

• No React, no DOM, no dependencies. ~2.6 kB in the edge bundle.
• Nothing throws. Every public function has a safe return.
• Fails open. A broken backend degrades A/B testing and nothing else.

Usage

import {
  getManifestSync, matchExperiment, isExcludedPath, pageTypeOf,
  drawArm, readArmCookie, serialiseArmCookie, selectionFor,
  ARM_COOKIE, ARM_COOKIE_MAX_AGE_S,
} from "@shopkit/ab";

try {
  if (isExcludedPath(pathname)) return response;
  if (!pageTypeOf(pathname)) return response;      // /cart, /search, /account…

  // Never awaits the network — middleware runs ahead of every render.
  const manifest = getManifestSync({ baseUrl, themeId });
  const exp = matchExperiment(pathname, manifest);
  if (!exp) return response;

  const stored = readArmCookie(request.cookies.get(ARM_COOKIE)?.value);
  const { arm, position } = drawArm(exp, stored);

  const out = NextResponse.rewrite(
    new URL(`${pathname}/variant/${arm}`, request.url),
  );

  // Store the POSITION, not the arm. The arm is derived from it and the
  // current split, which is what keeps a later split change cheap.
  out.cookies.set(ARM_COOKIE, serialiseArmCookie({ ...stored, [exp.id]: { position } }), {
    maxAge: ARM_COOKIE_MAX_AGE_S, path: "/", sameSite: "lax",
  });

  return out;
} catch {
  // Never break a request over an experiment.
}

Both arms are rewritten, including a. After a promote liveVariantId is b, so the untouched route would serve B to an arm-A visitor and the test would compare B with itself. The manifest carries no live orientation, so the edge cannot tell a promoted slot from a fresh one.

Behaviour worth knowing

| | | | --- | --- | | Manifest TTL | 10s per pod. getManifestSync never blocks, so a short window costs no latency | | Timeout | 1500ms, then the request proceeds as "no experiments" | | Failure backoff | 5s — a down backend costs one attempt per window, not one per request | | Concurrent misses | Coalesced to a single fetch | | Assignment | A stable 0–100 position in one cookie; the arm is position < split |

That last row is what makes re-aiming a test cheap. Moving 50 → 55 moves only the visitors sitting between 50 and 55 — about 5% of them. Drawing a fresh random on every split change produced the same ratio while moving half the audience, which reads as correct on a dashboard and quietly makes the two halves of the test different audiences.

A cookie written before positions existed keeps its arm and gains a position that agrees with it, so shipping this does not reshuffle an in-flight test.

Not in scope

Rendering. This package never sees page content and does not know what a variant contains — selectionFor() returns { [sectionId]: arm } for applySectionVariants in @shopkit/builder to apply.