@apptegy/experiments

Framework-agnostic A/B test bucketing for Apptegy frontends.

• Deterministic assignment via vendored MurmurHash3 — the same user is consistently in or out of a test, and consistently gets the same option.
• Decides inclusion (who is in the test) and variant selection (which option) with two independent hashes.
• Reports in-test exposures to Amplitude through an injectable adapter — the package never bundles the Amplitude SDK.
• Zero runtime dependencies, ESM-only, works in the browser and in Node/SSR.

Install

pnpm add @apptegy/experiments

How it works

Everything is derived from a stable userId plus the experiment key. No network calls, no shared state — given the same inputs you always get the same result, on any device, in any process.

1. Deterministic buckets

A string is hashed with MurmurHash3 (x86, 32-bit) and normalized into a bucket in the range [0, 1):

bucketOf(str) = murmur3_32(str) / 2^32

2. Two independent decisions

Each experiment makes two decisions, each keyed with a distinct salt suffix so they never interfere:

inclusion bucket = bucketOf(`${salt}:${userId}:inclusion`)
variant bucket   = bucketOf(`${salt}:${userId}:variant`)

• Inclusion — is the user in the test? included = inclusionBucket < allocation (e.g. allocation: 0.01 ≈ 1% of users). enabled: false or allocation: 0 excludes everyone; allocation: 1 includes everyone.
• Variant selection — which arm? The variant bucket is mapped onto the arms' cumulative normalized weights (equal weight by default), so a 2-arm test splits 50/50 among included users.

Because the two use separate hashes, changing allocation never moves a user's variant, and choosing a variant never affects who's included.

3. Excluded users get the control

The first arm you list is the control. Excluded users (and any error-safe fallback) always resolve to it, so callers can render a concrete option without null checks. assign() returns:

{ experiment: string; included: boolean; variant: string; bucket: number; forced: boolean }

Two properties worth knowing

• Monotonic ramp-up. Increasing allocation (1% → 10% → 100%) only adds users; everyone already in the test keeps their variant. Safe to ramp live.
• salt is the reshuffle knob. salt defaults to key. Changing key or salt re-randomizes membership and variants; keep them stable to keep results stable.

Quick start

The one-liner: give a user id, a test key, and a list of options, and get back the option that user should see.

import { getOption, amplitudeReporter } from '@apptegy/experiments';

const option = getOption(userId, 'checkout-cta', ['control', 'variant-b'], {
  reporter: amplitudeReporter(amplitude), // your existing Amplitude instance
});
// Users not in the test (default ~99%) get 'control' and nothing is reported.
// Users in the test (default ~1%) get a deterministic option, reported as a
// `$exposure` event.

For an app that runs several experiments for one user, create a client once:

import { createExperimentClient, amplitudeReporter, browserOverrides } from '@apptegy/experiments';

const experiments = createExperimentClient({
  context: { userId, clientId },
  reporter: amplitudeReporter(amplitude),
  overrides: browserOverrides(), // enables the QA console API (see below)
});

const { variant, included } = experiments.assign({
  key: 'checkout-cta',
  variants: ['control', 'variant-b'],
  allocation: 0.01, // fraction included; defaults to 0.01
});

Example implementation (Vue 3 / Nuxt, end-to-end)

A realistic wiring: create the client once per app, expose a reactive composable, render a variant in a component, and (optionally) drive allocation from a remote flag so you can ramp the rollout without a deploy.

1. Provide a client once (Nuxt plugin)

// plugins/experiments.client.js
import { defineNuxtPlugin } from '#app';
import { amplitudeReporter, browserOverrides, createExperimentClient } from '@apptegy/experiments/core';

export default defineNuxtPlugin((nuxtApp) => {
  const { $amplitude, $store } = nuxtApp;
  const userId = $store.getters['user/getUserId'];
  const clientId = $store.getters['user/getClientId'];

  const experiments = createExperimentClient({
    context: { userId, clientId },
    reporter: $amplitude ? amplitudeReporter($amplitude) : undefined,
    overrides: browserOverrides(), // QA console + URL/localStorage forcing
  });

  nuxtApp.provide('experiments', experiments); // -> useNuxtApp().$experiments
});

2. Wrap it in a reactive composable

// composables/useExperiment.ts
import { computed, toValue, type MaybeRefOrGetter } from 'vue';
import { useNuxtApp } from '#imports';

export function useExperiment(
  key: string,
  variants: string[] = ['control', 'variant-b'],
  allocation?: MaybeRefOrGetter<number | undefined>, // ref/getter -> reactive
) {
  const { $experiments } = useNuxtApp();

  const assignment = computed(() => {
    // Control fallback keeps SSR / pre-plugin renders safe.
    if (!$experiments) return { variant: variants[0], included: false };
    return $experiments.assign({ key, variants, allocation: toValue(allocation) });
  });

  return {
    variant: computed(() => assignment.value.variant),
    included: computed(() => assignment.value.included),
  };
}

3. Render the variant in a component

<script setup>
import { computed } from 'vue';
import { useExperiment } from '@/composables/useExperiment';

const { variant } = useExperiment('checkout-cta', ['control', 'variant-b']);
const isVariantB = computed(() => variant.value === 'variant-b');
</script>

<template>
  <PrimaryButton v-if="isVariantB" class="cta--wide">Get started</PrimaryButton>
  <IconButton v-else aria-label="Get started" />
</template>

Prefer to keep a shared component library generic? Compute the variant in the consuming app and pass it in as a plain prop (e.g. :cta-variant="variant"), so the library gains no dependency on @apptegy/experiments.

4. Drive allocation from a remote flag (optional)

allocation is read on every evaluation, so a reactive source (LaunchDarkly, remote config, env) lets you ramp the rollout live — no library republish, no consumer deploy once the plumbing exists:

const { $ld } = useNuxtApp();
// Flag serves an integer percentage (1 => 1% => 0.01); default 0% until it exists.
const { variant } = useExperiment('checkout-cta', ['control', 'variant-b'],
  () => ($ld?.flags?.checkoutCtaAllocation ?? 0) / 100,
);

Forcing a variant for QA

With a 1% allocation you cannot reliably land yourself in the treatment, so the package ships a browser console API (enabled by passing browserOverrides() as the overrides option). Open the JS console and run:

__experiments.force('checkout-cta', 'variant-b'); // pin a variant (persists)
__experiments.list();                              // show active overrides
__experiments.clear('checkout-cta');               // remove one
__experiments.clearAll();                           // remove all

You can also force a variant via a shareable URL: ?exp_checkout-cta=variant-b (URL takes precedence over localStorage). Forced assignments bypass hashing/allocation, return included: true, and report exposures flagged forced: true so QA traffic stays filterable.

Reporting to Amplitude

amplitudeReporter(instance) adapts any Amplitude-like object to the reporter interface. The instance only needs a track(name, props) method (the lms $amplitude wrapper and the raw @amplitude/analytics-browser namespace both qualify). On the first assign() for an included user it sends:

$exposure  { flag_key: '<key>', variant: '<variant>' }

Exposure is de-duplicated per (experiment, variant) for the client's lifetime, so calling assign() in a render loop won't spam Amplitude. If identify / Identify are present, a [Experiment] <key> user property is also set for chart segmentation.

Public API

import {
  getOption,
  createExperimentClient,
  amplitudeReporter,
  browserOverrides,
  noopReporter,
  bucketOf,
  murmur3_32,
  InvalidExperimentError,
} from '@apptegy/experiments';            // also '@apptegy/experiments/core'

import {
  createMockReporter,
  createMockExperimentClient,
  forceVariant,
} from '@apptegy/experiments/testing';

Testing your experiment code

The /testing entry lets consumers assert both variants without touching the hashing. createMockExperimentClient pins arms via forced; createMockReporter captures exposures.

import { createMockExperimentClient, createMockReporter } from '@apptegy/experiments/testing';

it('renders variant B and reports exposure', () => {
  const reporter = createMockReporter();
  const experiments = createMockExperimentClient({
    context: { userId: 'user-1' },
    reporter,
    forced: { 'checkout-cta': 'variant-b' }, // key -> variant
  });

  const { variant, included } = experiments.assign({ key: 'checkout-cta', variants: ['control', 'variant-b'] });

  expect(variant).toBe('variant-b');
  expect(included).toBe(true);
  expect(reporter.exposures).toContainEqual(
    expect.objectContaining({ experiment: 'checkout-cta', variant: 'variant-b' }),
  );
});

forceVariant(map) is also exported for building an OverrideStore to pass as overrides to a real createExperimentClient in tests.

Behavior

| Situation | variant | included | Exposure reported? | | --- | --- | --- | --- | | Not in test (default ~99%) | first option (control) | false | No | | In test (default ~1%) | deterministic option | true | Yes (once per experiment+variant) | | enabled: false or allocation: 0 | first option (control) | false | No | | Forced override (QA) | forced option | true | Yes, flagged forced: true | | Invalid config (empty variants, bad weights/allocation) | throws InvalidExperimentError | — | — |

Development

pnpm --filter @apptegy/experiments test       # run unit tests
pnpm --filter @apptegy/experiments build       # emit dist/ (js + d.ts)
pnpm --filter @apptegy/experiments lint
pnpm --filter @apptegy/experiments typecheck