Tachyon-Experiments

$ yarn add tachyon-experiments

This package provides an abstraction for loading and consuming experiment configurations in a React based application.

See experiment patterns for patterns Tachyon applications have built using this package.

Note: This package is tailored for use in server side rendered applications and limits the consumer to running 3 concurrent experiments. This trade off was made to support caching.

Example

import type { FC } from 'react';
import {
  ExperimentName,
  Experiment,
  Treatment,
  ExperimentGroup,
} from 'tachyon-experiments';

const ControlComponent = () => <div>Control</div>;
const TreatmentComponent = () => <div>Treatment</div>;

const ExperimentalFeature: FC = () => (
  <Experiment experimentUUID={'<SOME-EXPERIMENT-UUID>'}>
    <Control>
      <TreatmentComponent />
    </Control>
    <Treatment>
      <ControlComponent />
    </Treatment>
  </Experiment>
);

See the experiments example implementation to see more examples.

Registering the Experiment

1. Add the new experiment UUID to one of the three available slots used to configure getExperimentGroupsForUser.
2. Use Experiment and Group components to define the behavior of the experiment. See the example experiment implementation for a reference of setting up an A/B test.

Roadmap and Known Issues

• This package assumes that all experiments will be based on device_id (https://jira.twitch.com/browse/MWC-1095)
• This package assumes that it splits on mobile-web-esque buckets instead of device IDs (https://jira.twitch.com/browse/MWC-1099)

100% Treatment Experiments

The Tachyon platform is opinionated about tearing down experiment logic, which can tend to accumulate long after experiments have ended. To this end, when we recognize that an experiment is at 100% treatment in local dev, we treat it as 100% control in order to help push the removal of the experiment branching logic (along with the previous control code paths). There is an accompanying terminal console message as a helpful reminder of this behavior.

Testing

There aren't any special considerations when shallow testing a React component that renders Experiment. When mount testing, you will need mocks because of the Experiment internals. This package exports helper mocks for that purpose, enabling you to choose which experiment branch is selected.

import { MyComponent } from '.';
import {
  MockExperimentControl,
  MockExperimentTreatment
} from 'tachyon-experiments';

jest.mock('tachyon-experiments', () => ({
  ...jest.requireActual('tachyon-experiments'),
  Experiment: jest.fn(),
});

const mockExperiment = Experiment as jest.Mock;

describe('Experiments', () => {
  it('test the control branch', () => {
    mockExperiment.mockImplementationOnce(MockExperimentControl);
    // assert on things in control path
  });

  it('test the treatment branch', () => {
    mockExperiment.mockImplementationOnce(MockExperimentTreatment);
    // assert on things in treatment path
  });
});