A/B Testing

TO-DO before releasable

• Implement automatic impression tracking
• Finalize tooling
• Find a better name

Getting started

For a minimal setup six elements are needed: ABRoot, withABTest and VariantA/VariantB along with an ABManger with an ABTestSuite and an ABUser

A.jsx

import React, { Component } from 'react';
import { withABTest } from from 'abstesting';

class A extends Component {
  handleClick() {
    this.props.trackConversion();
  }

  render() {
    return (
      <div onClick={this.handleClick.bind(this)}>Variant A</div>
    );
  }
}

export default withABTest(A);

B.jsx

import React, { Component } from 'react';
import { withABTest } from from 'abstesting';

class B extends Component {
  handleClick() {
    this.props.trackConversion();
  }

  render() {
    return (
      <div onClick={this.handleClick.bind(this)}>Variant B</div>
    );
  }
}

export default withABTest(B);

App.jsx

import React from 'react';
import { VariantA, VariantB } from 'abtesting';
import A from './A';
import B from './B';

export default () => (
  <div>
    <VariantA name="test1" autoTrackImpression>
      <A />
    </VariantA>
    <VariantB name="test2" autoTrackImpression>
      <B />
    </VariantB>
  </div>
);

test1.js

import { FunctionTest } from 'abtesting';

export default new FunctionTest(
  (userInfo) => userInfo.age > 15,
  (userInfo) => userInfo.id % 2,
);

index.jsx

import React from 'react';
import ReactDOM from 'react-dom';
import { ABRoot, ABUser, ABTestSuite, ABManager } from 'abtesting';
import test1 from './tests';
import App from './App.js';

const manager = new ABManager({
  user: new ABUser(),
  testSuite = new TestSuite(),
});
manager.setUserInfo({
  id: 1234,
  age: 23,
});
manager.setTests({
  test1,
});

ReactDOM.render(
  <ABRoot manager={manager}>
    <App />
  </ABRoot>,
  document.body,
)

Using different components

Another usefull feature is to be able to serve different components depending on which test bucket the user is in. This can be done using the createABTest higher order function

A.jsx

export default ({value}) => <div>VariantA {value}</div>

B.jsx

export default ({value}) => <div>VariantB {value}</div>

Component.jsx

import { createABSplit } from 'abtesting';
import A from './A';
import B from './B';

export default createABSplit({
  name: 'test1',
  VariantA: A,
  VariantB: B,
})

Now when rendering <Component value="hello" />, the user which are presented the original version will see "VariantA hello" and user presented with the alternative version will see "VariantB hello".

User information

In order to figure out if users is in a given group, a set of user trades need to be provided. The test split is stateless, so therefore it can not store any information about previous a/b buckets a user has fallen into, and therefor this needs to be something which can be figured out based on values about the current user.

These are set by calling setUserInfo on the manager object, which was passed to ABRoot. When these values change, all A/B splits, currently presented to the user will be re-evaluated, and if a user changes bucket, these changes are reflected.

import { ABManager, ABUser, ABRoot } from 'abtesting';

const manager = new ABManager({
  ...
  user: new ABUser();
});
manager.setInfo({
  age: 23
});

const root = (
  <ABRoot manager={manager}>
  </ABRoot>
)

Tests

Test Suites

Creating a custom test

import { ABTest } from 'abstesting';
class CustomTest extends ABTest {
  isIncludedInTest(userInfo) {
    return userInfo.isLoggedIn;
  }

  isInTestGroup(userInfo) {
    return userInfo.id % 2;
  }
}

manager.setTest({
  someTest: new CustomTest(),
});

Services

The component comes with a few different service types preloaded and provides the tools for creating custom service

Creating a service

import { ABService, ABManager } from 'abstesting';

class CustomService extends ABService {
  constructor(endpoint) {
    super();
    this.endpoint = endpoint;
  }

  async updateTests() {
    const response = await fetch(endpoint).then(res => res.json());
    return response;
  }

  async sendData(data) {
    const data = new FormData();
    data.append( "json", JSON.stringify(data) );
    return fetch(endpoint, {
      method: "POST",
      body: data
    });
  }

  async trackImpression(name, type) {
    await this.sendData({
      type: 'impression',
      name,
      type,
    });
  }

  async trackConversion(name, type, params) {
    await this.sendData({
      type: 'impression',
      name,
      type,
    });
  }
}

const manager = new ABManager {
  ...
  service: new CustomService(),
}

const root = (
  <ABRoot manager={manager}>
  </ABRoot>
)