Imeasure SDK

This is a collection of TypeScript types and utility functions for working with Better Medicine's iMeasure API.

Table of Contents

• imeasure-sdk
  • Table of Contents
  • Installation
  • Usage
    • API request & response types
    • Cropping
      • Cropping region calulation
      • Executing the crop
        • Working with data from GPU texture buffers
        • Image data bit depth
      • Affines
    • World metadata
    • Composing the request payload

You can find the primary documentation site at https://docs.imeasure.bettermedicine.ai. Module references mentioned in this document can be located under the @bettermedicine/namespaces sidebar route.

Installation

npm install @bettermedicine/imeasure

Usage

This package should prove useful if you're trying to integrate with Better Medicine's iMeasure API, to varying extent depending on what type of application you're building.

| Type of application | Request & response types | Crop region utils | Cropping utils | World metadata parsing utils | Request body formatting | | ---------------------------------- | ------------------------ | ----------------- | -------------- | ---------------------------- | ----------------------- | | OHIF-based application | ✅ | ✅ | ✅ | ✅ | ✅ | | Other vtk.js-based application | ✅ | ✅ | ✅ | | ✅ | | Other JavaScript-based application | ✅ | ✅ | | | ✅ |

For integration targets not written in JavaScript/TypeScript, you can still review this package as a reference implementation.

API request & response types

import { Types } from '@bettermedicine/imeasure';

type RequestMetadata = Types.IO.IStaticIMeasurePayload;
type Response = Types.IO.TIMeasure3DResponse

Please refer to the Types.IO module in the documentation for more details on the request and response types.

Cropping

To generate a segmentation and derive measurements from it, the iMeasure API requires a subset of the series' images' pixel data. Sending us the entire series' pixel data is not feasible so instead, we expect a cropped 3D cuboid of the series' voxel data to be supplied with each request, along with some metadata both about the cuboid and the series' coordinate space.

You can find more information regarding data format requirements and a walkthrough of how to calculate the cropping regions in the main unit of our API documentation.

Cropping region calulation

import { math, cropping } from '@bettermedicine/imeasure';

// This example assumes you're working in a callback for a Cornerstone measurement event.
// Here, we expect the `points` property to conform to [[x1,y1,z1], [x2,y2,z2]] in the world (not voxel) coordinate system.
const { measurement: { points }} = evt;

// In order to make the calculation, we need some information about the world coordinate space.
// In this example, we abstract away locating and loading the vtk.js imageData object but it should be noted that
// there's more options for deriving this.
// see the Affines section below for more details.
const voxelToWorldAffine = math.affines.getAffineMatrixFromVTKImageData(relevantVTKVolume)

// this gets us all the information we need to execute the crop.
const cropRegion = math.calculateStaticImeasureBoundingBox(
    points,
    voxelToWorldAffine
);

Executing the crop

// To continue the above example,
const imageCrop =  cropping.cropFromVTKVolumeByCropBoundaries(
    relevantVTKVolume,
    cropRegion.cropBoundariesVoxel
)

const {
    voxelValues, // a flattened array of voxel values from the cropped region
    volume, // the cropped vtk.js imageData object
    cropMeta // crop metadata you'll need to supply in the request
} = imageCrop;

:bulb: Note: In case the measurement we're generating the crop region around is near the edge of the volume or in case the measurement is long enough, the crop region may extend beyond the volume's boundaries. In such cases, the consumer is expected to provide a non-cuboid input volume but still preserve metadata about the originally intended crop region. The utilities provided in the cropping module will handle this case automatically by supplying cropBoundariesUnclippedVoxel and cropCuboidCenterVoxel corresponding to the original crop region, while the cropBoundariesVoxel, cropBoundariesWorld and shape attributes in TCropMetadata will be adjusted to the actual effective cropped region.

Working with data from GPU texture buffers

In modern versions of Cornerstone 3D, unless forced by configuration, the image volume's voxel data only exists in the GPU texture buffer. If the above function outputs empty voxelValues, we need to take a few additional steps to ensure the data is reachable by our code:

import { cache as csCache } from '@cornerstonejs/core';
import vtkImageData from '@kitware/vtk.js/Common/DataModel/ImageData';
import vtkDataArray from '@kitware/vtk.js/Common/Core/DataArray';

// note: at least in the current version of OHIF, the volume ID conforms to
// `${volumeLoaderScheme}:${displaySetInstanceUID}`.
// as such, using standard volume viewports, you should be able to
// refer to `cornerstoneStreamingImageVolume:${displaySetInstanceUID}`
const volumeID = 'some-cs-volume-id';
const scalarsInstanceName = 'Scalars';


// 1. we get a reference to the Cornerstone volume from Cornerstone's own cache
const baseVolume = csCache.getVolume(volumeID);

if (!baseVolume) {
    throw new Error(`Volume with ID ${volumeID} not found in Cornerstone cache.`);
}

if (!baseVolume.voxelManager) {
    throw new Error(
        `Volume does not have voxel manager, ID ${volumeID}`
    );
}

// 2. we create a new vtkDataArray instance we'll be using as a container for the scalar data,
//    this time in RAM and accessible by the CPU
const da = vtkDataArray.newInstance({
    name: scalarsInstanceName,
    numberOfComponents: 1,
    // 3. This forces the data to be copied from the GPU texture buffer to the CPU memory.
    values: baseVolume.voxelManager.getCompleteScalarDataArray(),
});

const copiedVolume = vtkImageData.newInstance();
copiedVolume.setDimensions(baseVolume.getDimensions());
copiedVolume.setSpacing(baseVolume.getSpacing());
copiedVolume.setOrigin(baseVolume.getOrigin());
copiedVolume.setDirection(baseVolume.getDirection());
copiedVolume.getPointData().setScalars(da);

Given that image volumes are mostly static and that this process may be computationally expensive,we recommend caching the copied volume so you don't have to re-instantiate it every time you execute a new iMeasure request. As an example;

import { cache as csCache } from '@cornerstonejs/core';
import vtkImageData from '@kitware/vtk.js/Common/DataModel/ImageData';
import vtkDataArray from '@kitware/vtk.js/Common/Core/DataArray';

type VolumeID = string;
const vtkImageDataCache = new Map<VolumeID, vtkImageData>();

function getVTKImageData(cornerstoneVolumeID: VolumeID): vtkImageData {
    if (vtkImageDataCache.has(volumeID)) {
        return vtkImageDataCache.get(volumeID);
    }

    // If the volume is not cached, we create it as shown above
    const baseVolume = csCache.getVolume(volumeID);
    if (!baseVolume) {
        throw new Error(`Volume with ID ${volumeID} not found in Cornerstone cache.`);
    }

    if (!baseVolume.voxelManager) {
    throw new Error(
        `Volume does not have voxel manager, ID ${volumeID}`
    );
}

    const da = vtkDataArray.newInstance({
        name: 'Scalars',
        numberOfComponents: 1,
        values: baseVolume.voxelManager.getCompleteScalarDataArray(),
    });

    const copiedVolume = vtkImageData.newInstance();
    copiedVolume.setDimensions(baseVolume.getDimensions());
    copiedVolume.setSpacing(baseVolume.getSpacing());
    copiedVolume.setOrigin(baseVolume.getOrigin());
    copiedVolume.setDirection(baseVolume.getDirection());
    copiedVolume.getPointData().setScalars(da);

    vtkImageDataCache.set(volumeID, copiedVolume);
    return copiedVolume;
}

Image data bit depth

DICOM CT HU values typically fit within the range allowance of a 16-bit signed integer. As such, this is what the iMeasure API expects as the scalar data type. Unless you're forcing 16-bit textures via OHIF config, the scalar data may be of the Float32Array type. You can slash the converted vtkImageData object's memory footprint by half by using our formatting.float32ArrayToInt16Array utility inlined into the above vtkDataArray instantiation:

import { formatting } from '@bettermedicine/imeasure';

const da = vtkDataArray.newInstance({
    ...
    values: formatting.float32ArrayToInt16Array(
        baseVolume.voxelManager.getCompleteScalarDataArray()
    ),
});

Affines

In order to accurately calculate the crop region, we need some more information about the the shape, orientation, origin and spacing of the series' world coordinate system. The standard way to express this is through an affine transformation matrix - a 4x4 matrix that describes how to transform points from the voxel coordinate system to the world coordinate system.

The math.affines module provides utilities for deriving this from a number of sources:

import { math } from '@bettermedicine/imeasure';
// 1. from a vtk.js imageData object
const vtkImageData = ...; // your vtk.js imageData object
const voxelToWorldMatrix = math.affines.getAffineMatrixFromVTKImageData(vtkImageData);


// 2. from a Cornerstone volume
const cornerstoneVolume = csCache.getVolume(volumeID);
const voxelToWorldMatrix = math.affines.getAffineMatrixFromCornerstoneVolume(cornerstoneVolume);

// 3. ...or from primitives you collect yourself

type TAffinePrimitivesInput = {
  spacing: vec3;
  origin: vec3;
  direction: mat3;
  extent: [number, number, number, number, number, number];
};

const voxelToWorldMatrix = math.affines.getAffineMatrixFromPrimitives(
  {
    spacing: [1, 1, 1],
    origin: [0, 0, 0],
    direction: [[1, 0, 0], [0, 1, 0], [0, 0, 1]],
    extent: [0, 100, 0, 100, 0, 100],
  } as TAffinePrimitivesInput
);

The resulting matrix can then be used to convert voxel coordinates to world coordinates:

import { math } from '@bettermedicine/imeasure';
const voxelToWorldMatrix = ...; // your affine matrix
const voxelPoint = [10, 20, 30]; // a point in voxel coordinates
const worldPoint = math.affines.affineTranslateVec3(voxelPoint, voxelToWorldMatrix)

Should the inverse operation be required, you can invert the matrix:

const worldToVoxelMatrix = math.affines.invertAffine(voxelToWorldMatrix);
const worldPoint = [10, 20, 30]; // a point in world coordinates
const voxelPoint = math.affines.affineTranslateVec3(worldPoint, worldToVoxelMatrix);

World metadata

Please refer to the IO module's documentation for the IStaticIMeasurePayload type.

Composing the request payload

Once you have the cropped image data and the necessary world metadata, you can compose the request payload as follows:

import { formatting  } from '@bettermedicine/imeasure';
const worldMetadata = ...
const cropMetadata = ...
const clicks = ...
const id = uuidv4();
// note: this should be an Int16Array - see the bit depth section above.
const imageVoxelValues = ... // the flattened array of voxel values from the cropped region

const requestPayload = formatting.formatStaticPayload(
    {
        id,
        clicks,
        ...worldMetadata,
        ...cropMetadata,
    },
    imageVoxelValues
);

// the above returns a FormData object you can POST to the iMeasure API (ideally, through an authenticating proxy):
const resp = await fetch('https://api.bettermedicine.com/imeasure', {
    method: 'POST',
    body: requestPayload,
    // note: setting content-type headers manually here is not recommended as `fetch`
    // should handle it correctly automatically.
});

const responseData = await resp.json();

Please see the Types.IO module for in-depth information on both the request and response types.