@knaw-huc/text-annotation-segmenter

Utility functions to render annotations with character position offsets in a text.

Annotations on a text have a non-hierarchical nature, i.e., they can overlap:

         text:  abc
annotation ab:  __
annotation bc:   __

However, HTML is hierarchical. How to display annotations that do not live inside or next to each other, but that cut across each other?

The segment function creates an array of segments: a flat, non-overlapping list where each segment links to both the text and all the annotations that apply. Each segment translates into a single dom/jsx element. Elements linked to multiple overlapping annotations can now be decorated with their own styling, classes and callbacks:

<span class="ab">a</span>
<span class="ab bc">b</span>
<span class="bc">c</span>

API

Functions

• segment<T>(text, annotations, getOffsets): TextSegment<T>[] Split a text into TextSegments with character positions and a list of applying annotations.
• groupSegments<T>(segments, isGroup, getId): SegmentGroup<T>[] Recursively group segments into higher-level units (e.g., words, paragraphs, divs) by collecting all segments that share a matching annotation.
• collectGroupSegments<T>(group): TextSegment<T>[] Recursively collect all TextSegments from a Group.
• filterSegmentAnnotations<T>(segments, predicate): TextSegment<T>[] Filter annotations in segments using structural sharing.
• findSegmentRange<T>(segments): Map<T, SegmentRange> For each annotation, collect the first and last segment index it appears in, keyed by object reference.

Types

• SegmentGroup<T> A Group<T> (with annotation and recursive children) or Ungrouped<T> (with plain segments).
• SegmentRange Start and end segment index of an annotation (excluding last segment).
• TextSegment<T> Start and end positions of text segment (excluding last character), plus the annotations that apply.

Examples

Create text segments

A text 'abc' with two overlapping annotations at 'ab' and 'bc' will be split up in three segments:

         text:  abc
annotation ab:  __
annotation bc:   __

import {segment} from "@knaw-huc/text-annotation-segmenter";

const text = 'abc';
const ab = {id: 'ab', start: 0, end: 2};
const bc = {id: 'bc', start: 1, end: 3};

const getOffsets = annotation => annotation;

const segments = segment(text, [ab, bc], getOffsets);

expect(segments).toEqual([
  {index: 0, start: 0, end: 1, value: 'a', annotations: [ab]},
  {index: 1, start: 1, end: 2, value: 'b', annotations: [ab, bc]},
  {index: 2, start: 2, end: 3, value: 'c', annotations: [bc]},
]);

• More examples: segment.spec.ts.
• Benchmarks: segment.bench.ts.

Group segments

We can use segments to build up a hierachy of elements:

Given the text 'ab' with a section spanning the whole text and a paragraph spanning the first half:

     text:  aabb
  section:  ____
paragraph:  __

import {segment, groupSegments} from "@knaw-huc/text-annotation-segmenter";

const text = 'aabb';
const section = {id: 'section', type: 'section', start: 0, end: 4};
const paragraph = {id: 'paragraph', type: 'paragraph', start: 0, end: 2};

const getOffsets = annotation => annotation;
const segments = segment(text, [section, paragraph], getOffsets);

const isGroup = a => a.type === 'section' || a.type === 'paragraph';
const getId = a => a.id;

const groups = groupSegments(
  segments,
  isGroup,
  getId,
);
const aaSegment = segments[0];
const bbSegment = segments[1];

expect(groups).toEqual([
  {annotation: section, isGroup: true, children: [
    {annotation: paragraph, isGroup: true, children: [
        {segments: [aaSegment], isGroup: false}
    ]},
    {segments: [bbSegment], isGroup: false},
  ]}
]);

• More examples: groupSegments.spec.ts.
• Benchmarks: groupSegments.bench.ts.

Markers

A special case is the marker: an annotation of zero width marking a position in the text. Markers result in zero-width segments, including all annotations that cover, start or end at that position:

text with marker:  a‸b
paragraph 1:       _
paragraph 2:         _

const text = 'ab';
const marker: Annotation = {start: 1, end: 1, id: 'm'};
const p1: Annotation = {start: 0, end: 1, id: 'p1'};
const p2: Annotation = {start: 1, end: 2, id: 'p2'};

const segments = segment(text, [marker, p1, p2], getOffsets);
const markerSegment = segments[1];
expect(markerSegment).toEqual(
  {index: 1, start: 1, end: 1, value: '', annotations: [marker, p1, p2]},
);

Note how the marker segment contains both paragraphs as the marker borders both. We can filter the annotations of a marker segment, for example to control which block element contains the marker, using filterSegmentAnnotations. For example, if we would want to 'postfix' the marker, like a note at the end of a paragraph:

const result = filterSegmentAnnotations(segments, (annotation, segment) => {
  if (segment.start !== segment.end) {
    // Skip non-marker segments:
    return true;
  }
  if (annotation.type === 'marker') {
    // Keep marker in marker segments:
    return true;
  }
  // Keep annotations starting before marker:
  return annotation.start < segment.start;
});
const markerSegment = result[1];
expect(markerSegment).toEqual(
  {index: 1, start: 1, end: 1, value: '', annotations: [marker, p1]},
);

More examples:

• segment.spec.ts.
• filterSegmentAnnotations.spec.ts.