Data Model Indexing Support

Basic indexing support for model sources that support it.

Install: @travetto/model-indexed

npm install @travetto/model-indexed

# or

yarn add @travetto/model-indexed

This module provides computed index support for data model sources that support it. It enables efficient lookups and list operations using composite keys extracted from model fields, without requiring a full query engine.

Overview

The module allows you to define indexes on your models and use them for fast single-item lookups, uniqueness enforcement, and efficient paginated list operations. Indexes are computed from model field values and act as alternative keys for data access.

Index Types

Three types of indexes are supported:

• Keyed Indexes — Fast single-item lookups using composite keys
• Unique Indexes — Enforce uniqueness constraints on key fields
• Sorted Indexes — Enable range queries and paginated listing with sorting

Defining Indexes

Indexes are defined using factory functions provided by the module. Each index is registered with the model at decoration time.

Keyed Indexes

A keyedIndex provides fast lookups by computed key values. It's useful when you want to query records by specific field combinations.

Code: Creating a Keyed Index

import { Model } from '@travetto/model';
import { keyedIndex } from '@travetto/model-indexed';

@Model()
export class User {
  id: string;
  name: string;
  email: string;
}

export const userByName = keyedIndex(User, {
  name: 'userByName',
  key: { name: true }
});

The index definition specifies:

• name — The identifier for this index
• key — An object where each key path should be included in the index (set to true)

Unique Indexes

A uniqueIndex enforces uniqueness constraints on key fields. This is useful for emails, usernames, or any field that should be globally unique.

Code: Creating a Unique Index

import { Model } from '@travetto/model';
import { uniqueIndex } from '@travetto/model-indexed';

@Model()
export class User {
  id: string;
  name: string;
  email: string;
}

export const emailUnique = uniqueIndex(User, {
  name: 'uniqueEmail',
  key: { email: true }
});

Unique indexes work exactly like keyed indexes, but enforce a uniqueness constraint. A model service will reject writes that violate the uniqueness guarantee.

Sorted Indexes

A sortedIndex enables range queries and paginated listing. It requires both a key for filtering and a sort field for ordering.

Code: Creating a Sorted Index

import { Model } from '@travetto/model';
import { sortedIndex } from '@travetto/model-indexed';

@Model()
export class User {
  id: string;
  name: string;
  age: number;
  createdAt: Date;
}

export const usersByNameAge = sortedIndex(User, {
  name: 'usersByNameAge',
  key: { name: true },
  sort: { age: 1 } // 1 for ascending, -1 for descending
});

export const recentUsers = sortedIndex(User, {
  name: 'recentUsers',
  key: {}, // No key filtering
  sort: { createdAt: -1 } // Most recent first
});

The sort field must be numeric or a Date type. The value 1 means ascending order, -1 means descending.

Composite Keys

Indexes can use multiple fields or nested fields in their keys. This allows querying by combinations of values.

Code: Composite Key Index

import { Model } from '@travetto/model';
import { keyedIndex } from '@travetto/model-indexed';

@Model()
export class Order {
  id: string;
  customerId: string;
  status: string;
  productId: string;
}

// Find orders by customer and status
export const orders = keyedIndex(Order, {
  name: 'ordersByCustomerStatus',
  key: { customerId: true, status: true }
});

// Find orders by customer, status, and product
export const specificOrders = keyedIndex(Order, {
  name: 'ordersByCustomerStatusProduct',
  key: { customerId: true, status: true, productId: true }
});

Using Indexes

Model services that implement ModelIndexedSupport allow you to query using the indexes you've defined.

Service Interface

Code: ModelIndexedSupport Interface

export interface ModelIndexedSupport extends ModelBasicSupport {
  /**
   * Get entity by index as defined by fields of idx and the body fields
   * @param cls The type to search by
   * @param idx The index to search against
   * @param body The payload of fields needed to search
   */
  getByIndex<T extends ModelType, K extends KeyedIndexSelection<T>, S extends SortedIndexSelection<T>>(
    cls: Class<T>,
    idx: SingleItemIndex<T, K, S>,
    body: FullKeyedIndexBody<T, K, S>
  ): Promise<T>;

  /**
   * Delete entity by index as defined by fields of idx and the body fields
   * @param cls The type to search by
   * @param idx The index to search against
   * @param body The payload of fields needed to search
   */
  deleteByIndex<T extends ModelType, K extends KeyedIndexSelection<T>, S extends SortedIndexSelection<T>>(
    cls: Class<T>,
    idx: SingleItemIndex<T, K, S>,
    body: FullKeyedIndexBody<T, K, S>
  ): Promise<void>;

  /**
   * Upsert by index, allowing the index to act as a primary key
   * @param cls The type to create for
   * @param idx The index to use
   * @param body The document to potentially store
   */
  upsertByIndex<T extends ModelType, K extends KeyedIndexSelection<T>, S extends SortedIndexSelection<T>>(
    cls: Class<T>,
    idx: SingleItemIndex<T, K, S>,
    body: OptionalId<T>
  ): Promise<T>;

  /**
   * Update by index
   * @param cls The type to update for
   * @param idx The index to update by
   * @param body The document to update
   */
  updateByIndex<T extends ModelType, K extends KeyedIndexSelection<T>, S extends SortedIndexSelection<T>>(
    cls: Class<T>,
    idx: SingleItemIndex<T, K, S>,
    body: T
  ): Promise<T>;

  /**
   * Update partial by index
   * @param cls The type to update for
   * @param idx The index to update by
   * @param body The partial document to update
   */
  updatePartialByIndex<T extends ModelType, K extends KeyedIndexSelection<T>, S extends SortedIndexSelection<T>>(
    cls: Class<T>,
    idx: SingleItemIndex<T, K, S>,
    body: FullKeyedIndexWithPartialBody<T, K, S>
  ): Promise<T>;

  /**
   * Page through entities by ranged index as defined by fields of idx
   *
   * Note: Limit is generally honored, but can vary depending on the underlying storage implementation.
   *
   * @param cls The type to search by
   * @param idx The index to search against
   * @param body The payload of fields needed to search
   * @param options The configuration for pagination
   */
  pageByIndex<T extends ModelType, S extends SortedIndexSelection<T>, K extends KeyedIndexSelection<T>>(
    cls: Class<T>,
    idx: SortedIndex<T, K, S>,
    body: KeyedIndexBody<T, K>,
    options?: ModelPageOptions
  ): Promise<ModelPageResult<T>>;

  /**
   * List all entities by ranged index as defined by fields of idx
   *
   * Note: Limit is generally honored, but can vary depending on the underlying storage implementation.
   * Batch size hint can be used to optimize batch size, but is not guaranteed.
   *
   * @param cls The type to search by
   * @param idx The index to search against
   * @param body The payload of fields needed to search
   */
  listByIndex<T extends ModelType, S extends SortedIndexSelection<T>, K extends KeyedIndexSelection<T>>(
    cls: Class<T>,
    idx: SortedIndex<T, K, S>,
    body: KeyedIndexBody<T, K>,
    options?: ModelListOptions
  ): AsyncIterable<T[]>;

  /**
   * Suggest entities by ranged index as defined by fields of idx and a prefix
   *
   * Note: Limit is generally honored, but can vary depending on the underlying storage implementation.
   *
   * @param cls The type to search by
   * @param idx The index to search against
   * @param body The payload of fields needed to search
   * @param prefix The prefix to use for suggesting entities
   * @param options The configuration for pagination
   */
  suggestByIndex<
    T extends ModelType,
    S extends SortedIndexSelection<T>,
    K extends KeyedIndexSelection<T>,
    B extends SortedIndexSelectionType<T, S> & string
  >(
    cls: Class<T>,
    idx: SortedIndex<T, K, S>,
    body: KeyedIndexBody<T, K>,
    prefix: B,
    options?: ModelIndexedSearchOptions
  ): Promise<T[]>;
}

The service provides these operations:

• getByIndex — Fetch a single item by index
• deleteByIndex — Delete a single item by index
• upsertByIndex — Insert or update by index
• updateByIndex — Update an existing item by index
• updatePartialByIndex — Partially update an item by index
• pageByIndex — Fetch a page of items with pagination metadata
• listByIndex — Stream matching items from a sorted index in batches, optionally capped by limit

Getting Items

Use getByIndex to fetch a single item by providing all required key fields.

Code: Getting by Keyed Index

export async function getExample(modelService: ModelIndexedSupport) {
  const user = await modelService.getByIndex(User, userByName, {
    name: 'John Doe'
  });
  return user;
}

export async function getScopedExample(modelService: ModelIndexedSupport) {
  const user = await modelService.getByIndex(User, userByName, {
    name: 'John Doe',
    id: 'user-123'
  });
  return user;
}

For sorted indexes with key fields, you must provide all key values plus the sort value if using it to identify a specific item. All single-item index operations also accept an optional id in the request body. This is useful when the index is not unique and you need to ensure the supplied index values resolve to the same record as the provided id, such as enforcing a pattern like "userId matches".

Code: Disambiguating with id

export async function getScopedExample(modelService: ModelIndexedSupport) {
  const user = await modelService.getByIndex(User, userByName, {
    name: 'John Doe',
    id: 'user-123'
  });
  return user;
}

Deleting Items

Use deleteByIndex to remove an item by index.

Code: Deleting by Index

export async function deleteExample(modelService: ModelIndexedSupport) {
  await modelService.deleteByIndex(User, userByName, {
    name: 'John Doe'
  });
}

As with getByIndex, you can pass an optional id to ensure the computed index values resolve to the expected record before deleting it.

Upserting Items

Use upsertByIndex to insert a new item or update an existing one. The index acts as a primary key.

Code: Upserting by Index

export async function upsertExample(modelService: ModelIndexedSupport) {
  const user = await modelService.upsertByIndex(User, userByName, {
    id: 'user-1',
    name: 'John Doe',
    email: '[email protected]'
  });
  return user;
}

Updating Items

Use updateByIndex to update an existing item, or updatePartialByIndex for partial updates.

Code: Updating by Index

export async function updateExample(modelService: ModelIndexedSupport) {
  // Full update — all fields required
  const user = await modelService.updateByIndex(User, userByName, {
    id: 'user-1',
    name: 'John Doe',
    email: '[email protected]',
    age: 31
  });
  return user;
}

export async function updatePartialExample(modelService: ModelIndexedSupport) {
  // Partial update — only updated fields required
  const user = await modelService.updatePartialByIndex(User, userByName, {
    name: 'John Doe',
    email: '[email protected]'
  });
  return user;
}

Listing Items

Use pageByIndex when you want paginated access to a sorted index.

Code: Paging by Sorted Index

export async function listExample(modelService: ModelIndexedSupport) {
  const result = await modelService.pageByIndex(
    User,
    recentUsers,
    {},
    {
      limit: 20,
      offset: '0'
    }
  );

  console.log(result.items); // Array of users
  console.log(result.nextOffset); // Token for next page, if more results exist
  return result;
}

Use listByIndex when you want to iterate through matching items as an async stream of batches. The same list options used by list are supported here, including limit when you want to stop after a fixed number of records.

Code: Streaming by Sorted Index

export async function listStreamExample(modelService: ModelIndexedSupport) {
  const items: User[] = [];

  for await (const batch of modelService.listByIndex(User, recentUsers, {}, { limit: 25 })) {
    items.push(...batch);
  }

  return items;
}

You can also provide key values to filter within a sorted index with pageByIndex:

Code: Listing with Key Filter

export async function listWithFilterExample(modelService: ModelIndexedSupport) {
  // Get all users named 'John' sorted by age
  const result = await modelService.pageByIndex(
    User,
    usersByNameAge,
    {
      name: 'John'
    },
    {
      limit: 10
    }
  );
  return result;
}

Integration

Index registration happens automatically when models are decorated with @Model. Model services like Memory Model Support, MongoDB Model Support, and SQL Model Service implement the ModelIndexedSupport interface to provide indexed access.

Reading Registry Information

You can access registered indexes via ModelRegistryIndex at runtime:

Code: Accessing Model Indexes

export function registryAccessExample() {
  const registry = ModelRegistryIndex.getConfig(User);
  const indexes = registry.indices; // Map of all indexes for the model

  // Access a specific index
  const userByName = indexes?.userByName;
  return userByName;
}

Best Practices

• Plan indexes strategically — Define indexes for your common query patterns
• Use composite keys — When filtering by multiple fields, include all of them in a single index
• Leverage sorting — Use sorted indexes for paginated lists and range queries
• Enforce uniqueness — Use uniqueIndex for fields that must be globally unique
• Handle errors gracefully — Catch IndexedFieldError when working with user input