@flatfile/implementation-utils-collection-wrapper

Provides a wrapper on top of @flatfile/api to provide a more convenient interface for working with V2 records using collect.js. This package includes three main components: Item for individual record management, ItemCollection for working with groups of records, and CollectionClient as a simplified API client wrapper.

Features

• Item: A wrapper class for a single V2 API record that provides additional functionality for working with records such as tracking mutations, merging, hashing, and type-safe property access.
• ItemCollection: A collection wrapper built on collect.js that provides specialized filtering methods for working with groups of Items.
• CollectionClient: A simplified wrapper around the Flatfile API client that automatically converts raw records to Items and handles batch operations.
• Type Safety: Full TypeScript support with strong typing for record operations.
• Cast Utilities: Helper functions for safely casting values to strings and numbers.

Installation

npm install @flatfile/implementation-utils-collection-wrapper

Usage

Basic Example

import {
  CollectionClient,
  Item,
  ItemCollection,
} from "@flatfile/implementation-utils-collection-wrapper";

// Initialize the client
const client = new CollectionClient();

// Define your record type
interface Person {
  firstName?: string;
  lastName?: string;
  email?: string;
  age?: number;
}

// Fetch records from a sheet
const records = await client.get<Person>("sheet-id", { limit: 100 });

// Work with individual items
records.each((person) => {
  if (person.str("firstName") === "John") {
    person.set("lastName", "Updated");
    person.info("firstName", "Name was processed");
  }
});

// Update records back to Flatfile
await client.update(records, { sheetId: "sheet-id" });

API Reference

CollectionClient

A wrapper around the Flatfile API client that simplifies record operations.

class CollectionClient {
  constructor(client?: FlatfileClient);

  // Fetch records from a sheet and convert to ItemCollection
  async get<T>(
    sheetId: string,
    options?: GetRecordsRequestOptions
  ): Promise<ItemCollection<T>>;

  // Update changed records back to Flatfile
  async update<T>(
    records: ItemCollection<T>,
    options?: WriteRecordsRequestOptions
  ): Promise<void>;
}

Methods:

• get() - Fetches records from a sheet, filters invalid records, and returns an ItemCollection
• update() - Sends only changed records back to Flatfile and commits the changes

Item

A wrapper class for individual V2 API records with change tracking and utility methods.

class Item<T> {
  // Property access
  get<K>(key: K): T[K] | undefined;
  set<K>(key: K, value: T[K]): this;
  unset(key: K): this;

  // Type-safe getters
  str(key: K): string | undefined; // Get as trimmed string
  defStr(key: K): string; // Get as string with empty fallback
  bool(key: K): boolean; // Get as boolean
  num(key: K): number; // Get as parsed number

  // State management
  isDirty(options?: { key?: K }): boolean;
  commit(): void;

  // Messages
  info(key: K, message: string): this;
  warn(key: K, message: string): this;
  err(key: K, message: string): this;

  // Utility methods
  keys(): K[];
  values(): Record<K, T[K]>;
  entries(): [K, T[K]][];
  hash(...keys: K[]): string;

  // Record operations
  merge<U>(item: Item<U>, options?: { overwrite?: boolean }): Item<T & U>;
  hasConflict(item: Item<T>, options?: { keys?: K[] }): boolean;
  copy<U>(options: CopyOptions<U>): Item<T & U>;

  // Deletion
  delete(): void;
  isDeleted(): boolean;
}

Key Features:

• Change Tracking: Automatically tracks which properties have been modified
• Message Management: Add info, warning, and error messages to specific fields
• Type Safety: Type-safe property access with casting utilities
• Merging: Merge data from other items with conflict detection
• Hashing: Generate hashes from specific fields for deduplication

ItemCollection

A collection wrapper built on collect.js with specialized filtering methods.

class ItemCollection<T> extends Collection<Item<T>> {
  // Specialized filters
  forSheet(slugOrId: string): ItemCollection<T>;
  changes(options?: { writeRequired?: boolean }): ItemCollection<T>;
  onlyPresent(): ItemCollection<T>;
  deletions(): ItemCollection<T>;
}

Methods:

• forSheet() - Filter items by sheet slug or ID
• changes() - Get only items with pending changes
• onlyPresent() - Exclude deleted items
• deletions() - Get only deleted items

Plus all standard collect.js methods: filter(), map(), each(), first(), count(), etc.

Advanced Usage

Working with Messages

const item = new Item<Person>({ firstName: "John", email: "invalid-email" });

// Add different types of messages
item.info("firstName", "Name looks good");
item.warn("email", "Email format seems unusual");
item.err("email", "Invalid email format");

// Messages are included in changesets
const changeset = item.changeset();
console.log(changeset.__i); // Array of message objects

Record Merging and Conflict Detection

const person1 = new Item({
  firstName: "John",
  lastName: "Doe",
  email: "[email protected]",
});
const person2 = new Item({
  firstName: "John",
  lastName: "Smith",
  phone: "555-1234",
});

// Check for conflicts
if (person1.hasConflict(person2)) {
  console.log("Records have conflicting values");
}

// Merge records
const merged = person1.merge(person2, { overwrite: false }); // Only adds missing fields
// or
const merged = person1.merge(person2, { overwrite: true }); // Overwrites with truthy values

Batch Operations

// Fetch records
const records = await client.get<Person>("sheet-id");

// Process records
records
  .filter((person) => person.str("email")?.includes("@gmail.com"))
  .each((person) => {
    person.set("emailProvider", "Gmail");
    person.info("emailProvider", "Detected Gmail address");
  });

// Update only changed records
await client.update(records.changes(), { sheetId: "sheet-id" });

Filtering and Querying

const records = await client.get<Person>("sheet-id");

// Get records with changes
const dirtyRecords = records.changes();

// Get records for a specific sheet
const sheetRecords = records.forSheet("contacts");

// Get non-deleted records
const activeRecords = records.onlyPresent();

// Get deleted records
const deletedRecords = records.deletions();

// Chain filters
const recentUpdates = records
  .changes({ writeRequired: true })
  .filter((person) => person.str("lastName")?.startsWith("S"))
  .onlyPresent();

Type Safety

The package provides full TypeScript support:

interface Contact {
  firstName?: string;
  lastName?: string;
  email?: string;
  phone?: string;
}

const contact = new Item<Contact>({ firstName: "John" });

// Type-safe property access
contact.set("firstName", "Jane"); // Valid
contact.set("invalid", "value"); // TypeScript error

// Type-safe getters
const name: string | undefined = contact.get("firstName"); // Correctly typed
const age: number = contact.get("age"); // TypeScript error - 'age' not in Contact

Error Handling

try {
  const records = await client.get<Person>("sheet-id");
  await client.update(records, { sheetId: "sheet-id" });
} catch (error) {
  console.error("Failed to process records:", error);
}

// Individual item validation
const item = new Item<Person>({
  __i: [{ key: "email", message: "Invalid email" }],
});
if (!item.valid) {
  console.log("Item has validation errors");
}