@memberjunction/data-context

The @memberjunction/data-context library provides a metadata-driven framework for managing collections of related data items in MemberJunction applications. It enables developers to define, load, persist, and manipulate data from multiple sources -- views, queries, entities, single records, and raw SQL -- through a unified, type-safe API that works across both client and server tiers.

Architecture Overview

graph TD
    subgraph Consumer["Consumer Code"]
        style Consumer fill:#2d6a9f,stroke:#1a4971,color:#fff
        A["Application / Agent / Workflow"]
    end

    subgraph DataContextPkg["@memberjunction/data-context"]
        style DataContextPkg fill:#7c5295,stroke:#563a6b,color:#fff
        DC["DataContext"]
        DCI["DataContextItem"]
        DCFI["DataContextFieldInfo"]
    end

    subgraph Sources["Data Sources"]
        style Sources fill:#2d8659,stroke:#1a5c3a,color:#fff
        V["Views (RunView)"]
        Q["Queries (RunQuery)"]
        E["Full Entities"]
        SR["Single Records"]
        SQL["SQL Statements"]
    end

    subgraph Persistence["Persistence Layer"]
        style Persistence fill:#b8762f,stroke:#8a5722,color:#fff
        DCE["DataContextEntity"]
        DCIE["DataContextItemEntity"]
    end

    A --> DC
    DC -->|"manages"| DCI
    DCI -->|"field metadata"| DCFI
    DCI -->|"loads from"| V
    DCI -->|"loads from"| Q
    DCI -->|"loads from"| E
    DCI -->|"loads from"| SR
    DCI -->|"loads from"| SQL
    DC -->|"persists via"| DCE
    DC -->|"persists items via"| DCIE

Installation

npm install @memberjunction/data-context

Key Concepts

A Data Context is a named collection of Data Context Items, each of which represents a distinct data source. An item can be one of five types:

| Type | Description | Required Fields | |-----------------|--------------------------------------------------|----------------------------------| | view | A saved MemberJunction User View | ViewID, EntityID | | query | A registered MemberJunction Query | QueryID | | full_entity | All records from an entity | EntityID | | single_record | One record, optionally with related entity data | EntityID, RecordID | | sql | A raw SQL statement (server-side only) | SQL |

Class Hierarchy

classDiagram
    class DataContext {
        +ID: string
        +DataContextEntity: DataContextEntity
        +Items: DataContextItem[]
        +LoadMetadata(id, contextUser, provider) Promise~boolean~
        +LoadData(dataSource, forceRefresh, ...) Promise~boolean~
        +Load(id, dataSource, ...) Promise~boolean~
        +SaveItems(contextUser, persistItemData) Promise~boolean~
        +AddDataContextItem() DataContextItem
        +ValidateDataExists(ignoreFailedLoadItems) boolean
        +ConvertToSimpleObject(itemPrefix, includeFailedLoadItems) object
        +CreateSimpleObjectTypeDefinition(itemPrefix, includeFailedLoadItems) string
        +LoadDataFromObject(data) boolean
        +Clone(context, includeData, contextUser)$ Promise~DataContext~
        +FromRawData(rawData)$ Promise~DataContext~
        +CreateDataContextItem()$ DataContextItem
        +MapEntityFieldsToDataContextFields(entity)$ DataContextFieldInfo[]
    }

    class DataContextItem {
        +Type: ItemType
        +RecordID: string
        +EntityID: string
        +ViewID: string
        +QueryID: string
        +RecordName: string
        +SQL: string
        +CodeName: string
        +EntityName: string
        +Fields: DataContextFieldInfo[]
        +DataContextItemID: string
        +Data: object[]
        +DataLoaded: boolean
        +DataLoadingError: string
        +Description: string
        +AdditionalDescription: string
        +LoadData(dataSource, ...) Promise~boolean~
        +LoadDataFromObject(data) boolean
        +ValidateDataExists(ignoreFailedLoad) boolean
        +FromViewEntity(viewEntity)$ DataContextItem
        +FromSingleRecord(record)$ DataContextItem
        +FromQuery(query)$ DataContextItem
        +FromFullEntity(entity)$ DataContextItem
        +FromRawItem(rawItem)$ DataContextItem
        #LoadFromView(contextUser) Promise~boolean~
        #LoadFromFullEntity(contextUser) Promise~boolean~
        #LoadFromSingleRecord(contextUser, ...) Promise~boolean~
        #LoadFromQuery(contextUser) Promise~boolean~
        #LoadFromSQL(dataSource, contextUser) Promise~boolean~
    }

    class DataContextFieldInfo {
        +Name: string
        +Type: string
        +Description: string
    }

    class DataContextItemServer {
        #LoadFromSQL(dataSource, contextUser) Promise~boolean~
    }

    DataContext "1" --> "*" DataContextItem : contains
    DataContextItem "1" --> "*" DataContextFieldInfo : describes fields
    DataContextItemServer --|> DataContextItem : extends

Data Lifecycle

flowchart LR
    subgraph Step1["1. Create"]
        style Step1 fill:#2d6a9f,stroke:#1a4971,color:#fff
        C1["new DataContext()"]
    end

    subgraph Step2["2. Load Metadata"]
        style Step2 fill:#7c5295,stroke:#563a6b,color:#fff
        C2["LoadMetadata(id)"]
    end

    subgraph Step3["3. Load Data"]
        style Step3 fill:#2d8659,stroke:#1a5c3a,color:#fff
        C3["LoadData(dataSource)"]
    end

    subgraph Step4["4. Process"]
        style Step4 fill:#b8762f,stroke:#8a5722,color:#fff
        C4["ValidateDataExists()\nConvertToSimpleObject()"]
    end

    subgraph Step5["5. Persist"]
        style Step5 fill:#2d6a9f,stroke:#1a4971,color:#fff
        C5["SaveItems(contextUser)"]
    end

    Step1 --> Step2 --> Step3 --> Step4 --> Step5

Usage

Loading an Existing Data Context

Use Load() to fetch both metadata and data in a single call:

import { DataContext } from '@memberjunction/data-context';

const context = new DataContext();
const loaded = await context.Load(
  dataContextID,   // ID of the data context record
  dataSource,      // Required only for SQL-type items (server-side)
  false,           // forceRefresh -- reload even if data is cached
  true,            // loadRelatedDataOnSingleRecords
  10,              // maxRecordsPerRelationship
  contextUser      // required on server-side
);

if (loaded) {
  context.Items.forEach(item => {
    console.log(`${item.Description}: ${item.Data?.length ?? 0} rows`);
  });
}

Alternatively, call LoadMetadata() and LoadData() separately for finer control:

const context = new DataContext();
await context.LoadMetadata(dataContextID, contextUser);

// Inspect or modify items before loading data
console.log(`Items to load: ${context.Items.length}`);

await context.LoadData(dataSource, false, true, 10, contextUser);

Creating Items Programmatically

Each item type has a dedicated static factory method.

From a View

import { UserViewEntityExtended } from '@memberjunction/core-entities';

const viewEntity: UserViewEntityExtended = await md.GetEntityObject<UserViewEntityExtended>('User Views');
await viewEntity.Load(viewID);

const viewItem = DataContextItem.FromViewEntity(viewEntity);
context.Items.push(viewItem);

From a Single Record

import { BaseEntity } from '@memberjunction/core';

const record: BaseEntity = await md.GetEntityObject('Customers');
await record.Load(recordID);

const recordItem = DataContextItem.FromSingleRecord(record);
context.Items.push(recordItem);

From a Query

import { QueryInfo } from '@memberjunction/core';

const queryInfo = md.Queries.find(q => q.Name === 'Monthly Revenue');
if (queryInfo) {
  const queryItem = DataContextItem.FromQuery(queryInfo);
  context.Items.push(queryItem);
}

From a Full Entity

import { EntityInfo } from '@memberjunction/core';

const entityInfo = md.Entities.find(e => e.Name === 'Products');
if (entityInfo) {
  const entityItem = DataContextItem.FromFullEntity(entityInfo);
  context.Items.push(entityItem);
}

Loading Data for Items

// Load data for all items in the context
const allLoaded = await context.LoadData(
  dataSource,  // required for SQL-type items
  false,       // forceRefresh
  true,        // loadRelatedDataOnSingleRecords
  10           // maxRecordsPerRelationship
);

// Or load data for a single item
const itemLoaded = await context.Items[0].LoadData(
  dataSource, false, true, 10, contextUser
);

Loading Pre-fetched Data

If you already have the data available (for example, from a cache or external API), load it directly:

// Load into a single item
const item = context.Items[0];
item.LoadDataFromObject(myDataArray);

// Load into all items at once (2D array, one sub-array per item in order)
context.LoadDataFromObject([
  rowsForItem0,
  rowsForItem1,
  rowsForItem2
]);

Saving Data Context Items

const saved = await context.SaveItems(
  contextUser,  // context user for server-side operations
  true          // persistItemData -- also save loaded data as JSON
);

if (saved) {
  // Each item now has a DataContextItemID populated
  context.Items.forEach(item => {
    console.log(`Saved item ${item.DataContextItemID}`);
  });
}

Items are saved in a single transaction group -- either all succeed or none are committed.

Working with Loaded Data

// Validate all items have data
if (context.ValidateDataExists()) {
  // Convert to a flat keyed object
  const simpleObj = context.ConvertToSimpleObject('item_', false);
  // Result: { item_0: [...], item_1: [...], ... }

  // Generate a TypeScript-style type definition string
  const typeDef = context.CreateSimpleObjectTypeDefinition('item_');
  // Output: {item_0: []; // View: Customer List, From Entity: Customers\n...}
}

// Check individual items for errors
context.Items.forEach(item => {
  if (item.DataLoaded && item.Data) {
    console.log(`${item.Description}: ${item.Data.length} rows`);
  } else if (item.DataLoadingError) {
    console.error(`Failed: ${item.DataLoadingError}`);
  }
});

Cloning a Data Context

const clonedContext = await DataContext.Clone(
  originalContext,
  true,         // includeData -- copy loaded data into the clone
  contextUser
);

if (clonedContext) {
  console.log(`Cloned context ID: ${clonedContext.ID}`);
}

Reconstructing from Raw Data

const context = await DataContext.FromRawData(rawObject);
// rawObject should have { ID, Items: [ { Type, RecordID, ... }, ... ] }

Server-Side SQL Support

The base DataContextItem class throws an error when LoadFromSQL() is called because raw SQL execution is inherently a server-side operation. The companion package @memberjunction/data-context-server provides DataContextItemServer, which overrides this method using an mssql connection pool:

flowchart TD
    subgraph Client["Client Tier"]
        style Client fill:#2d6a9f,stroke:#1a4971,color:#fff
        DCI_C["DataContextItem\n(base class)"]
    end

    subgraph Server["Server Tier"]
        style Server fill:#2d8659,stroke:#1a5c3a,color:#fff
        DCI_S["DataContextItemServer\n(@RegisterClass override)"]
        MSSQL["mssql ConnectionPool"]
    end

    DCI_C -->|"LoadFromSQL() throws error"| X["Not supported client-side"]
    DCI_S -->|"LoadFromSQL() executes via"| MSSQL
    DCI_S -.->|"extends"| DCI_C

DataContextItemServer is registered via @RegisterClass with a higher priority, so the MemberJunction class factory automatically returns the server variant when the server package is included. No code changes are needed -- just include @memberjunction/data-context-server in your server project's dependencies.

API Reference

DataContext

| Method | Returns | Description | |--------|---------|-------------| | LoadMetadata(id, contextUser?, provider?) | Promise<boolean> | Loads metadata for the context and all its items from the database | | LoadData(dataSource, forceRefresh?, loadRelated?, maxRecords?, contextUser?) | Promise<boolean> | Loads data for all items; must call LoadMetadata() first | | Load(id, dataSource, forceRefresh?, loadRelated?, maxRecords?, contextUser?) | Promise<boolean> | Combined metadata + data load in one call | | SaveItems(contextUser?, persistItemData?) | Promise<boolean> | Persists all items to the database in a single transaction | | AddDataContextItem() | DataContextItem | Creates a new item and adds it to Items | | ValidateDataExists(ignoreFailedLoadItems?) | boolean | Returns true if all items have their Data property set | | ConvertToSimpleObject(itemPrefix?, includeFailedLoadItems?) | object | Converts context to a flat keyed object | | CreateSimpleObjectTypeDefinition(itemPrefix?, includeFailedLoadItems?) | string | Generates a type definition string for the simple object | | LoadDataFromObject(data) | boolean | Loads pre-fetched data as a 2D array mapped by item index | | Clone(context, includeData?, contextUser?) (static) | Promise<DataContext> | Deep-clones a context and its items into new database records | | FromRawData(rawData) (static) | Promise<DataContext> | Reconstructs a DataContext from a plain object | | CreateDataContextItem() (static) | DataContextItem | Creates a new item via the class factory (does not add to context) | | MapEntityFieldsToDataContextFields(entity) (static) | DataContextFieldInfo[] | Maps EntityInfo fields to simplified field info objects |

DataContextItem

| Property | Type | Description | |----------|------|-------------| | Type | 'view' \| 'query' \| 'full_entity' \| 'sql' \| 'single_record' | The data source type | | RecordID | string | Primary key for single_record items (comma-separated for composite keys) | | EntityID | string | Entity identifier (not used for query or sql types) | | ViewID | string | View identifier (only for view type) | | QueryID | string | Query identifier (only for query type) | | RecordName | string | Display name of the view, query, or entity | | SQL | string | SQL statement (only for sql type) | | CodeName | string | System-generated unique code name within the context | | EntityName | string | Entity name (not used for query or sql types) | | Fields | DataContextFieldInfo[] | Field metadata for the item | | DataContextItemID | string | Database record ID (populated after save) | | Data | object[] | The loaded data rows | | DataLoaded | boolean | Whether data has been successfully loaded | | DataLoadingError | string | Error message if loading failed | | Description | string (readonly) | Auto-generated description based on type | | AdditionalDescription | string | Optional supplementary description |

| Static Factory Method | Description | |----------------------|-------------| | FromViewEntity(viewEntity) | Creates an item from a UserViewEntityExtended | | FromSingleRecord(record) | Creates an item from a BaseEntity instance | | FromQuery(query) | Creates an item from a QueryInfo | | FromFullEntity(entity) | Creates an item from an EntityInfo | | FromRawItem(rawItem) | Creates an item from a plain object |

DataContextFieldInfo

| Property | Type | Description | |----------|------|-------------| | Name | string | Field name | | Type | string | Field data type | | Description | string (optional) | Field description |

Dependencies

| Package | Purpose | |---------|---------| | @memberjunction/global | Class factory and @RegisterClass decorator | | @memberjunction/core | Core framework: Metadata, RunView, RunQuery, BaseEntity, EntityInfo, UserInfo | | @memberjunction/core-entities | Generated entity classes: DataContextEntity, DataContextItemEntity, UserViewEntityExtended |

Best Practices

1. Always load metadata before data. Use LoadMetadata() then LoadData(), or use the combined Load() method.
2. Check DataLoaded and DataLoadingError on each item after loading. LoadData() does not throw exceptions for individual item failures -- it returns false and populates the error properties.
3. Pass contextUser on the server side. Server code serves multiple users concurrently and must include the context user for proper security and audit tracking.
4. Use maxRecordsPerRelationship to limit the amount of related data loaded for single_record items, especially in production environments.
5. Validate before processing. Call ValidateDataExists() before working with data to ensure all items are ready.
6. Leverage transactions. SaveItems() automatically wraps all item saves in a transaction group for atomicity.
7. Include @memberjunction/data-context-server in server projects if any items use the sql type. The class factory handles the override automatically.

License

ISC