@3dsource/data-loader

Schema-driven data loading for Angular applications, plus a Node.js CLI for converting Google Sheets data into local CSV/JSON assets.

What it provides

• DataLoaderService, a root-provided Angular service
• CSV, JSON, object and language parsers selected by a JSON schema
• one generated loader method per parser id
• drive-parser, a CLI that reads Google Sheets and writes generated assets

The package does not expose an NgModule and does not implement generic binary asset loading, caching, progress events or batch-download APIs.

Requirements

• Node.js 20 or newer for the CLI and repository tooling
• Angular >=19.0.0 <23.0.0
• @3dsource/utils ^1.0.21

Installation

npm install @3dsource/data-loader @3dsource/utils

Angular itself is a peer dependency and is normally already present in the application.

Angular usage

The service loads a schema, creates an API method for every enabled parser and then loads each parser's default asset from:

<document-root>/assets/data/<id>/<subId?>.<extension>

Without subId, the path is <document-root>/assets/data/<id>.<extension>. The default extension is .json.

import { Component, inject } from '@angular/core';
import { DataLoaderService } from '@3dsource/data-loader';

@Component({
  selector: 'app-data-consumer',
  template: '',
})
export class DataConsumerComponent {
  private readonly dataLoader = inject(DataLoaderService);
  readonly loadedData: Record<string, unknown> = {};

  async load(): Promise<void> {
    await this.dataLoader.initialize('/assets/data/app.data-schema.json', this.loadedData);

    console.log(this.loadedData['products']);
  }
}

An application can override a parser's default URL or provide inline data through the global appReady hook. Its API contains one method per schema parser id:

interface DataLoaderApi {
  products: (urlOrData: unknown) => Promise<unknown>;
}

const globalScope = globalThis as typeof globalThis & {
  appReady?: (loader: DataLoaderApi) => void;
};

globalScope.appReady = (loader) => {
  void loader.products('/assets/custom/products.json');
};

Set appReady before calling initialize(). A parser started from this hook is not loaded again from its default URL.

DataLoaderService is root-provided and keeps parser promises between calls; treat initialize() as a one-time application bootstrap operation. It resolves only after every enabled parser returns a truthy result. Source and parse errors are logged internally, so applications that load remote data should race initialization against their own timeout if they need a visible failure state.

Schema

The package contains schema.json, which can be referenced by editors and validation tools. A minimal schema looks like this:

{
  "$schema": "./node_modules/@3dsource/data-loader/schema.json",
  "version": "1.0",
  "description": "Application data",
  "path": "data/",
  "parsers": [
    {
      "id": "products",
      "save": true,
      "load": true,
      "processor": {
        "callBack": "JsonParser",
        "extension": ".json",
        "arguments": {
          "rowAsHead": 0,
          "splitBy": []
        }
      }
    }
  ]
}

Supported processor.callBack values are:

• CSVParser
• JsonParser
• ObjectParser
• LanguageParser

Set load to false to exclude a parser from Angular loading. The save, sheetId, googleDoc and processor.saveFunction fields are primarily used by the CLI.

Drive parser CLI

After installing the published package:

drive-parser <google-doc-config.json> <schema.json> <output-directory>
drive-parser --help

The Google Doc config must contain a GoogleDoc URL prefix. For every schema parser whose save value is not false, the CLI appends sheetId (or id), parses the response and writes it under the output directory.

Pass the output directory with a trailing path separator because the CLI concatenates it with each generated filename, for example:

drive-parser ./GoogleDoc.json ./app.data-schema.json ./src/assets/data/

Repository development

Run these commands from the repository root:

pnpm --filter @3dsource/data-loader build
pnpm --filter @3dsource/data-loader typecheck:node
pnpm --filter @3dsource/data-loader clean

The package test script exercises the built CLI against repository fixtures, so build the package first and expect it to access the configured Google Sheet:

pnpm --filter @3dsource/data-loader build
pnpm --filter @3dsource/data-loader test

To version and publish the package, use the verified root release flow:

pnpm release:package -- data-loader patch latest

The release command publishes to npm; it is not a version-only command. It checks only data-loader and its local workspace dependency @3dsource/utils; unrelated libraries and the demo are not built or tested.

See the live example in projects/demo/src/app/demo-layout/info-pages/data-loader/.