@homedev/objector

Object processing library for YAML/JSON with support for includes, field processors, conditional logic, and extensibility.

Overview

Objector is a comprehensive tool for loading, processing, and transforming YAML/JSON documents with advanced features like:

• 🔄 Document includes and composition
• 🔌 Extensible field processors and data sources
• 🎯 Conditional logic and flow control
• 📦 Module system with custom functions
• 🔍 Variable substitution and selectors
• 📝 Output generation and file writing
• 🎨 Template inheritance and extensions

Installation

bun add @homedev/objector
# or
npm install @homedev/objector
# or
pnpm add @homedev/objector

Quick Start

import { Objector } from '@homedev/objector'

const objector = new Objector()

// Load and process a YAML/JSON file
const result = await objector.load('config.yaml')

import { Objector } from '@homedev/objector'

const objector = new Objector()
      .variables({ env: 'production', version: '1.0.0' }) // Add custom variables
      .include('settings.yaml') // Include additional files

// Process the document
const processed = await objector.load('main.yaml')

Core Features

1. Document Loading and Includes

Load YAML/JSON files with automatic format detection and includes:

const objector = new Objector()

// Set include directories
objector.includeDirectory('./config', './templates')

// Load a file (automatically resolves from include directories)
const config = await objector.load('app-config.yaml')

In your YAML/JSON:

includes:
  - base-config.yaml
  - settings/*.yaml
  - ../shared/defaults.yaml

myConfig:
  setting: value

2. Variables and Substitution

Inject and reference variables throughout your documents:

objector.variables({
  env: 'production',
  apiUrl: 'https://api.example.com',
  version: '2.1.0'
})

Usage in documents:

app:
  environment: $(env)
  api: $(apiUrl)
  version: $(version)

3. Conditional Logic

Control document structure with conditions:

features:
  - name: cache
    .if: $(eq:env, "production")
    .then:
      enabled: true
      ttl: 3600
    .else:
      enabled: false

  - name: debug
    .skip: $(eq:env, "production")
    logging: verbose

4. Loops and Iteration

Generate repeated structures with each:

environments:
  .each:
    .from:
      - dev
      - staging
      - prod
    .model:
      name: $(item)
      url: https://$(item).example.com
      replicas: $(switch:
        from: $(item)
        cases:
          prod: 5
          staging: 2
        default: 1)

5. Template Inheritance

Extend and merge templates:

baseTemplate:
  server:
    port: 8080
    timeout: 30

production:
  extends: .baseTemplate
  server:
    port: 443
    ssl: true
    # Merged result: port=443, timeout=30, ssl=true

6. File Operations

Read files and scan directories:

certificate: $(file:./certs/server.crt)

migrations:
  .from: $(scan:"*.sql", "./migrations")
  .model:
    name: $(item)
    content: $(file:$(item))

7. Custom Modules

Add JavaScript functions as data sources:

modules:
  utils: |
    export function formatDate(ctx, date) {
      return new Date(date).toISOString()
    }
    export function generateId(ctx, prefix) {
      return `${prefix}-${Math.random().toString(36).substr(2, 9)}`
    }

records:
  - id: $(utils.generateId:"rec")
    created: $(utils.formatDate:"2024-01-01")

8. Output Generation

Generate multiple output files from one source:

output:
  - file: dist/config.json
    content:
      version: $(version)
      settings: $(config)
  
  - file: dist/README.md
    content: |
      # Configuration
      Version: $(version)

// Write all outputs to disk
await objector.writeAll()

Built-in Keys

Keys are object properties prefixed with . and are used for structure/flow control.

• .output - Register output payload(s)
• .load - Load one or more additional files
• .each - Repeat a model over a source list
• .map - Map a source list into a transformed list
• .from - Resolve a selector/reference into current node
• .concat - Concatenate arrays from selectors
• .extends - Merge from one or many templates
• .group - Expand grouped items into parent list
• .if - Conditional branch using .then / .else
• .switch - Case-based branching with cases / default
• .skip - Remove parent when condition is true
• .metadata - Attach metadata for a path
• .modules - Register source functions from inline JS modules
• .try - Guard expression with optional .catch
• .let - Introduce temporary variables into context
• .tap - Debug current path/value to console

Built-in Data Sources

Sources are used with $() syntax.

Core Sources

• Files: include, exists, file, scan
• String/format: substring, repeat, pad, formatAs
• List: merge, join, range, filter
• Conditions: if, check, and, or, xor, true, false, eq, ne, lt, le, gt, ge, in, notin, contains, notcontains, containsi, notcontainsi, null, notnull, empty, notempty, nullorempty, notnullorempty
• Types: convert, isType, typeOf
• Flow helpers: each, switch, default, section

Built-in Macro Sources

These are also available as $() sources by default.

• String/text: env, uuid, pascal, camel, capital, snake, kebab, len, reverse, concat, trim, ltrim, rtrim, lower, upper, encode, decode
• Collection/object: list, object, unique, groupBy, chunk, flatten, compact, head, tail
• Path/filepath: pathJoin, resolve, dirname, basename, normalize, extname, relative, isAbsolute, segments
• Hash/encoding: md5, sha1, sha256, sha512, hash, hmac, base64Encode, base64Decode, hexEncode, hexDecode
• Math: add, subtract, multiply, divide, modulo, power, sqrt, abs, floor, ceil, round, min, max, clamp, random
• Date/time: timestamp, iso, utc, formatDate, parseDate, year, month, day, dayOfWeek, hours, minutes, seconds
• Regex: regexTest, regexMatch, regexMatchAll, regexReplace, regexReplaceAll, regexSplit, regexExec, regexSearch
• Validation/predicates: isEmail, isUrl, isJson, isUuid, minLength, maxLength, lengthEquals, isNumber, isString, isBoolean, isArray, isObject, isEmpty, isTruthy, isFalsy, inRange, matchesPattern, includes, startsWith, endsWith
• Debug: log

API Reference

Objector Class

Constructor

const objector = new Objector()

Configuration Methods

• use(handler: (o: Objector) => void): this
  Apply a configuration handler

• includeDirectory(...dirs: string[]): this
  Add directories to search for includes

• include(...files: string[]): this
  Add include files directly

• keys(keys: FieldProcessorMap): this
  Add or override key processors

• sources(sources: FieldProcessorMap): this
  Add or override data sources

• sections(sections: Record<string, FieldProcessorSectionFunc>): this
  Register section handlers by type

• variables(vars: Record<string, any>): this
  Set global variables

• functions(funcs: Record<string, AsyncFunction>): this
  Register callable functions for expressions and script sections

• fieldOptions(options: ProcessFieldsOptions): this
  Customize field processing behavior

• filter(f: RegExp): this
  Add a filter for field processing

• output(o: Output): this
  Add an output definition

• metadata(path: string, value: unknown): this
  Store metadata for a path

• section(section: FieldProcessorSection): this
  Register a named section

• encoders(encoders: Record<string, Encoder>): this
  Add output encoders (used by formatAs)

• decoders(decoders: Record<string, LoadFormatParser>): this
  Add input decoders for load()

• setCacheMaxSize(size: number): this
  Set include/content cache size

• clearCache(): this
  Clear cached loaded content

Processing Methods

• async load<T>(fileName: string, cwd?: string, container?: any, noProcess?: boolean): Promise<T | null>
  Load and process a file

• async loadObject<T>(data: T, container?: any, options?: LoadObjectOptions): Promise<T>
  Process an object with field processors

• async writeAll(options?: WriteOutputsOption): Promise<void>
  Write all registered outputs to disk

Query Methods

• getIncludeDirectories(): string[]
  Get all include directories

• getFunctions(): Record<string, AsyncFunction>
  Get registered functions

• getOutputs(): Output[]
  Get all registered outputs

• getMetadata(path: string): any
  Get metadata for a path

• getAllMetadata(): Record<string, any>
  Get all metadata

• getCurrentContext(): LoadContext | null
  Get current processing context

• getSection(name: string): FieldProcessorSection | undefined
  Get a stored section by name

• getEncoder(name: string): Encoder | undefined
  Get an encoder by name

• getDecoder(name: string): LoadFormatParser | undefined
  Get a decoder by name

Advanced Usage

Custom Data Sources

import { Objector } from '@homedev/objector'

const objector = new Objector()

objector.sources({
  // Simple value source
  timestamp: () => Date.now(),
  
  // Source with arguments
  add: (ctx) => {
    const [a, b] = ctx.args
    return a + b
  },
  
  // Async source
  fetchData: async (ctx) => {
    const url = ctx.args[0]
    const response = await fetch(url)
    return response.json()
  }
})

Usage:

data:
  created: $(timestamp)
  sum: $(add:10, 20)
  remote: $(fetchData:"https://api.example.com/data")

Custom Key Processors

objector.keys({
  // Custom transformation key
  uppercase: (ctx) => {
    return ctx.value.toUpperCase()
  },
  
  // Conditional key
  when: (ctx) => {
    const condition = ctx.value
    if (!condition) {
      return NavigateResult.DeleteParent()
    }
    return NavigateResult.DeleteItem()
  }
})

Usage:

settings:
  name:
    .uppercase: hello world
    # Result: "HELLO WORLD"

feature:
  .when: $(eq:env, "production")
  enabled: true
  # Only included when condition is true

Metadata Collection

const objector = new Objector()

// Metadata is collected during processing
await objector.load('config.yaml')

// Retrieve metadata
const metadata = objector.getMetadata('some.path')
const allMetadata = objector.getAllMetadata()

In documents:

service:
  .metadata:
    version: 1.0
    author: team-a
  name: api
  port: 8080

Section Blocks (<@ section ... @>)

Objector can process inline section blocks inside strings. This is useful for generated text files (Markdown, scripts, templates).

Basic Syntax

<@ section[:type] @>
# optional YAML config
---
# section body
<@ /section @>

• :type is optional. Example: section:script.
• --- separates YAML config from section content.
• If no --- is present, the block is treated as content-only.

Script Sections

script is the default built-in section type.

doc: |
  <@ section:script @>
  using: [shout, project]
  condition: context.enabled
  each: context.items
  ---
  section.writeLine(`# ${project}`)
  return shout(String(item))
  <@ /section @>

For section:script, the body runs as async JavaScript with access to:

• section: helper object (write, writeLine, prepend, clear, setOptions, getLines)
• context: current root/context object
• config: section YAML config
• system: current Objector instance
• item: current item when each is used
• values listed in using (from registered functions() or root variables)

condition can skip rendering, and each can iterate an array expression. If the script returns a string, that value becomes the rendered section output.

Reusing Named Sections

You can store a section by name and render it later with $(section:name):

doc: |
  <@ section @>
  name: greeting
  show: true
  ---
  Hello $(user)
  <@ /section @>

footer: $(section:greeting)

Examples

Example 1: Multi-Environment Configuration

# base.yaml
includes:
  - vars-$(env).yaml

application:
  name: myapp
  version: 1.0.0
  
database:
  host: $(db.host)
  port: $(db.port)
  name: $(db.name)

features:
  - name: caching
    .if: $(eq:env, "production")
    .then:
      enabled: true
      ttl: 3600
    .else:
      enabled: false

  - name: logging
    level: $(switch:
      from: $(env)
      cases:
        production: warn
        staging: info
      default: debug)

Example 2: Service Generation

services:
  .each:
    .from:
      - name: api
        port: 8080
        replicas: 3
      - name: worker
        port: 8081
        replicas: 5
      - name: web
        port: 3000
        replicas: 2
    .model:
      name: $(item.name)
      type: deployment
      spec:
        replicas: $(item.replicas)
        template:
          containers:
            - name: $(item.name)
              image: myapp/$(item.name):$(version)
              ports:
                - containerPort: $(item.port)
              env:
                - name: NODE_ENV
                  value: $(env)

Example 3: Dynamic Output Generation

outputs:
  .each:
    .from: $(scan:"*.template.yaml", "./templates")
    .model:
      .output:
        file: dist/$(replace:$(item), ".template.yaml", ".yaml")
        content: $(file:$(item))

FAQ

Q: How do I debug field processing?
A: Use the $(log:message) data source to print values during processing.

Q: Can I use async operations in custom sources?
A: Yes! All field processors and data sources support async/await.

Q: How do I handle circular includes?
A: The system tracks included files to prevent circular dependencies.

Q: Can I extend the built-in processors?
A: Yes, use objector.keys() and objector.sources() to add or override processors.

Q: How do I access nested values?
A: Use the selector syntax: $(some.nested.value) or $(@root.some.value) for root access.

Q: What's the difference between keys and sources?
A: Keys are prefixed with . and control structure/flow (.if, .each). Sources use $() syntax and provide values ($(env:VAR), $(file:path)).