@daneren2005/shared-memory-ecs

v1.11.0

Published

A small, reusable Entity/Component/System core backed by shared memory.

Readme

@daneren2005/shared-memory-ecs

A small, reusable Entity / Component / System core backed by shared memory (@daneren2005/shared-memory-objects). It contains only engine-agnostic ECS concepts. Games supply their own components, entity classes, templates, systems, and domain data.

Concepts

  • Component – a plain object with an index (its block inside a shared-memory pool) plus getters/ setters over that memory. Games decide what components exist.
  • ComponentDefinition – describes a component: its typed array type, block size, the config keys that trigger loading (loadProperties), and the two halves that build it — toBlock(config) (maps the config to the raw block values) and attach(entity, memory, index) (builds the accessor over that block). Loading a component is attach(entity, memory, memory.create(toBlock(config))); splitting it this way lets a worker build the block off-thread (it calls only toBlock) and the main thread wrap it (attach) when it adopts the entity. attach also receives the load config as an optional fourth argument, for main-thread-only state kept off the block (e.g. a display name); it is undefined when adopting a worker-built block. Plus an optional save(component), free(component), and died(component, entity, world). A component's data splits into Config (defining props supplied up front, e.g. maxHealth) and Serialization (runtime-derived state, e.g. current health); toBlock sees Config & Serialization while save returns only the Serialization slice. free runs when the component is torn down (see Freeing extra resources); died runs when the entity is killed, with it still readable (see Reacting to death).
  • ComponentRegistry<C> – the map of all component definitions for a game.
  • EntityFactory<C> – maps an entity type name to a base (template) config. Loading an entity layers the caller's config over its type's template, so shared static data lives in one place and a save only needs the type plus the entity's serialization. Optionally, a template's immutable class selects a game-provided BaseEntity subclass and restricts which components that template may load.
  • BaseWorld<C> – builds one MemoryComponent per registered component (attached to that component's definition as world.registry[name].memoryComponent) and runs systems. It is generic over your component map C, so world.registry, entity.components, setComponent, etc. are fully typed. Its entities live in world.entities, a Map keyed by eid — see Iterating entities.
  • BaseEntity<C> – an eid, its id alias, immutable type, dead state, and a bag of memory-backed components. Every entity has an entity component whose immutable type and optional immutable class are interned strings stored in shared memory. A game can subclass BaseEntity to expose direct convenience getters for one entity class.
  • Systems – System, IterableSystem, EntitySystem (main-thread iteration over entities with a given set of components), EntityWorkerSystem (runs an update function over raw memory blocks, off-thread when Web Workers + SharedArrayBuffer are available), and WorkerSystem (an EntityWorkerSystem that runs once per tick over its sub-queries instead of once per entity — see Per-tick systems).

Defining components

import { Component } from '@daneren2005/shared-memory-ecs';
import type { ComponentDefinition } from '@daneren2005/shared-memory-ecs';

interface HealthComponent {
	index: number
	health: number
	maxHealth: number
}

const HEALTH_INDEX = 0;
const HEALTH_MAX_INDEX = 1;

// The accessor is a subclass of `Component`, so its get/set live on one shared prototype: reading a component off
// thousands of entities stays monomorphic and inline, and each instance is a single allocation rather than a
// closure per accessor. Index `this.block` (the typed-array view, set for you) by the exported *_INDEX constants.
class Health extends Component<Int32Array> implements HealthComponent {
	get health() { return this.block[HEALTH_INDEX]; }
	set health(value: number) { this.block[HEALTH_INDEX] = value; }
	get maxHealth() { return this.block[HEALTH_MAX_INDEX]; }
}

// `maxHealth` is the defining Config prop; `health` is optional, runtime-derived Serialization.  `toBlock`
// and `save` both see the combined `{ maxHealth: number, health?: number }`.
const healthDefinition: ComponentDefinition<HealthComponent, Int32Array, { maxHealth: number }, { health?: number }> = {
	type: Int32Array,
	size: 2,
	// Configs are flat and shared: this component loads whenever `maxHealth` is present, then reads what it
	// needs off the same config object.  Only Config props may appear here.
	loadProperties: ['maxHealth'],
	// The block values, purely from config — so a worker can build the block off-thread. No entity/world access.
	toBlock(config) {
		return [config.health ?? config.maxHealth, config.maxHealth];
	},
	// The accessor over an already-allocated block. Runs on the main thread both when loading and when adopting a
	// worker-built entity; a component's own extra allocations (a SharedList, a resource block) go here — a
	// subclass that owns extra memory takes more constructor args (the entity, another pool) and stores them.
	attach(entity, memoryComponent, index) {
		return new Health(memoryComponent.getBlock(index), index);
	},
	save(component) {
		// save returns only Serialization; maxHealth is Config and comes back from the template on reload.
		return { health: component.health };
	}
};

// One place defines the whole registry + the component map type it implies.
const registry = {
	health: healthDefinition
	// ...other component definitions
};
type Components = { [K in keyof typeof registry]: ReturnType<(typeof registry)[K]['attach']> };

Using the world

import { BaseWorld } from '@daneren2005/shared-memory-ecs';

const world = new BaseWorld<Components>(registry);

const entity = world.loadEntity({ maxHealth: 100 });
entity.components.health!.health -= 10;

console.log(entity.save()); // { health: 90 }

Entity types via the factory

Register per-type base configs, then load entities by type. The template supplies the defining config, so only the type and runtime serialization need to be saved:

import { BaseWorld, EntityFactory } from '@daneren2005/shared-memory-ecs';

const factory = new EntityFactory<Components>({
	goblin: { maxHealth: 20 },
});
const world = new BaseWorld<Components>(registry, { factory });

const goblin = world.loadEntity({ type: 'goblin', health: 10 });
console.log(goblin.components.health!.maxHealth); // 20 (from the template)
console.log(goblin.save()); // { type: 'goblin', health: 10 } - no templated maxHealth

// A save re-expands back through the factory:
world.loadEntity(goblin.save());

An entity's type lives in shared memory (worker-visible), not as a plain field. Each distinct type string is interned once as an immutable ConstantString in the heap — 15 "Space Ship" entities share one allocation — and the entity block stores only a pointer to it at TYPE_INDEX. Reading entity.components.entity.type resolves that pointer back to the string through the world's cache; a worker can do the same (see Reading an entity's type in a worker).

Entity classes and scoped templates

An entity type names one concrete template. Its optional class names the stable family that chooses a wrapper constructor and the components that family permits. Callers still pass only { type, ...overrides }: the factory resolves class from the template, and neither identity can change after construction.

import {
	BaseEntity,
	BaseWorld,
	EntityFactory,
	defineEntityClasses,
	defineEntityConfigs,
} from '@daneren2005/shared-memory-ecs';
import type { ComponentsOf, EntityConfigOf, EntityInstancesOf } from '@daneren2005/shared-memory-ecs';

const registry = {
	health: healthDefinition,
	value: valueDefinition,
};
type Components = ComponentsOf<typeof registry>;
type Config = EntityConfigOf<typeof registry>;

class ActorEntity extends BaseEntity<Components, Config> {
	get health() { return this.components.health?.health; }
}
class ItemEntity extends BaseEntity<Components, Config> {
	get value() { return this.components.value?.value; }
}

const classes = defineEntityClasses(registry, {
	actor: { entity: ActorEntity, components: ['health'] },
	item: { entity: ItemEntity, components: ['value'] },
} as const);

const templates = defineEntityConfigs(classes, {
	guardian: { type: 'guardian', class: 'actor', maxHealth: 100 },
	token: { type: 'token', class: 'item', value: 5 },
} as const);

type Entities = EntityInstancesOf<typeof classes>;
const factory = new EntityFactory<Components, Config, Entities>(templates, classes);
const world = new BaseWorld<typeof registry, Components, Config, Entities>(registry, { factory });

const item = world.loadEntity({ type: 'token' });
console.log(item instanceof ItemEntity);            // true
console.log(item.components.entity.class);          // "item"
console.log(item.id, item.type, item.dead);          // BaseEntity identity accessors

defineEntityConfigs checks each template against its class's component definitions, so adding a health-only property to the item template is a type error. The factory repeats the boundary at runtime: loading or attaching a component outside the class allowlist throws, protecting dynamically loaded data too. Client/editor metadata that no component consumes should remain in a separate authoring type and be transformed into the scoped ECS template.

Classless factories keep the original behavior: they create BaseEntity and scan the complete registry.

System scheduling

world.update(elapsedTime) uses DefaultScheduler to dispatch systems in registration order. Synchronous systems run immediately, workers can overlap after dispatch, and iterable systems can continue across updates. Systems keep their existing shouldRun(), deltaBetweenRuns, and firstRun behavior. The world advances clocks, applies timeScale, skips new updates while paused, prepares shared memory, and manages deferred component frees.

Select a scheduler at construction; world.scheduler is readonly:

import { BaseWorld, DefaultScheduler } from '@daneren2005/shared-memory-ecs';

const world = new BaseWorld(registry, {
	scheduler: new DefaultScheduler<Components>()
});
world.update(16); // Synchronous result: { lastSystemError: null } when no dispatch error occurred.

Custom policies implement Scheduler<C>. Its synchronous update(context, elapsedTime) receives the scaled delta and a SchedulerContext<C> exposing the live systems array, the world's emit method, and optional gameTime (supplied by BaseWorld). It returns SchedulerUpdateResult ({ lastSystemError?: Error | null }). DefaultScheduler preserves the existing dispatch timing events, error reporting, and continuation after a system throws.

Optional systemAdded(context, system) and systemRemoved(context, system) hooks receive notifications after the array changes and before the corresponding world event. Duplicate names remain supported; addSystemIfNotExists checks names and removeSystem removes the first matching name. Direct assignment or mutation of world.systems bypasses membership hooks, so policies must read the live array when needed. The default policy preserves native forEach mutation behavior during dispatch.

Optional reset(context) runs after systems are cleared during reload or non-pristine clear(); pristine worlds skip that teardown. destroy(context) runs once after system destruction. runCompleted(context, system) receives the existing completion boundary: a synchronous run returned, an iterable queue drained, or a current-generation worker reply's effects were applied. Fallback completion can arrive inside scheduler dispatch. The world processes eligible deferred frees before calling this hook. Dispatch return values and system-<name>-finished events do not imply completion. These hooks expose lifecycle notifications; they do not establish deterministic ticks or safe edit barriers.

Use ConflictScheduler to prevent systems from overlapping when one writes a component the other reads or writes. Conflicts follow registration order; read-only systems and systems accessing independent components can overlap:

import { BaseWorld, ConflictScheduler, EntityWorkerSystem } from '@daneren2005/shared-memory-ecs';

const world = new BaseWorld(registry, {
	scheduler: new ConflictScheduler<Components>()
});

class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }> {
	constructor() {
		super(world, {
			name: 'Damage',
			required: ['health'],
			writes: ['health'],
			updateFunction: damageUpdate,
			getWorker: () => new Worker(new URL('./damage-worker.ts', import.meta.url), { type: 'module' })
		});
	}
}
world.addSystem(new DamageSystem());

The scheduler compiles its schedule when systems are added or removed. Inspect it before the first update using the scheduler instance:

const scheduler = new ConflictScheduler<Components>();
const world = new BaseWorld(registry, { scheduler });
// Register your systems here.
console.log(scheduler.schedule.order.map(system => system.name));
console.table(scheduler.schedule.batches.map((batch, index) => ({
	batch: index,
	systems: batch.map(system => system.name).join(', ')
})));

schedule is a frozen ConflictSchedule<C> containing system references. order flattens the compiled dependency levels, and batches group systems that can run concurrently within one update. Every conflict points from the earlier registered system to the later one. Each system goes into the earliest level after its conflicting predecessors. These groups describe dependencies, not runtime barriers: a system starts when its own predecessors finish, even if unrelated systems in an earlier group are still running. Actual dispatch can also overlap independent work from later host updates.

writes lists every component a system changes, including components reached outside its main query. Use writes: [] for a read-only system. Reads are inferred from EntitySystem.components and from worker required, optional, not, and all named queries. The built-in entity component is also a read. Exclusions count because adding or removing an excluded component changes query membership. Add reads: ['otherComponent'] for access outside those queries, including filters and hooks. A plain System or IterableSystem supplies both reads and writes; an EntitySystem without a component query supplies reads explicitly. Component names are checked against the system's component type.

Omitting writes, or leaving reads unknown, makes a system exclusive. exclusive: true also reserves exclusive execution for a system whose callbacks, structural changes, or shared resources cannot be described through component access. Workers with createsEntities: true or addsComponents: true remain exclusive. A component list such as addsComponents: ['onFire'] instead schedules each listed component as a write, so the system is ordered against readers, writers, and exclusions of what it adds. List every component the system can add through addComponentWorker. Declare writes to entity when changing its flags; arbitrary callback effects still need explicit declarations or exclusive execution.

For parallel worker creation, EntityWorkerSystem and WorkerSystem accept a component list in the existing capability option, for example createsEntities: ['health', 'movement']. List every game component the system can create, including components supplied by factory templates. configComponents(world, { type: 'rock', x: 0, y: 0 }) returns the components a creation config loads after merging its template; only key presence matters, so placeholder values are enough. The built-in entity component is included automatically. writes still declares changes to existing components; creation declarations do not authorize undeclared callback or attachment effects on existing data. Both boolean and list forms enable the same worker allocator, factory templates, and stable atomic eids. The default scheduler treats both forms identically and adopts results immediately on completion.

Under ConflictScheduler, compatible creators can allocate and initialize blocks concurrently, even in the same component pool. Publication stays in scheduling order and waits for earlier readers and writers of the affected components; later readers and writers wait until publication is applied. A creator can therefore compute beside an earlier reader without changing that reader's query snapshot. A later query waits for publication when the creator's component list can supply every required component of that query. For example, creating ['transform', 'item'] does not block a query requiring ['transform', 'tree'], even when it optionally reads item or excludes it with not. Main and named queries are checked separately; a match with any one retains the dependency. An empty or entity-only query still observes every creation. Explicit reads outside queries and all writes retain their component conflicts, so accessing world.entities directly must still be declared.

Creation lists describe all components a system can create, not one exact entity shape. Optional components and exclusions therefore cannot prove a query unrelated when its required components could all be created. Custom query filters need explicit reads for accesses outside their declared components. Custom membership overrides retain conservative component-level creation access unless they provide accurate creationReadQueries metadata. This optional system getter exposes separate required-component lists to the conflict scheduler; undefined uses creationReadComponents as before. Queued updates keep their original query and creation snapshots across schedule rebuilds.

Worker completion effects are kept together, preserving component-change, event, death, and adoption order within each run. Attachment hooks and callbacks affecting other shared data still need truthful writes or exclusive: true. Completion promises and memory-free notifications settle only after application; cancellation discards unpublished blocks. These rules apply only to the conflict scheduler.

Each host update queues one update per registered system with its scaled delta and logical game time. Updates of the same system stay sequential. Read/write and write/write conflicts preserve host-update order, then registration order within an update, including dependencies on work that has not started yet. Read/read access across different systems can overlap. Independent systems can advance into later updates while a slow worker is unfinished; there is no whole-update or whole-batch completion barrier. Exclusive work remains a barrier to all earlier and later work, including workers using the boolean creation capability.

When asynchronous systems finish applying effects, a microtask dispatches newly ready work after their completion promises settle, without waiting for another animation frame. Deltas stay in each system's order without merging or dropping them; deltaBetweenRuns, remainders and firstRun retain their existing behavior. While unpaused, sliced systems continue at most once per host update with zero extra delta, and release their dependents when the slice queue drains. Conflicting work can still build a backlog.

During scheduled shouldRun() and update() calls, System.gameTime reports the logical time of the queued update. Worker world.gameTime uses that captured value, on real workers and the synchronous fallback alike. Delayed dispatch therefore does not borrow a later host update's clock. Outside that scope, System.gameTime returns the live world clock. Custom schedulers can use system.withGameTime(time, () => system.update(delta)) to establish the same scope. Plain systems and addDataToWorld() implementations should use this.gameTime when they need scheduled time; direct reads of this.world.gameTime still see the host clock. The default scheduler continues using its existing update(delta) path and live clock.

The compiled schedule is reused until membership changes or scheduler.rebuild(world) is called. Call rebuild after changing access declarations, or to inspect direct edits to world.systems before the next update; direct membership edits are otherwise checked before each dispatch. Queued updates retain their original access reservations, even if metadata is rebuilt while a worker is running. Additions join future host updates; removed pending updates are skipped when their predecessors finish, and removed running systems retain their reservations until completion. With ConflictScheduler, world.pause() or world.paused = true stops enqueueing new updates but finishes draining existing work without further world.update() calls. Queued systems retain their captured deltas and logical times; sliced runs continue through timer tasks with zero extra delta, yielding between slices. Worker effects publish as their dependencies finish. Draining does not advance either world clock; calling world.update() while paused still advances only playerTime. world.resume() allows new work to be queued on the next host update. Reload/clear and destroy discard queued updates and cancel scheduled microtasks and slice timers. The world prepares shared memory before deferred dispatch and prevents new dispatch while clearing or destroyed. Errors from automatic dispatch emit system-error; they cannot change a result already returned by world.update().

Custom asynchronous systems must report isCurrentlyRunning() and notify completion through onRunFinished() when their work and effects finish. Schedulers may implement the optional prepareEffects(context, system) hook: it captures the dispatched run and returns a handler receiving SchedulerEffects with apply() and discard(). The handler must eventually call one of them. beforeClear(context) lets such schedulers discard pending effects before the world awaits workers. The optional pause(context) hook is called when the world's paused flag becomes true, including direct assignment. context.paused indicates whether new host updates are suspended; context.dispatchAllowed and prepareForSystemDispatch() gate deferred dispatch during teardown. Schedulers without these hooks retain immediate application and the existing clear path.

This scheduler provides component conflict ordering with the existing synchronous update API. It does not implement fixed simulation ticks or impose a stable order on unrelated workers' callbacks. World clocks continue advancing with host updates. Iterable slicing uses wall-clock budgets; entity workers can opt into deterministic work batches instead. For a fixed input-update sequence, declared conflicting operations and each system's deltas/logical times remain ordered independently of worker completion order. Independent callbacks and timing events can arrive in different orders; shared effects must be declared as conflicts or marked exclusive. Repeatable simulations also need deterministic inputs and system code, and safe external world edits.

Iterating entities

world.entities is a Map keyed by eid, not an array, and so is entities on EntitySystem and EntityWorkerSystem:

world.entities.forEach(entity => { ... });   // in the order they were added
world.entities.get(eid);                     // same as world.getEntityByEid(eid)
world.entities.size;
for(const entity of world.entities.values()) { ... }

// For the array methods a Map does not have:
Array.from(world.entities.values()).filter(entity => !!entity.components.health);

An entity leaves from anywhere in the middle of that collection — every death is one — and finding it in an array meant a scan of the whole world, then a shift of everything after it, repeated for each system that held it as well. A few thousand entities with a few dozen deaths a run made that the most expensive thing the main thread did on a busy frame; a Map deletes in constant time. It also replaced the separate entitiesByEid lookup, so there is only ever one collection to keep straight.

Iteration order is still insertion order, so load / save round-trip in the order they always did.

Reading components on a hot path

entity.components.health!.health is the way to read a component, and for almost everything it is the right one: it is typed, it is readable, and one read costs nothing worth measuring.

It is not free, though, and the cost shows up in exactly one situation — reading the same component off thousands of entities, every frame. The accessor walks several objects to get there and ends in a getter call over the shared block. Component-subclass accessors are prototype getters (all instances of a type share one hidden class, so those reads stay monomorphic and inline — unlike the old per-entity closures, which went megamorphic), but a getter call plus the property walk still loses to a raw indexed read in the very tightest per-frame loops.

Where that matters, hold the block instead. It is the same memory the accessors read, so nothing changes about what you get, and it is what the update functions already work on:

import { TRANSFORM_X_INDEX } from '@daneren2005/shared-memory-physics';

// Resolve once, when whatever is doing the reading is set up. The component caches its view on `.block` when it
// is attached, so read that rather than paying for another getBlock (each call allocates a fresh subarray).
const block = entity.components.health!.block as Int32Array;

// Then per frame, per entity:
block[HEALTH_INDEX];

Two things come with that. The block is only valid while the component is: re-read .block if the component can be removed and re-added, or hang it off something that dies with the entity. And it is indexed rather than named, so the offsets have to be exported alongside the definition — which is why every component in the physics library exports its *_INDEX constants.

Reach for this when a profile says to, not by default. A menu, a save, a system that touches a dozen entities: use the accessors.

EntityWorkerSystem workers

EntityWorkerSystem needs a getWorker() that returns a real Worker, and an updateFunction. Your worker entry file calls createEntitySystemWorker(self, updateFunction), importing it from the /worker subpath so the worker bundle stays small. When Web Workers or SharedArrayBuffer are unavailable it transparently falls back to running the same update function on the main thread. Attach any extra per-run data (the equivalent of the old faction/fog-of-war fields) by overriding addDataToWorld(world). Declare its shape with the W type parameter (an interface extending EntityWorkerSystemWorld) so both addDataToWorld and the updateFunction see it typed:

interface DamageWorld extends EntityWorkerSystemWorld {
	damage: number
}

const damageUpdate: EntityUpdateFunction<Components, { health: Int32Array }, DamageWorld> =
	(world, entityId, components) => { components.health[0] -= world.damage; };

class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }, DamageWorld> {
	addDataToWorld(world: DamageWorld) { world.damage = 5; }
}

Incremental entity work

Set workBatchSize: 1024 in an EntityWorkerSystem configuration to process at most 1,024 snapshot positions per scheduled update. The size must be a positive safe integer. Batch boundaries depend on entity count and insertion order, never elapsed wall time. The last batch can be smaller, and removed entities consume their original positions without being updated. New members wait for the next pass; current component bundles and named queries are refreshed before every batch.

deltaBetweenRuns starts each full pass. All its batches share the original elapsedTime, logical gameTime, and data from addDataToWorld(). Delta accumulated during the pass is carried toward the next pass. preRun receives the full current entity list once at the start, while optional preBatch receives only the selected entities before each batch. Worker-local fields assigned to world survive until the pass ends. queryChanged and entityRemoved still receive each batch's membership changes.

Each batch publishes callbacks, creations and component changes independently. Under ConflictScheduler, its applied completion releases conflicting systems, allowing physics or other frequent work between batches. Access declarations must cover both hooks and entity updates. This opt-in permits other systems to change shared state between batches; use preBatch to refresh live indexes and avoid retaining component block references across batch boundaries. A failed preparation hook abandons the remaining pass. Clear/reload drops the pending snapshot and worker-local pass state.

This also works with DefaultScheduler and the main-thread fallback. It does not apply to WorkerSystem, whose function runs once over named queries. Full-query preparation and query-delta handling remain unbounded, so a count limit alone cannot guarantee a frame deadline. Paused conflict scheduling drains queued batches; unqueued pass work resumes with future host updates. waitForRunToComplete() waits for the active batch and its publication.

Update-function hooks: init, queryChanged, preRun and preBatch

addDataToWorld runs on the main thread and re-sends its data every run. When the data instead needs to live in the worker — computed once, or too big to ship each frame — attach hooks to the update function itself. Both are optional properties on the EntityUpdateFunction.

init runs on system.finishLoading() — once at startup (world.init() calls it), and again on each world.load() so a reused world re-seeds its workers. It receives whatever the system's getInitData() returned (structured-cloned across the boundary) and returns a Partial<W> that is merged onto world on every subsequent run. The same load message also carries the system's current query membership, so the worker has every loaded entity cached before finishLoading() resolves. init(world, data, entities, queries) receives those populated lists plus a base world with gameTime, getString, heap and random (no elapsedTime or callbacks: init seeds worker state and must not modify entities); that initial membership never reaches queryChanged, so build any index over the loaded entities in init. Use it for state that must be seeded from the main thread but then persist inside the worker — a lookup table, a config object — without paying to re-send it each frame. Type the init data with the D type parameter (the fourth on EntityUpdateFunction / EntityWorkerSystem) so getInitData and the init hook agree on its shape; it defaults to unknown when a system has no init data:

interface DamageWorld extends EntityWorkerSystemWorld {
	damage: number
}
interface DamageInitData {
	baseDamage: number
}

const damageUpdate: EntityUpdateFunction<Components, { health: Int32Array }, DamageWorld, DamageInitData> =
	(world, entityId, components) => { components.health[0] -= world.damage; };

// Runs once in the worker; its return is merged onto `world` before every run. `data` is typed as
// `DamageInitData | undefined` (undefined when the system supplies no getInitData).
damageUpdate.init = (_world, data) => ({ damage: data?.baseDamage ?? 1 });

class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }, DamageWorld, DamageInitData> {
	constructor(world: BaseWorld<Components>) {
		super(world, {
			name: 'DamageSystem',
			required: ['health'],
			updateFunction: damageUpdate,
			getWorker: () => new Worker(/* your worker entry */),
			// Runs on the main thread; its result is structured-cloned to the worker's `init`.
			getInitData: () => ({ baseDamage: 5 }),
		});
	}
}

Unlike addDataToWorld (an overridable method on the system), getInitData is a config option — it lives in the options passed to the EntityWorkerSystem constructor, so a system used without a subclass can supply it there directly.

preRun runs once per run, before any entity is updated, with the run's world, the full entity list, the query results, and the same callbacks. Use it for setup that spans the whole batch — seeding a spatial index, resetting an accumulator, or emitting a run-level event — that would be wasteful or wrong to repeat inside the per-entity loop:

damageUpdate.preRun = (world, entities, queries, callbacks) => {
	// Runs before the per-entity pass; `entities` is everything this run will touch.
};

queryChanged runs before preRun, once for each sub-query in queries whose membership changed this run, with that query's name and its { added, removed } delta. Use it to keep a worker-local structure in sync with a set of entities that rarely changes - a navigation grid of static walls, a lookup of static targets - instead of rescanning the whole query every run. added can repeat an entity already in the query (its components changed and its blocks were re-sent), so treat it as an upsert. Seed that structure in init from the loaded queries, since entities present at load are never reported to queryChanged; a reload re-runs init, rebuilding it:

wallUpdate.init = (_world, _data, _entities, queries) => ({
	walls: new Map((queries.walls ?? []).map(({ entityId, components }) => [entityId, rectOf(components.transform)])),
});
wallUpdate.queryChanged = (world, queryName, delta) => {
	delta.removed.forEach(entityId => world.walls.delete(entityId));
	delta.added.forEach(({ entityId, components }) => world.walls.set(entityId, rectOf(components.transform)));
};

(An entityRemoved(world, entityId, callbacks) hook completes the set — it fires once per entity that left the system this run, so a worker can release any per-entity state it was holding.)

Per-tick systems: WorkerSystem

Some systems have no entities of their own — they run once per tick to scan or count other entities, and maybe spawn something. A WorkerSystem is an EntityWorkerSystem whose function is called once per run over the named sub-queries instead of once per entity. It has no main query, so it runs every interval even with zero entities, and it reuses the whole worker/query/createsEntities/addDataToWorld machinery:

// once per run: no per-entity loop, just the queries
const spawnerUpdate: WorkerSystemRunFunction<Components, SpawnerWorld> = (world, queries, callbacks) => {
	const asteroids = queries.asteroids ?? [];
	if(asteroids.length < world.minAsteroids) {
		createEntityWorker(world, { type: 'asteroid', x: 0, y: 0 }, callbacks);
	}
};

class SpawnerSystem extends WorkerSystem<Components, SpawnerWorld> {
	minAsteroids = 20;
	constructor(world: TestWorld) {
		super(world, {
			name: 'Spawner',
			updateFunction: spawnerUpdate,
			createsEntities: true,
			deltaBetweenRuns: 1000,
			getWorker: () => new Worker(SPAWNER_WORKER_URL, { type: 'module' }),
			queries: { asteroids: { required: ['asteroid'] } },
		});
	}
	// Fresh per-run data — structured-cloned to the worker each run, so keep it plain/cloneable.
	addDataToWorld(world: SpawnerWorld) { world.minAsteroids = this.minAsteroids; }
}

Its worker entry uses createSystemWorker (the run-function counterpart of createEntitySystemWorker), passing the registry only when it creates entities or adds components:

import { createSystemWorker } from '@daneren2005/shared-memory-ecs/worker';
createSystemWorker(self, spawnerUpdate, registry);

The run function may carry the same init / queryChanged / entityRemoved hooks as an updateFunction; its init(world, data, queries) omits the entity list, since there is no main query. Everything else — callbacks, worker fallback, off-thread creation — behaves exactly as an EntityWorkerSystem.

The worker helpers report unhandled promise rejections on the main thread with the system name, preserving the error message and stack. Rejections before the startup handshake completes reject system.init(); later rejections are logged. Custom worker scopes can provide addEventListener('unhandledrejection', listener) to enable this reporting.

Importing in workers

Each worker entry file is bundled on its own, and a single-entry bundle cannot tree-shake this package's barrel: importing createEntitySystemWorker from @daneren2005/shared-memory-ecs drags the whole library - BaseWorld, every system, their @daneren2005/shared-memory-objects dependencies - into the worker, even though a worker never runs any of it (easily ~20kb of dead code per worker). Import worker-side helpers from the @daneren2005/shared-memory-ecs/worker subpath instead. It exposes only what runs in a worker - createEntitySystemWorker, createSystemWorker, createEntityWorker, addComponentWorker, removeComponentWorker, killEntityWorker, DEAD_INDEX, TYPE_INDEX (plus the worker-relevant types) - so the bundle stays tiny:

// damage.worker.ts - the worker entry file
import { createEntitySystemWorker } from '@daneren2005/shared-memory-ecs/worker';
import { damageUpdate } from './damage-update';

createEntitySystemWorker(self, damageUpdate);

The same applies to any module the worker file pulls in: an update function that calls createEntityWorker or killEntityWorker should import them from /worker too. Type-only imports (EntityUpdateFunction, EntityWorkerSystemWorld, ...) can come from either path since types are erased, and main-thread code (EntityWorkerSystem, BaseWorld, EntityFactory, ...) keeps importing from the package root.

A worker that creates entities or adds components passes your component registry as the third argument - createEntitySystemWorker(self, shipUpdate, registry) - so it has each component's toBlock. Only do this in workers that need it; it pulls the registry (and whatever it imports) into that worker's bundle.

Reading an entity's type in a worker

The entity component carries the entity's type in shared memory as a pointer, so a worker can resolve it back to a string. Pull entity into the system's query (its block only reaches the worker if it is required or optional), read the pointer at TYPE_INDEX, and hand it to world.getString — the framework injects that resolver on world before every run:

import { TYPE_INDEX } from '@daneren2005/shared-memory-ecs/worker';
import type { EntityUpdateFunction } from '@daneren2005/shared-memory-ecs/worker';

const update: EntityUpdateFunction<Components, { entity: Uint32Array }> = (world, entityId, components) => {
	const type = world.getString(components.entity[TYPE_INDEX]); // e.g. "Space Ship"
	// ...branch on type, etc.
};

class TypedSystem extends EntityWorkerSystem<Components, { entity: Uint32Array }> {
	constructor(world: BaseWorld<Components>) {
		super(world, { name: 'TypedSystem', required: ['entity'], updateFunction: update, getWorker: () => new Worker(/* ... */) });
	}
}

world.getString(pointer) is a Map lookup before it ever rebuilds the string from memory, and returns '' for the empty type or a pointer whose buffer has not synced to the worker yet. This works identically on the main-thread fallback. It is not limited to type — any pointer to a ConstantString (via world.constantStrings on the main thread) resolves the same way.

Reporting back to the main thread

An update function runs on shared memory, so anything it writes is already visible on the main thread. What it cannot do from there is touch the world, so the things that have to happen back on it go through callbacks: entityComponentChanged (emitted on the entity as component-property-updated), entityDied (as death), createEntity (see Creating entities from a worker), and component changes requested by addComponentWorker / removeComponentWorker. All of them are collected during the run and applied once it completes.

emitEntityEvent is the escape hatch for an event of your own: name it whatever you like and give it whatever args suit it, and it is emitted on the entity under that name. It exists so a system does not have to spend one component-property-updated per property when the listener only cares about all of them together - a move that reports x and y as one position-updated is half the events of one per axis:

// in the update function
callbacks.emitEntityEvent(entityId, 'position-updated', x, y);

// on the main thread
entity.on('position-updated', (x: number, y: number) => { ... });

The args are structured-cloned across the worker boundary, so they have to be plain values - no functions, no class instances. Nothing about the name or the args is checked against your component map, since the event is the system's own concept rather than a component, so export both alongside the update function that emits them.

Reporting something that happens to everything, every run

Every callback above costs an event object per entity: an allocation in the worker, a structured clone of it, an eid lookup on the main thread, and an emit on that entity. That is fine for a death or a hit, which happen to a handful of entities a run. It is not fine for a move, which happens to nearly all of them, every run - at ten thousand entities that is ten thousand objects and ten thousand emits a frame, and it can easily cost more than the simulation it is reporting.

emitSystemEvent is the version of that with everything avoidable taken out. Name the event and give it an entity id; the whole run arrives on the system as one call with one array of ids:

// in the update function - just the id, once per entity that moved
callbacks.emitSystemEvent('position-updated', entityId);

// on the main thread - one call for the run, however many entities are in it
system.on('position-updated', (entityIds: Array<number>) => {
	for(const eid of entityIds) {
		const sprite = sprites.get(eid);
		// The worker wrote the position into shared memory, so it is already here to read.
		if(sprite) {
			const transform = world.getEntityByEid(eid)?.components.transform;
			...
		}
	}
});

Nothing travels with the id on purpose. The blocks the update just wrote are shared memory, so the main thread already holds the values - sending them along would only pay to copy what is already there. Reach for emitEntityEvent when the thing you want to report is not in a component block, and for this when it is.

System events are dispatched before the per-entity events of the same run, so an entity a run both moved and killed is still in the world when its move is reported.

Creating entities from a worker

An update function can spawn a whole entity, off-thread, from a factory config - the same { type, ...overrides } you would pass to world.loadEntity. Because component pools live in shared memory, the worker merges the type's factory template, allocates each component's block and writes it, then reports what it made; the main thread only wraps the result:

// in the update function - spawn a ship, overriding two fields of its template
createEntityWorker(world, { type: 'ship', x: 100, y: 150 }, callbacks);

createEntityWorker layers your overrides over the ship template, mints a unique id from a shared atomic counter (so it never collides with one the main thread or another worker hands out), and for each component the merged config triggers, pushes a block into its pool and writes the values - all off-thread. With entity classes registered, the worker also resolves the template's class and considers only its allowed components. It reports back an id-plus-block-indexes descriptor; when completion effects are applied the main thread adopts it, building the always-present entity component there (interning type and class is a main-thread job), asking the factory for the registered wrapper, and attaching each block the worker wrote - no block is copied or re-allocated. The new entity joins queries at adoption and is available to subsequent dispatches. The default scheduler adopts immediately on completion; the conflict scheduler can defer adoption until publication is safe.

Two things make this work, both opt-in so only the workers that create entities pay for them:

  • Register the system with createsEntities: true, or a component list for parallel conflict scheduling. Both forms ship the factory templates to its worker on load.

  • In that system's worker entry, pass your component registry to createEntitySystemWorker, so the worker has each component's block builder:

    createEntitySystemWorker(self, shipUpdate, registry);

Every component is already defined as two halves for exactly this — toBlock(config) (the block values) and attach(entity, memory, index) (the accessor), see Defining components. The worker runs only toBlock (off-thread, no entity/world), and the main thread runs attach when it adopts the entity. So creating a component in a worker needs nothing extra as long as its toBlock is genuinely pure over config — no entity/world access:

const shipHealth: ComponentDefinition<HealthComponent, Int32Array, HealthConfig> = {
	// ...
	toBlock(config) {
		return [config.health ?? config.maxHealth, config.maxHealth];
	},
	attach(entity, memory, index) {
		const block = memory.getBlock(index);
		// ...build the accessor over `block`
	},
};

This runs identically on the main-thread fallback. A loadInFinishLoading component (one that reads other entities) is skipped, since the worker has no world to read. A worker does not receive constructors; it returns the serializable class key, and the main-thread factory constructs the wrapper during adoption.

Adding and removing components at runtime

On the main thread, use the entity directly:

ship.loadComponent('shield', { capacity: 100 });
ship.removeComponent('shield');

Both operations notify every system. Required and excluded (not) queries change membership as needed. An entity already present through other required components is re-sent to worker systems too, so their cached component bundle gains or loses an optional component on the next run.

Inside an update function, use the worker helpers instead. They record structural changes without changing the component snapshot currently being iterated:

import { addComponentWorker, removeComponentWorker } from '@daneren2005/shared-memory-ecs/worker';

if(equippedShieldGenerator && !components.shield) {
	addComponentWorker(world, entityId, 'shield', { capacity: 100 }, callbacks);
} else if(!equippedShieldGenerator && components.shield) {
	removeComponentWorker(entityId, 'shield', callbacks);
}

The main thread applies those commands after the worker run completes, emits the normal component-added or component-removed event, and queues query deltas. The originating run therefore keeps a stable snapshot and all systems see the new structure no later than their next run. Removing a component uses the same deferred-free safety described below.

Worker-side addition needs two opt-ins because it allocates a block:

  • Set addsComponents: true on the EntityWorkerSystem, or a list of the components it adds so ConflictScheduler can run it in parallel with unrelated systems.
  • Pass the component registry to that system's createEntitySystemWorker(self, update, registry) call.

addComponentWorker calls the definition's worker-safe toBlock(config) and cannot add the always-present entity component or a loadInFinishLoading component. Worker-side removal needs no allocation opt-in.

Freeing component memory safely

Component blocks live in a shared pool, so a freed block gets handed straight back out to the next entity that needs one. That is a problem across workers: if one system kills an entity, a second reuses its freed block for a brand-new entity, and a third is still mid-run over what it thinks is the old entity, the third system writes into the new entity's memory.

So the library never frees a block the instant it is orphaned. When an entity dies or you call removeComponent, the block is deferred — the component is gone from the entity immediately, but the memory is held until every system that could be mid-run over it has finished a run. Only then is the block returned to the pool for reuse. This is automatic; there is nothing to call. The one visible effect is that memoryComponent.length can briefly sit one higher than the number of live components, until the next update lets the holding systems finish. If a system stays stuck (never completes a run) for more than ten seconds of unscaled time, the world logs a warning naming that system and frees the blocks anyway rather than leak them — a stuck system there is a bug worth chasing down.

Freeing extra resources

Everything above frees a component's own block. A component that allocates something else in attach — another heap structure (a SharedList, a SharedString) or child entities it owns — needs to release that too, and the block-level deferred free won't do it. Give the definition an optional free(component):

const cargoDefinition: ComponentDefinition<Cargo, Uint32Array, CargoConfig> = {
  type: Uint32Array,
  size: 3,
  loadProperties: ['cargoSpace'],
  toBlock(config) {
    /* the block's own values */
  },
  attach(entity, memory, index) {
    /* allocate a SharedList in the heap, stash its pointer in the block, return accessors */
  },
  free(component) {
    component.items.free(); // release the SharedList + the item entities it holds
  },
};

free runs exactly once per component teardown: on removeComponent, and on every path that removes an entity — removeEntity, a death (from killEntity), a load that drops the old batch, and clear. It is deferred to the same safe point as the component's own block (see above), not fired the instant the entity dies — so a system still mid-run over that memory can't see your resource released early. Reload calls it without any death event, so it is the reliable place to avoid leaks when a world is reused.

Reacting to death

To react when an entity dies — spawn drops at its position, credit a kill, play an effect — a component definition can carry an optional died(component, entity, world). It fires exactly once per death, on the main thread, with the dying entity still fully readable (its blocks are only deferred-freed, never torn down before the hook runs), for both a worker kill (killEntityWorker) and a main-thread killEntity:

const oreDepositDefinition: ComponentDefinition<OreDeposit, Int32Array, OreDepositConfig> = {
  type: Int32Array,
  size: 3,
  loadProperties: ['oreAmount'],
  toBlock(config) { /* ... */ },
  attach(entity, memory, index) { /* ... */ },
  died(component, entity, world) {
    // The asteroid is still readable here, so spawn its drop where it was.
    const { x, y } = entity.components.position!;
    world.loadEntity({ type: 'Ore', x, y, amount: component.oreAmount });
  },
};

The hook is engine-agnostic: attach death behavior to the component that represents it. A throwing died can't abort the death — it is caught and surfaced as a system-error event (phase: 'died') while the entity is still removed. Unlike free, died fires only on a real death, never on world.load()/clear() teardown.

For a non-declarative hook, the world also emits world.on('entity-died', entity) at the same point (distinct from entity-removed, which fires for every removal, reloads included).

Reusing a world: load and clear

A world is meant to be reused rather than rebuilt. world.load(config) swaps in a fresh scenario — it removes the old entities, clears every system, and loads the new batch (deferred frees from the old contents drain over the following updates, as above). Loading itself is synchronous, but it returns a promise that resolves once every system's finishLoading() (e.g. a worker re-seeding its init data) completes — await world.load(config) before starting the game loop. await world.clear() instead tears the world all the way back down to an empty, reusable state: it waits for every system's in-flight worker run to finish so nothing is still reading the memory, frees the held blocks immediately, and resolves once the world is ready to load into again.

Prefer world.load() over looping world.loadEntity() for a large batch. It still emits entity-added for every entity, but systems ignore those while world.isBulkLoading is set and instead join the whole batch once at the end through system.addEntities(batch). Entities with the same component keys are grouped, so each query's component test runs once per group instead of once per entity; system membership keeps load order. A system that overrides checkAddEntity (or an EntityWorkerSystem that overrides matchesQuery) still has it called for each entity.

Loading in slices with progress

A big save can block the main thread for seconds. Pass sliceMs and load() yields to the event loop whenever a slice runs over, reporting progress as it goes:

await world.load(save, {
	sliceMs: 25,
	onProgress: ({ phase, done, total, fraction }) => setBar(fraction),
	// Systems hold every entity and their load messages are posted; only worker replies are left.
	onEntitiesReady: () => buildIndexes(world),
});

Phases run entities → linking (finishLoading, entity-added) → systems (addEntities, then finishLoading) → workers (replies). fraction weights them 0.6 / 0.1 / 0.1 / 0.2. yieldTo overrides the yield (default scheduler.yield(), else setTimeout(0)). While world.isLoading is set, update() does nothing; don't touch the world until load() resolves. A load(), clear() or destroy() during a sliced load, or aborting its signal, rejects it with an AbortError, and the next load or clear tears down the partly loaded world. Without sliceMs, load() runs synchronously up to awaiting the workers, and onProgress/onEntitiesReady still fire.

Compressing saves

Big maps are mostly entities whose save is a type plus a few integers. compressEntities(entities, definitions) packs each config that fits a definition into a short ASCII string and leaves every other config as an object:

import { compressEntities } from '@daneren2005/shared-memory-ecs';

const save = compressEntities(entities, [
	{ type: 'grass', properties: [
		// Stored as (value - offset) / step; tile centres become small tile indices.
		{ name: 'x', type: 'uint', step: 32, offset: 16 },
		{ name: 'y', type: 'uint', step: 32, offset: 16 },
		{ name: 'harvestAmount', type: 'uint' },
		{ name: 'harvestProgress', type: 'uint' },
	] },
]);
// { type: 'grass', x: 7920, y: 15440, harvestProgress: 633000 } becomes '#g(q-!ok{)'; a worker stays an object.
// => { entityDefinitions: { 0: { type: 'grass', properties: [...] } }, entities: ['#g(q-!ok{)', { type: 'worker', ... }] }
await world.load({ ...save, gameTime });

world.load() expands string entities with the bundled entityDefinitions, and decompressEntities / decompressEntity do the same for tools. Property types are int (signed), uint, boolean, float32 and string. int and uint decode as stored * step / 10 ** decimals + offset (defaults 1, 0, 0), so decimals: 2 keeps hundredths. float32 takes any number a Float32Array holds exactly, such as positions and velocities read from shared memory. A config stays an object when it has a property its definition lacks, or a value that wouldn't decode back exactly. Only definitions some string uses are bundled, so defining rare types costs nothing until one appears.

Types with the same properties can share one definition by listing types instead of type; the save then stores the property list once:

{ types: ['oak', 'pine', 'birch'], properties: [...] }
``` Missing values decode as absent. The definitions travel inside the save, so a later build can append,
reorder or drop properties without breaking older saves.

The string format lets other tools read saves without this library. A string is the definition key, then one field
per property in definition order. A shared definition saved at key `n` gives its `types` the keys `n`, `n + 1`, ... in
order, so every type still has its own key and sharing adds nothing to the strings. A field is `!` for absent, or a varint: little-endian base-45 digits, where every
digit except the last comes from `` PQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|} `` and the last from
`#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO` (digit = position in its alphabet). Fields after the last present
value are left out. `int` values are zigzag encoded (`n >= 0 ? 2n : -2n - 1`), booleans are 0 or 1, and a `float32` is
the varint of its 32 IEEE 754 bits. A `string` is a varint length in UTF-16 code units followed by the raw characters,
which may be any character; length 0 means `null`, and empty strings stay objects.

### Seeded randomness

`world.random` is a seeded generator stored in the world's shared memory. The main thread and every worker
(`world.random` inside update functions and `init`) draw from the same stream with no lock: each draw atomically
claims the next position and encrypts it with Philox4x32-10 keyed by the seed, so two threads drawing at once
never share a position. Philox is the counter-based generator from Random123 (also used by numpy and cuRAND)
and passes the BigCrush and PractRand test suites.

```ts
world.random.next();     // [0, 1) with 53 bits of precision, like Math.random
world.random.int(1, 7);  // integer in [1, 7)
world.random.uint32();   // raw 32-bit value

Every world.load() seeds it from the config's random before any entity loads:

await world.load({ entities });                          // fresh random seed
await world.load({ entities, random: { seed: 1234 } });  // chosen seed (integer up to 2^53), e.g. a shared map seed

// Saving the state stops save scumming: reloading replays the same upcoming values.
const save = { entities, gameTime: world.gameTime, random: world.random.getState() }; // { seed, position }
await world.load(save);

Leave random out of the save if you don't care; each load then gets a new random seed.

The stream is reproducible, but when several threads draw at the same time, which thread gets which value depends on timing. A save restores what comes next, not which system draws it.

Measuring performance

PerformanceTiming watches a world and reports what running it costs. Hand it the world and it hooks itself up to the events the world already emits - there is nothing to call per frame and nothing added to the hot path:

const timing = new PerformanceTiming(world);
timing.on('stats-updated', (stats: PerformanceStats) => renderDebugPanel(stats));

It gathers samples every frame and collapses them into a fresh timing.stats snapshot once ticksBetweenUpdates (default 1_000) worth of elapsed time has gone by, then emits stats-updated with it. The window is measured in whatever unit you drive world.update with, so a game running on milliseconds gets a snapshot a second. Frames the world was paused for are skipped, since it does no work on them.

Every entry is an { avg, min, max, samples } over the window just closed:

  • stats.update - one whole world.update call on the thread the world lives on.
  • stats.systems[] - per system, in run order: run is the run itself on its worker, as the worker measured it, and events is what handling that run's results (the events it reported onto entities, the entities it asked to be created) cost back on the calling thread. Both worker and main-thread fallback runs report these timings. For workBatchSize, one sample sums all batches in a complete pass: batches taking 3, 5, and 4 ms report a 12 ms run with samples: 1. Event handling is summed across the pass too; gaps between batches are excluded. Failed preparation closes the attempt with the time spent so far.
  • stats.events - every system's event handling added together. Workers finish on their own schedule rather than on a frame boundary, so there is no per-frame combined sample to take; these are the per-system figures summed at the end of the window, giving what a run of every system costs the main thread between them.

To coordinate reporting with an external clock, construct with { autoUpdate: false } and call timing.flush() at each reporting boundary. It publishes and returns a snapshot of all collected update, worker, and event samples, then starts a fresh window. This also works while paused: windows without work report zeroes. A batched pass retains its partial totals across reporting boundaries and contributes one sample when its final batch arrives (event timing completes when its results are handled). Clear/reload drops unfinished passes, and the next pass starts fresh totals. Automatic reporting remains enabled by default.

getSystemStats(name) pulls one system out of the latest snapshot, reset() throws away everything collected so far (worth doing after loading a new scene, when the samples either side are not comparable), and destroy() unhooks it from the world.

Building

npm install
npm run build      # emits dist/ (js + d.ts)
npm run type-check

Workers invoked directly outside scheduler dispatch publish their effects immediately. Replies from before a scheduler reset are discarded.