@daneren2005/shared-memory-ecs
v1.11.0
Published
A small, reusable Entity/Component/System core backed by shared memory.
Readme
@daneren2005/shared-memory-ecs
A small, reusable Entity / Component / System core backed by shared memory
(@daneren2005/shared-memory-objects).
It contains only engine-agnostic ECS concepts. Games supply their own components, entity classes, templates,
systems, and domain data.
Concepts
- Component – a plain object with an
index(its block inside a shared-memory pool) plus getters/ setters over that memory. Games decide what components exist. ComponentDefinition– describes a component: its typed arraytype, blocksize, the config keys that trigger loading (loadProperties), and the two halves that build it —toBlock(config)(maps the config to the raw block values) andattach(entity, memory, index)(builds the accessor over that block). Loading a component isattach(entity, memory, memory.create(toBlock(config))); splitting it this way lets a worker build the block off-thread (it calls onlytoBlock) and the main thread wrap it (attach) when it adopts the entity.attachalso receives the load config as an optional fourth argument, for main-thread-only state kept off the block (e.g. a display name); it isundefinedwhen adopting a worker-built block. Plus an optionalsave(component),free(component), anddied(component, entity, world). A component's data splits intoConfig(defining props supplied up front, e.g.maxHealth) andSerialization(runtime-derived state, e.g. currenthealth);toBlockseesConfig & Serializationwhilesavereturns only theSerializationslice.freeruns when the component is torn down (see Freeing extra resources);diedruns when the entity is killed, with it still readable (see Reacting to death).ComponentRegistry<C>– the map of all component definitions for a game.EntityFactory<C>– maps an entitytypename to a base (template) config. Loading an entity layers the caller's config over its type's template, so shared static data lives in one place and a save only needs thetypeplus the entity's serialization. Optionally, a template's immutableclassselects a game-providedBaseEntitysubclass and restricts which components that template may load.BaseWorld<C>– builds oneMemoryComponentper registered component (attached to that component's definition asworld.registry[name].memoryComponent) and runs systems. It is generic over your component mapC, soworld.registry,entity.components,setComponent, etc. are fully typed. Its entities live inworld.entities, aMapkeyed byeid— see Iterating entities.BaseEntity<C>– aneid, itsidalias, immutabletype,deadstate, and a bag of memory-backed components. Every entity has anentitycomponent whose immutabletypeand optional immutableclassare interned strings stored in shared memory. A game can subclassBaseEntityto expose direct convenience getters for one entity class.- Systems –
System,IterableSystem,EntitySystem(main-thread iteration over entities with a given set of components),EntityWorkerSystem(runs an update function over raw memory blocks, off-thread when Web Workers +SharedArrayBufferare available), andWorkerSystem(anEntityWorkerSystemthat runs once per tick over its sub-queries instead of once per entity — see Per-tick systems).
Defining components
import { Component } from '@daneren2005/shared-memory-ecs';
import type { ComponentDefinition } from '@daneren2005/shared-memory-ecs';
interface HealthComponent {
index: number
health: number
maxHealth: number
}
const HEALTH_INDEX = 0;
const HEALTH_MAX_INDEX = 1;
// The accessor is a subclass of `Component`, so its get/set live on one shared prototype: reading a component off
// thousands of entities stays monomorphic and inline, and each instance is a single allocation rather than a
// closure per accessor. Index `this.block` (the typed-array view, set for you) by the exported *_INDEX constants.
class Health extends Component<Int32Array> implements HealthComponent {
get health() { return this.block[HEALTH_INDEX]; }
set health(value: number) { this.block[HEALTH_INDEX] = value; }
get maxHealth() { return this.block[HEALTH_MAX_INDEX]; }
}
// `maxHealth` is the defining Config prop; `health` is optional, runtime-derived Serialization. `toBlock`
// and `save` both see the combined `{ maxHealth: number, health?: number }`.
const healthDefinition: ComponentDefinition<HealthComponent, Int32Array, { maxHealth: number }, { health?: number }> = {
type: Int32Array,
size: 2,
// Configs are flat and shared: this component loads whenever `maxHealth` is present, then reads what it
// needs off the same config object. Only Config props may appear here.
loadProperties: ['maxHealth'],
// The block values, purely from config — so a worker can build the block off-thread. No entity/world access.
toBlock(config) {
return [config.health ?? config.maxHealth, config.maxHealth];
},
// The accessor over an already-allocated block. Runs on the main thread both when loading and when adopting a
// worker-built entity; a component's own extra allocations (a SharedList, a resource block) go here — a
// subclass that owns extra memory takes more constructor args (the entity, another pool) and stores them.
attach(entity, memoryComponent, index) {
return new Health(memoryComponent.getBlock(index), index);
},
save(component) {
// save returns only Serialization; maxHealth is Config and comes back from the template on reload.
return { health: component.health };
}
};
// One place defines the whole registry + the component map type it implies.
const registry = {
health: healthDefinition
// ...other component definitions
};
type Components = { [K in keyof typeof registry]: ReturnType<(typeof registry)[K]['attach']> };Using the world
import { BaseWorld } from '@daneren2005/shared-memory-ecs';
const world = new BaseWorld<Components>(registry);
const entity = world.loadEntity({ maxHealth: 100 });
entity.components.health!.health -= 10;
console.log(entity.save()); // { health: 90 }Entity types via the factory
Register per-type base configs, then load entities by type. The template supplies the defining config, so
only the type and runtime serialization need to be saved:
import { BaseWorld, EntityFactory } from '@daneren2005/shared-memory-ecs';
const factory = new EntityFactory<Components>({
goblin: { maxHealth: 20 },
});
const world = new BaseWorld<Components>(registry, { factory });
const goblin = world.loadEntity({ type: 'goblin', health: 10 });
console.log(goblin.components.health!.maxHealth); // 20 (from the template)
console.log(goblin.save()); // { type: 'goblin', health: 10 } - no templated maxHealth
// A save re-expands back through the factory:
world.loadEntity(goblin.save());An entity's type lives in shared memory (worker-visible), not as a plain field. Each distinct type string is
interned once as an immutable ConstantString in the heap — 15 "Space Ship" entities share one allocation —
and the entity block stores only a pointer to it at TYPE_INDEX. Reading entity.components.entity.type
resolves that pointer back to the string through the world's cache; a worker can do the same (see
Reading an entity's type in a worker).
Entity classes and scoped templates
An entity type names one concrete template. Its optional class names the stable family that chooses a
wrapper constructor and the components that family permits. Callers still pass only { type, ...overrides }:
the factory resolves class from the template, and neither identity can change after construction.
import {
BaseEntity,
BaseWorld,
EntityFactory,
defineEntityClasses,
defineEntityConfigs,
} from '@daneren2005/shared-memory-ecs';
import type { ComponentsOf, EntityConfigOf, EntityInstancesOf } from '@daneren2005/shared-memory-ecs';
const registry = {
health: healthDefinition,
value: valueDefinition,
};
type Components = ComponentsOf<typeof registry>;
type Config = EntityConfigOf<typeof registry>;
class ActorEntity extends BaseEntity<Components, Config> {
get health() { return this.components.health?.health; }
}
class ItemEntity extends BaseEntity<Components, Config> {
get value() { return this.components.value?.value; }
}
const classes = defineEntityClasses(registry, {
actor: { entity: ActorEntity, components: ['health'] },
item: { entity: ItemEntity, components: ['value'] },
} as const);
const templates = defineEntityConfigs(classes, {
guardian: { type: 'guardian', class: 'actor', maxHealth: 100 },
token: { type: 'token', class: 'item', value: 5 },
} as const);
type Entities = EntityInstancesOf<typeof classes>;
const factory = new EntityFactory<Components, Config, Entities>(templates, classes);
const world = new BaseWorld<typeof registry, Components, Config, Entities>(registry, { factory });
const item = world.loadEntity({ type: 'token' });
console.log(item instanceof ItemEntity); // true
console.log(item.components.entity.class); // "item"
console.log(item.id, item.type, item.dead); // BaseEntity identity accessorsdefineEntityConfigs checks each template against its class's component definitions, so adding a health-only
property to the item template is a type error. The factory repeats the boundary at runtime: loading or attaching
a component outside the class allowlist throws, protecting dynamically loaded data too. Client/editor metadata
that no component consumes should remain in a separate authoring type and be transformed into the scoped ECS
template.
Classless factories keep the original behavior: they create BaseEntity and scan the complete registry.
System scheduling
world.update(elapsedTime) uses DefaultScheduler to dispatch systems in registration order.
Synchronous systems run immediately, workers can overlap after dispatch, and iterable systems can
continue across updates. Systems keep their existing shouldRun(), deltaBetweenRuns, and firstRun
behavior. The world advances clocks, applies timeScale, skips new updates while paused, prepares shared
memory, and manages deferred component frees.
Select a scheduler at construction; world.scheduler is readonly:
import { BaseWorld, DefaultScheduler } from '@daneren2005/shared-memory-ecs';
const world = new BaseWorld(registry, {
scheduler: new DefaultScheduler<Components>()
});
world.update(16); // Synchronous result: { lastSystemError: null } when no dispatch error occurred.Custom policies implement Scheduler<C>. Its synchronous update(context, elapsedTime) receives the
scaled delta and a SchedulerContext<C> exposing the live systems array, the world's emit method,
and optional gameTime (supplied by BaseWorld).
It returns SchedulerUpdateResult ({ lastSystemError?: Error | null }). DefaultScheduler preserves
the existing dispatch timing events, error reporting, and continuation after a system throws.
Optional systemAdded(context, system) and systemRemoved(context, system) hooks receive notifications
after the array changes and before the corresponding world event. Duplicate names remain supported;
addSystemIfNotExists checks names and removeSystem removes the first matching name. Direct assignment
or mutation of world.systems bypasses membership hooks, so policies must read the live array when needed.
The default policy preserves native forEach mutation behavior during dispatch.
Optional reset(context) runs after systems are cleared during reload or non-pristine clear();
pristine worlds skip that teardown. destroy(context) runs once after system destruction.
runCompleted(context, system) receives the existing completion boundary: a synchronous run returned,
an iterable queue drained, or a current-generation worker reply's effects were applied. Fallback
completion can arrive inside scheduler dispatch. The world processes eligible deferred frees before
calling this hook. Dispatch return values and system-<name>-finished events do not imply completion.
These hooks expose lifecycle notifications; they do not establish deterministic ticks or safe edit barriers.
Use ConflictScheduler to prevent systems from overlapping when one writes a component the other reads
or writes. Conflicts follow registration order; read-only systems and systems accessing independent
components can overlap:
import { BaseWorld, ConflictScheduler, EntityWorkerSystem } from '@daneren2005/shared-memory-ecs';
const world = new BaseWorld(registry, {
scheduler: new ConflictScheduler<Components>()
});
class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }> {
constructor() {
super(world, {
name: 'Damage',
required: ['health'],
writes: ['health'],
updateFunction: damageUpdate,
getWorker: () => new Worker(new URL('./damage-worker.ts', import.meta.url), { type: 'module' })
});
}
}
world.addSystem(new DamageSystem());The scheduler compiles its schedule when systems are added or removed. Inspect it before the first update using the scheduler instance:
const scheduler = new ConflictScheduler<Components>();
const world = new BaseWorld(registry, { scheduler });
// Register your systems here.
console.log(scheduler.schedule.order.map(system => system.name));
console.table(scheduler.schedule.batches.map((batch, index) => ({
batch: index,
systems: batch.map(system => system.name).join(', ')
})));schedule is a frozen ConflictSchedule<C> containing system references. order flattens the
compiled dependency levels, and batches group systems that can run concurrently within one update.
Every conflict points from the earlier registered system to the later one. Each system goes into the
earliest level after its conflicting predecessors. These groups describe dependencies, not runtime
barriers: a system starts when its own predecessors finish, even if unrelated systems in an earlier
group are still running. Actual dispatch can also overlap independent work from later host updates.
writes lists every component a system changes, including components reached outside its main query.
Use writes: [] for a read-only system. Reads are inferred from EntitySystem.components and from worker
required, optional, not, and all named queries. The built-in entity component is also a read.
Exclusions count because adding or removing an excluded component changes query membership. Add
reads: ['otherComponent'] for access outside those queries, including filters and hooks. A plain
System or IterableSystem supplies both reads and writes; an EntitySystem without a component
query supplies reads explicitly. Component names are checked against the system's component type.
Omitting writes, or leaving reads unknown, makes a system exclusive. exclusive: true also reserves
exclusive execution for a system whose callbacks, structural changes, or shared resources cannot be
described through component access. Workers with createsEntities: true or addsComponents: true remain
exclusive. A component list such as addsComponents: ['onFire'] instead schedules each listed
component as a write, so the system is ordered against readers, writers, and exclusions of what it adds.
List every component the system can add through addComponentWorker. Declare writes to entity when changing its flags; arbitrary callback effects still need
explicit declarations or exclusive execution.
For parallel worker creation, EntityWorkerSystem and WorkerSystem accept a component list in the
existing capability option, for example createsEntities: ['health', 'movement']. List every game
component the system can create, including components supplied by factory templates.
configComponents(world, { type: 'rock', x: 0, y: 0 }) returns the components a creation config loads after
merging its template; only key presence matters, so placeholder values are enough. The built-in
entity component is included automatically. writes still declares changes to existing components;
creation declarations do not authorize undeclared callback or attachment effects on existing data.
Both boolean and list forms enable the same worker allocator, factory templates, and stable atomic eids.
The default scheduler treats both forms identically and adopts results immediately on completion.
Under ConflictScheduler, compatible creators can allocate and initialize blocks concurrently, even
in the same component pool. Publication stays in scheduling order and waits for earlier readers and
writers of the affected components; later readers and writers wait until publication is applied.
A creator can therefore compute beside an earlier reader without changing that reader's query snapshot.
A later query waits for publication when the creator's component list can supply every required
component of that query. For example, creating ['transform', 'item'] does not block a query requiring
['transform', 'tree'], even when it optionally reads item or excludes it with not. Main and named
queries are checked separately; a match with any one retains the dependency. An empty or entity-only
query still observes every creation. Explicit reads outside queries and all writes retain their
component conflicts, so accessing world.entities directly must still be declared.
Creation lists describe all components a system can create, not one exact entity shape. Optional
components and exclusions therefore cannot prove a query unrelated when its required components could
all be created. Custom query filters need explicit reads for accesses outside their declared
components. Custom membership overrides retain conservative component-level creation access unless
they provide accurate creationReadQueries metadata. This optional system getter exposes separate
required-component lists to the conflict scheduler; undefined uses creationReadComponents as before.
Queued updates keep their original query and creation snapshots across schedule rebuilds.
Worker completion effects are kept together, preserving component-change, event, death, and adoption
order within each run. Attachment hooks and callbacks affecting other shared data still need truthful
writes or exclusive: true. Completion promises and memory-free notifications settle only after
application; cancellation discards unpublished blocks. These rules apply only to the conflict scheduler.
Each host update queues one update per registered system with its scaled delta and logical game time. Updates of the same system stay sequential. Read/write and write/write conflicts preserve host-update order, then registration order within an update, including dependencies on work that has not started yet. Read/read access across different systems can overlap. Independent systems can advance into later updates while a slow worker is unfinished; there is no whole-update or whole-batch completion barrier. Exclusive work remains a barrier to all earlier and later work, including workers using the boolean creation capability.
When asynchronous systems finish applying effects, a microtask dispatches newly ready work after
their completion promises settle, without waiting for another animation frame. Deltas stay in each
system's order without merging or dropping them; deltaBetweenRuns, remainders and firstRun retain
their existing behavior. While unpaused, sliced systems continue at most once per host update with
zero extra delta, and release their dependents when the slice queue drains. Conflicting work can still
build a backlog.
During scheduled shouldRun() and update() calls, System.gameTime reports the logical time of the
queued update. Worker world.gameTime uses that captured value, on real workers and the synchronous
fallback alike. Delayed dispatch therefore does not borrow a later host update's clock. Outside that
scope, System.gameTime returns the live world clock. Custom schedulers can use
system.withGameTime(time, () => system.update(delta)) to establish the same scope. Plain systems and
addDataToWorld() implementations should use this.gameTime when they need scheduled time; direct
reads of this.world.gameTime still see the host clock. The default scheduler continues using its
existing update(delta) path and live clock.
The compiled schedule is reused until membership changes or scheduler.rebuild(world) is called.
Call rebuild after changing access declarations, or to inspect direct edits to world.systems before
the next update; direct membership edits are otherwise checked before each dispatch. Queued updates
retain their original access reservations, even if metadata is rebuilt while a worker is running.
Additions join future host updates; removed pending updates are skipped when their predecessors finish,
and removed running systems retain their reservations until completion. With ConflictScheduler,
world.pause() or world.paused = true stops enqueueing new updates but finishes draining existing
work without further world.update() calls. Queued systems retain their captured deltas and logical
times; sliced runs continue through timer tasks with zero extra delta, yielding between slices.
Worker effects publish as their dependencies finish. Draining does not advance either world clock;
calling world.update() while paused still advances only playerTime. world.resume() allows new
work to be queued on the next host update. Reload/clear and destroy discard queued updates and
cancel scheduled microtasks and slice timers. The world prepares shared memory before deferred dispatch and prevents
new dispatch while clearing or destroyed. Errors from automatic dispatch emit system-error; they
cannot change a result already returned by world.update().
Custom asynchronous systems must report isCurrentlyRunning() and notify completion through
onRunFinished() when their work and effects finish. Schedulers may implement the optional
prepareEffects(context, system) hook: it captures the dispatched run and returns a handler receiving
SchedulerEffects with apply() and discard(). The handler must eventually call one of them.
beforeClear(context) lets such schedulers discard pending effects before the world awaits workers.
The optional pause(context) hook is called when the world's paused flag becomes true, including
direct assignment. context.paused indicates whether new host updates are suspended;
context.dispatchAllowed and prepareForSystemDispatch() gate deferred dispatch during teardown.
Schedulers without these hooks retain immediate application and the existing clear path.
This scheduler provides component conflict ordering with the existing synchronous update API. It does not implement fixed simulation ticks or impose a stable order on unrelated workers' callbacks. World clocks continue advancing with host updates. Iterable slicing uses wall-clock budgets; entity workers can opt into deterministic work batches instead. For a fixed input-update sequence, declared conflicting operations and each system's deltas/logical times remain ordered independently of worker completion order. Independent callbacks and timing events can arrive in different orders; shared effects must be declared as conflicts or marked exclusive. Repeatable simulations also need deterministic inputs and system code, and safe external world edits.
Iterating entities
world.entities is a Map keyed by eid, not an array, and so is entities on EntitySystem and
EntityWorkerSystem:
world.entities.forEach(entity => { ... }); // in the order they were added
world.entities.get(eid); // same as world.getEntityByEid(eid)
world.entities.size;
for(const entity of world.entities.values()) { ... }
// For the array methods a Map does not have:
Array.from(world.entities.values()).filter(entity => !!entity.components.health);An entity leaves from anywhere in the middle of that collection — every death is one — and finding it in an
array meant a scan of the whole world, then a shift of everything after it, repeated for each system that
held it as well. A few thousand entities with a few dozen deaths a run made that the most expensive thing the
main thread did on a busy frame; a Map deletes in constant time. It also replaced the separate
entitiesByEid lookup, so there is only ever one collection to keep straight.
Iteration order is still insertion order, so load / save round-trip in the order they always did.
Reading components on a hot path
entity.components.health!.health is the way to read a component, and for almost everything it is the right
one: it is typed, it is readable, and one read costs nothing worth measuring.
It is not free, though, and the cost shows up in exactly one situation — reading the same component off
thousands of entities, every frame. The accessor walks several objects to get there and ends in a getter
call over the shared block. Component-subclass accessors are prototype getters (all instances of a type share
one hidden class, so those reads stay monomorphic and inline — unlike the old per-entity closures, which went
megamorphic), but a getter call plus the property walk still loses to a raw indexed read in the very tightest
per-frame loops.
Where that matters, hold the block instead. It is the same memory the accessors read, so nothing changes about what you get, and it is what the update functions already work on:
import { TRANSFORM_X_INDEX } from '@daneren2005/shared-memory-physics';
// Resolve once, when whatever is doing the reading is set up. The component caches its view on `.block` when it
// is attached, so read that rather than paying for another getBlock (each call allocates a fresh subarray).
const block = entity.components.health!.block as Int32Array;
// Then per frame, per entity:
block[HEALTH_INDEX];Two things come with that. The block is only valid while the component is: re-read .block if the component
can be removed and re-added, or hang it off something that dies with the entity. And it is indexed rather than
named, so the offsets have to be exported alongside the definition — which is why every component in the
physics library exports its *_INDEX constants.
Reach for this when a profile says to, not by default. A menu, a save, a system that touches a dozen entities: use the accessors.
EntityWorkerSystem workers
EntityWorkerSystem needs a getWorker() that returns a real Worker, and an updateFunction. Your
worker entry file calls createEntitySystemWorker(self, updateFunction), importing it from the
/worker subpath so the worker bundle stays small. When Web Workers or
SharedArrayBuffer are unavailable it transparently falls back to running the same update function on
the main thread. Attach any extra per-run data (the equivalent of the old faction/fog-of-war fields)
by overriding addDataToWorld(world). Declare its shape with the W type parameter (an interface
extending EntityWorkerSystemWorld) so both addDataToWorld and the updateFunction see it typed:
interface DamageWorld extends EntityWorkerSystemWorld {
damage: number
}
const damageUpdate: EntityUpdateFunction<Components, { health: Int32Array }, DamageWorld> =
(world, entityId, components) => { components.health[0] -= world.damage; };
class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }, DamageWorld> {
addDataToWorld(world: DamageWorld) { world.damage = 5; }
}Incremental entity work
Set workBatchSize: 1024 in an EntityWorkerSystem configuration to process at most
1,024 snapshot positions per scheduled update. The size must be a positive safe integer.
Batch boundaries depend on entity count and insertion order, never elapsed wall time.
The last batch can be smaller, and removed entities consume their original positions without
being updated. New members wait for the next pass; current component bundles and named queries
are refreshed before every batch.
deltaBetweenRuns starts each full pass. All its batches share the original elapsedTime,
logical gameTime, and data from addDataToWorld(). Delta accumulated during the pass is
carried toward the next pass. preRun receives the full current entity list once at the
start, while optional preBatch receives only the selected entities before each batch.
Worker-local fields assigned to world survive until the pass ends. queryChanged and
entityRemoved still receive each batch's membership changes.
Each batch publishes callbacks, creations and component changes independently. Under
ConflictScheduler, its applied completion releases conflicting systems, allowing physics
or other frequent work between batches. Access declarations must cover both hooks and
entity updates. This opt-in permits other systems to change shared state between batches;
use preBatch to refresh live indexes and avoid retaining component block references across
batch boundaries. A failed preparation hook abandons the remaining pass. Clear/reload drops
the pending snapshot and worker-local pass state.
This also works with DefaultScheduler and the main-thread fallback. It does not apply to
WorkerSystem, whose function runs once over named queries. Full-query preparation and
query-delta handling remain unbounded, so a count limit alone cannot guarantee a frame deadline.
Paused conflict scheduling drains queued batches; unqueued pass work resumes with future
host updates. waitForRunToComplete() waits for the active batch and its publication.
Update-function hooks: init, queryChanged, preRun and preBatch
addDataToWorld runs on the main thread and re-sends its data every run. When the data instead needs to
live in the worker — computed once, or too big to ship each frame — attach hooks to the update function
itself. Both are optional properties on the EntityUpdateFunction.
init runs on system.finishLoading() — once at startup (world.init() calls it), and again on each
world.load() so a reused world re-seeds its workers. It receives whatever the system's getInitData() returned
(structured-cloned across the boundary) and returns a Partial<W> that is merged onto world on every
subsequent run. The same load message also carries the system's current query membership, so the worker has
every loaded entity cached before finishLoading() resolves. init(world, data, entities, queries) receives those
populated lists plus a base world with gameTime, getString, heap and random (no elapsedTime or callbacks: init
seeds worker state and must not modify entities); that initial membership never reaches queryChanged, so build any index over the loaded entities
in init. Use it for state that must be seeded from the main thread but then persist inside the worker
— a lookup table, a config object — without paying to re-send it each frame. Type the init
data with the D type parameter (the fourth on EntityUpdateFunction / EntityWorkerSystem) so getInitData
and the init hook agree on its shape; it defaults to unknown when a system has no init data:
interface DamageWorld extends EntityWorkerSystemWorld {
damage: number
}
interface DamageInitData {
baseDamage: number
}
const damageUpdate: EntityUpdateFunction<Components, { health: Int32Array }, DamageWorld, DamageInitData> =
(world, entityId, components) => { components.health[0] -= world.damage; };
// Runs once in the worker; its return is merged onto `world` before every run. `data` is typed as
// `DamageInitData | undefined` (undefined when the system supplies no getInitData).
damageUpdate.init = (_world, data) => ({ damage: data?.baseDamage ?? 1 });
class DamageSystem extends EntityWorkerSystem<Components, { health: Int32Array }, DamageWorld, DamageInitData> {
constructor(world: BaseWorld<Components>) {
super(world, {
name: 'DamageSystem',
required: ['health'],
updateFunction: damageUpdate,
getWorker: () => new Worker(/* your worker entry */),
// Runs on the main thread; its result is structured-cloned to the worker's `init`.
getInitData: () => ({ baseDamage: 5 }),
});
}
}Unlike addDataToWorld (an overridable method on the system), getInitData is a config option — it lives in
the options passed to the EntityWorkerSystem constructor, so a system used without a subclass can supply it
there directly.
preRun runs once per run, before any entity is updated, with the run's world, the full entity list,
the query results, and the same callbacks. Use it for setup that spans the whole batch — seeding a
spatial index, resetting an accumulator, or emitting a run-level event — that would be wasteful or wrong to
repeat inside the per-entity loop:
damageUpdate.preRun = (world, entities, queries, callbacks) => {
// Runs before the per-entity pass; `entities` is everything this run will touch.
};queryChanged runs before preRun, once for each sub-query in queries whose membership changed this run,
with that query's name and its { added, removed } delta. Use it to keep a worker-local structure in sync with a
set of entities that rarely changes - a navigation grid of static walls, a lookup of static targets - instead of
rescanning the whole query every run. added can repeat an entity already in the query (its components changed
and its blocks were re-sent), so treat it as an upsert. Seed that structure in init from the loaded queries,
since entities present at load are never reported to queryChanged; a reload re-runs init, rebuilding it:
wallUpdate.init = (_world, _data, _entities, queries) => ({
walls: new Map((queries.walls ?? []).map(({ entityId, components }) => [entityId, rectOf(components.transform)])),
});
wallUpdate.queryChanged = (world, queryName, delta) => {
delta.removed.forEach(entityId => world.walls.delete(entityId));
delta.added.forEach(({ entityId, components }) => world.walls.set(entityId, rectOf(components.transform)));
};(An entityRemoved(world, entityId, callbacks) hook completes the set — it fires once per entity that left
the system this run, so a worker can release any per-entity state it was holding.)
Per-tick systems: WorkerSystem
Some systems have no entities of their own — they run once per tick to scan or count other entities, and
maybe spawn something. A WorkerSystem is an EntityWorkerSystem whose function is called once per run over
the named sub-queries instead of once per entity. It has no main query, so it runs every interval even with
zero entities, and it reuses the whole worker/query/createsEntities/addDataToWorld machinery:
// once per run: no per-entity loop, just the queries
const spawnerUpdate: WorkerSystemRunFunction<Components, SpawnerWorld> = (world, queries, callbacks) => {
const asteroids = queries.asteroids ?? [];
if(asteroids.length < world.minAsteroids) {
createEntityWorker(world, { type: 'asteroid', x: 0, y: 0 }, callbacks);
}
};
class SpawnerSystem extends WorkerSystem<Components, SpawnerWorld> {
minAsteroids = 20;
constructor(world: TestWorld) {
super(world, {
name: 'Spawner',
updateFunction: spawnerUpdate,
createsEntities: true,
deltaBetweenRuns: 1000,
getWorker: () => new Worker(SPAWNER_WORKER_URL, { type: 'module' }),
queries: { asteroids: { required: ['asteroid'] } },
});
}
// Fresh per-run data — structured-cloned to the worker each run, so keep it plain/cloneable.
addDataToWorld(world: SpawnerWorld) { world.minAsteroids = this.minAsteroids; }
}Its worker entry uses createSystemWorker (the run-function counterpart of createEntitySystemWorker),
passing the registry only when it creates entities or adds components:
import { createSystemWorker } from '@daneren2005/shared-memory-ecs/worker';
createSystemWorker(self, spawnerUpdate, registry);The run function may carry the same init / queryChanged / entityRemoved hooks as an updateFunction; its
init(world, data, queries) omits the entity list, since there is no main query. Everything else —
callbacks, worker fallback, off-thread creation — behaves exactly as an EntityWorkerSystem.
The worker helpers report unhandled promise rejections on the main thread with the system name,
preserving the error message and stack. Rejections before the startup handshake completes reject
system.init(); later rejections are logged. Custom worker scopes can provide
addEventListener('unhandledrejection', listener) to enable this reporting.
Importing in workers
Each worker entry file is bundled on its own, and a single-entry bundle cannot tree-shake this package's
barrel: importing createEntitySystemWorker from @daneren2005/shared-memory-ecs drags the whole library -
BaseWorld, every system, their @daneren2005/shared-memory-objects dependencies - into the worker, even
though a worker never runs any of it (easily ~20kb of dead code per worker). Import worker-side helpers from
the @daneren2005/shared-memory-ecs/worker subpath instead. It exposes only what runs in a worker -
createEntitySystemWorker, createSystemWorker, createEntityWorker, addComponentWorker,
removeComponentWorker, killEntityWorker, DEAD_INDEX,
TYPE_INDEX (plus the worker-relevant types) - so the bundle stays tiny:
// damage.worker.ts - the worker entry file
import { createEntitySystemWorker } from '@daneren2005/shared-memory-ecs/worker';
import { damageUpdate } from './damage-update';
createEntitySystemWorker(self, damageUpdate);The same applies to any module the worker file pulls in: an update function that calls createEntityWorker
or killEntityWorker should import them from /worker too. Type-only imports (EntityUpdateFunction,
EntityWorkerSystemWorld, ...) can come from either path since types are erased, and main-thread code
(EntityWorkerSystem, BaseWorld, EntityFactory, ...) keeps importing from the package root.
A worker that creates entities or adds components passes your component registry as the third argument -
createEntitySystemWorker(self, shipUpdate, registry) - so it has each component's toBlock. Only do this in
workers that need it; it pulls the registry (and whatever it imports) into that worker's bundle.
Reading an entity's type in a worker
The entity component carries the entity's type in shared memory as a pointer, so a worker can resolve it
back to a string. Pull entity into the system's query (its block only reaches the worker if it is required
or optional), read the pointer at TYPE_INDEX, and hand it to world.getString — the framework injects that
resolver on world before every run:
import { TYPE_INDEX } from '@daneren2005/shared-memory-ecs/worker';
import type { EntityUpdateFunction } from '@daneren2005/shared-memory-ecs/worker';
const update: EntityUpdateFunction<Components, { entity: Uint32Array }> = (world, entityId, components) => {
const type = world.getString(components.entity[TYPE_INDEX]); // e.g. "Space Ship"
// ...branch on type, etc.
};
class TypedSystem extends EntityWorkerSystem<Components, { entity: Uint32Array }> {
constructor(world: BaseWorld<Components>) {
super(world, { name: 'TypedSystem', required: ['entity'], updateFunction: update, getWorker: () => new Worker(/* ... */) });
}
}world.getString(pointer) is a Map lookup before it ever rebuilds the string from memory, and returns '' for
the empty type or a pointer whose buffer has not synced to the worker yet. This works identically on the
main-thread fallback. It is not limited to type — any pointer to a ConstantString (via world.constantStrings
on the main thread) resolves the same way.
Reporting back to the main thread
An update function runs on shared memory, so anything it writes is already visible on the main thread. What
it cannot do from there is touch the world, so the things that have to happen back on it go through
callbacks: entityComponentChanged (emitted on the entity as component-property-updated), entityDied
(as death), createEntity (see Creating entities from a worker), and
component changes requested by addComponentWorker / removeComponentWorker.
All of them are collected during the run and applied once it completes.
emitEntityEvent is the escape hatch for an event of your own: name it whatever you like and give it
whatever args suit it, and it is emitted on the entity under that name. It exists so a system does not have
to spend one component-property-updated per property when the listener only cares about all of them
together - a move that reports x and y as one position-updated is half the events of one per axis:
// in the update function
callbacks.emitEntityEvent(entityId, 'position-updated', x, y);
// on the main thread
entity.on('position-updated', (x: number, y: number) => { ... });The args are structured-cloned across the worker boundary, so they have to be plain values - no functions, no class instances. Nothing about the name or the args is checked against your component map, since the event is the system's own concept rather than a component, so export both alongside the update function that emits them.
Reporting something that happens to everything, every run
Every callback above costs an event object per entity: an allocation in the worker, a structured clone of it, an eid lookup on the main thread, and an emit on that entity. That is fine for a death or a hit, which happen to a handful of entities a run. It is not fine for a move, which happens to nearly all of them, every run - at ten thousand entities that is ten thousand objects and ten thousand emits a frame, and it can easily cost more than the simulation it is reporting.
emitSystemEvent is the version of that with everything avoidable taken out. Name the event and give it an
entity id; the whole run arrives on the system as one call with one array of ids:
// in the update function - just the id, once per entity that moved
callbacks.emitSystemEvent('position-updated', entityId);
// on the main thread - one call for the run, however many entities are in it
system.on('position-updated', (entityIds: Array<number>) => {
for(const eid of entityIds) {
const sprite = sprites.get(eid);
// The worker wrote the position into shared memory, so it is already here to read.
if(sprite) {
const transform = world.getEntityByEid(eid)?.components.transform;
...
}
}
});Nothing travels with the id on purpose. The blocks the update just wrote are shared memory, so the main
thread already holds the values - sending them along would only pay to copy what is already there. Reach for
emitEntityEvent when the thing you want to report is not in a component block, and for this when it is.
System events are dispatched before the per-entity events of the same run, so an entity a run both moved and killed is still in the world when its move is reported.
Creating entities from a worker
An update function can spawn a whole entity, off-thread, from a factory config - the same
{ type, ...overrides } you would pass to world.loadEntity. Because component pools live in shared memory,
the worker merges the type's factory template, allocates each component's block and writes it, then reports
what it made; the main thread only wraps the result:
// in the update function - spawn a ship, overriding two fields of its template
createEntityWorker(world, { type: 'ship', x: 100, y: 150 }, callbacks);createEntityWorker layers your overrides over the ship template, mints a unique id from a shared atomic
counter (so it never collides with one the main thread or another worker hands out), and for each component
the merged config triggers, pushes a block into its pool and writes the values - all off-thread. With entity
classes registered, the worker also resolves the template's class and considers only its allowed components. It reports
back an id-plus-block-indexes descriptor; when completion effects are applied the main thread adopts it, building the
always-present entity component there (interning type and class is a main-thread job), asking the factory
for the registered wrapper, and attaching each block the worker wrote - no block is copied or re-allocated. The new entity joins queries at adoption and is available to subsequent dispatches. The default scheduler
adopts immediately on completion; the conflict scheduler can defer adoption until publication is safe.
Two things make this work, both opt-in so only the workers that create entities pay for them:
Register the system with
createsEntities: true, or a component list for parallel conflict scheduling. Both forms ship the factory templates to its worker on load.In that system's worker entry, pass your component registry to
createEntitySystemWorker, so the worker has each component's block builder:createEntitySystemWorker(self, shipUpdate, registry);
Every component is already defined as two halves for exactly this — toBlock(config) (the block values) and
attach(entity, memory, index) (the accessor), see Defining components. The worker
runs only toBlock (off-thread, no entity/world), and the main thread runs attach when it adopts the entity.
So creating a component in a worker needs nothing extra as long as its toBlock is genuinely pure over config —
no entity/world access:
const shipHealth: ComponentDefinition<HealthComponent, Int32Array, HealthConfig> = {
// ...
toBlock(config) {
return [config.health ?? config.maxHealth, config.maxHealth];
},
attach(entity, memory, index) {
const block = memory.getBlock(index);
// ...build the accessor over `block`
},
};This runs identically on the main-thread fallback. A loadInFinishLoading component (one that reads other
entities) is skipped, since the worker has no world to read. A worker does not receive constructors; it returns
the serializable class key, and the main-thread factory constructs the wrapper during adoption.
Adding and removing components at runtime
On the main thread, use the entity directly:
ship.loadComponent('shield', { capacity: 100 });
ship.removeComponent('shield');Both operations notify every system. Required and excluded (not) queries change membership as needed. An
entity already present through other required components is re-sent to worker systems too, so their cached
component bundle gains or loses an optional component on the next run.
Inside an update function, use the worker helpers instead. They record structural changes without changing the component snapshot currently being iterated:
import { addComponentWorker, removeComponentWorker } from '@daneren2005/shared-memory-ecs/worker';
if(equippedShieldGenerator && !components.shield) {
addComponentWorker(world, entityId, 'shield', { capacity: 100 }, callbacks);
} else if(!equippedShieldGenerator && components.shield) {
removeComponentWorker(entityId, 'shield', callbacks);
}The main thread applies those commands after the worker run completes, emits the normal component-added or component-removed event, and queues query deltas. The originating run therefore keeps a stable snapshot and all systems see the new structure no later than their next run. Removing a component uses the same deferred-free safety described below.
Worker-side addition needs two opt-ins because it allocates a block:
- Set
addsComponents: trueon theEntityWorkerSystem, or a list of the components it adds soConflictSchedulercan run it in parallel with unrelated systems. - Pass the component registry to that system's
createEntitySystemWorker(self, update, registry)call.
addComponentWorker calls the definition's worker-safe toBlock(config) and cannot add the always-present
entity component or a loadInFinishLoading component. Worker-side removal needs no allocation opt-in.
Freeing component memory safely
Component blocks live in a shared pool, so a freed block gets handed straight back out to the next entity that needs one. That is a problem across workers: if one system kills an entity, a second reuses its freed block for a brand-new entity, and a third is still mid-run over what it thinks is the old entity, the third system writes into the new entity's memory.
So the library never frees a block the instant it is orphaned. When an entity dies or you call
removeComponent, the block is deferred — the component is gone from the entity immediately, but the memory is
held until every system that could be mid-run over it has finished a run. Only then is the block returned to the
pool for reuse. This is automatic; there is nothing to call. The one visible effect is that
memoryComponent.length can briefly sit one higher than the number of live components, until the next update
lets the holding systems finish. If a system stays stuck (never completes a run) for more than ten seconds of
unscaled time, the world logs a warning naming that system and frees the blocks anyway rather than leak them —
a stuck system there is a bug worth chasing down.
Freeing extra resources
Everything above frees a component's own block. A component that allocates something else in attach —
another heap structure (a SharedList, a SharedString) or child entities it owns — needs to release that
too, and the block-level deferred free won't do it. Give the definition an optional free(component):
const cargoDefinition: ComponentDefinition<Cargo, Uint32Array, CargoConfig> = {
type: Uint32Array,
size: 3,
loadProperties: ['cargoSpace'],
toBlock(config) {
/* the block's own values */
},
attach(entity, memory, index) {
/* allocate a SharedList in the heap, stash its pointer in the block, return accessors */
},
free(component) {
component.items.free(); // release the SharedList + the item entities it holds
},
};free runs exactly once per component teardown: on removeComponent, and on every path that removes an
entity — removeEntity, a death (from killEntity), a load that drops the old batch, and clear.
It is deferred to the same safe point as the component's own block (see above), not fired the instant the
entity dies — so a system still mid-run over that memory can't see your resource released early. Reload calls
it without any death event, so it is the reliable place to avoid leaks when a world is reused.
Reacting to death
To react when an entity dies — spawn drops at its position, credit a kill, play an effect — a component
definition can carry an optional died(component, entity, world). It fires exactly once per death, on the main
thread, with the dying entity still fully readable (its blocks are only deferred-freed, never torn down
before the hook runs), for both a worker kill (killEntityWorker) and a main-thread killEntity:
const oreDepositDefinition: ComponentDefinition<OreDeposit, Int32Array, OreDepositConfig> = {
type: Int32Array,
size: 3,
loadProperties: ['oreAmount'],
toBlock(config) { /* ... */ },
attach(entity, memory, index) { /* ... */ },
died(component, entity, world) {
// The asteroid is still readable here, so spawn its drop where it was.
const { x, y } = entity.components.position!;
world.loadEntity({ type: 'Ore', x, y, amount: component.oreAmount });
},
};The hook is engine-agnostic: attach death behavior to the component that represents it. A throwing died can't
abort the death — it is caught and surfaced as a system-error event (phase: 'died') while the entity is still
removed. Unlike free, died fires only on a real death, never on world.load()/clear() teardown.
For a non-declarative hook, the world also emits world.on('entity-died', entity) at the same point (distinct
from entity-removed, which fires for every removal, reloads included).
Reusing a world: load and clear
A world is meant to be reused rather than rebuilt. world.load(config) swaps in a fresh scenario — it removes
the old entities, clears every system, and loads the new batch (deferred frees from the old contents drain over
the following updates, as above). Loading itself is synchronous, but it returns a promise that resolves once every
system's finishLoading() (e.g. a worker re-seeding its init data) completes — await world.load(config) before
starting the game loop. await world.clear() instead tears the world all the way back down to an
empty, reusable state: it waits for every system's in-flight worker run to finish so nothing is still reading
the memory, frees the held blocks immediately, and resolves once the world is ready to load into again.
Prefer world.load() over looping world.loadEntity() for a large batch. It still emits entity-added for every
entity, but systems ignore those while world.isBulkLoading is set and instead join the whole batch once at the end
through system.addEntities(batch). Entities with the same component keys are grouped, so each query's component
test runs once per group instead of once per entity; system membership keeps load order. A system that overrides
checkAddEntity (or an EntityWorkerSystem that overrides matchesQuery) still has it called for each entity.
Loading in slices with progress
A big save can block the main thread for seconds. Pass sliceMs and load() yields to the event loop whenever a
slice runs over, reporting progress as it goes:
await world.load(save, {
sliceMs: 25,
onProgress: ({ phase, done, total, fraction }) => setBar(fraction),
// Systems hold every entity and their load messages are posted; only worker replies are left.
onEntitiesReady: () => buildIndexes(world),
});Phases run entities → linking (finishLoading, entity-added) → systems (addEntities, then
finishLoading) → workers (replies). fraction weights them 0.6 / 0.1 / 0.1 / 0.2. yieldTo overrides the yield
(default scheduler.yield(), else setTimeout(0)). While world.isLoading is set, update() does nothing; don't touch
the world until load() resolves. A load(), clear() or destroy() during a sliced load, or aborting its signal, rejects it
with an AbortError, and the next load or clear tears down the partly loaded world. Without sliceMs, load() runs
synchronously up to awaiting the workers, and onProgress/onEntitiesReady still fire.
Compressing saves
Big maps are mostly entities whose save is a type plus a few integers. compressEntities(entities, definitions)
packs each config that fits a definition into a short ASCII string and leaves every other config as an object:
import { compressEntities } from '@daneren2005/shared-memory-ecs';
const save = compressEntities(entities, [
{ type: 'grass', properties: [
// Stored as (value - offset) / step; tile centres become small tile indices.
{ name: 'x', type: 'uint', step: 32, offset: 16 },
{ name: 'y', type: 'uint', step: 32, offset: 16 },
{ name: 'harvestAmount', type: 'uint' },
{ name: 'harvestProgress', type: 'uint' },
] },
]);
// { type: 'grass', x: 7920, y: 15440, harvestProgress: 633000 } becomes '#g(q-!ok{)'; a worker stays an object.
// => { entityDefinitions: { 0: { type: 'grass', properties: [...] } }, entities: ['#g(q-!ok{)', { type: 'worker', ... }] }
await world.load({ ...save, gameTime });world.load() expands string entities with the bundled entityDefinitions, and decompressEntities /
decompressEntity do the same for tools. Property types are int (signed), uint, boolean, float32 and string. int
and uint decode as stored * step / 10 ** decimals + offset (defaults 1, 0, 0), so decimals: 2 keeps hundredths.
float32 takes any number a Float32Array holds exactly, such as positions and velocities read from shared memory. A config stays an
object when it has a property its definition lacks, or a value that wouldn't decode back exactly. Only definitions some
string uses are bundled, so defining rare types costs nothing until one appears.
Types with the same properties can share one definition by listing types instead of type; the save then stores
the property list once:
{ types: ['oak', 'pine', 'birch'], properties: [...] }
``` Missing values decode as absent. The definitions travel inside the save, so a later build can append,
reorder or drop properties without breaking older saves.
The string format lets other tools read saves without this library. A string is the definition key, then one field
per property in definition order. A shared definition saved at key `n` gives its `types` the keys `n`, `n + 1`, ... in
order, so every type still has its own key and sharing adds nothing to the strings. A field is `!` for absent, or a varint: little-endian base-45 digits, where every
digit except the last comes from `` PQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|} `` and the last from
`#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNO` (digit = position in its alphabet). Fields after the last present
value are left out. `int` values are zigzag encoded (`n >= 0 ? 2n : -2n - 1`), booleans are 0 or 1, and a `float32` is
the varint of its 32 IEEE 754 bits. A `string` is a varint length in UTF-16 code units followed by the raw characters,
which may be any character; length 0 means `null`, and empty strings stay objects.
### Seeded randomness
`world.random` is a seeded generator stored in the world's shared memory. The main thread and every worker
(`world.random` inside update functions and `init`) draw from the same stream with no lock: each draw atomically
claims the next position and encrypts it with Philox4x32-10 keyed by the seed, so two threads drawing at once
never share a position. Philox is the counter-based generator from Random123 (also used by numpy and cuRAND)
and passes the BigCrush and PractRand test suites.
```ts
world.random.next(); // [0, 1) with 53 bits of precision, like Math.random
world.random.int(1, 7); // integer in [1, 7)
world.random.uint32(); // raw 32-bit valueEvery world.load() seeds it from the config's random before any entity loads:
await world.load({ entities }); // fresh random seed
await world.load({ entities, random: { seed: 1234 } }); // chosen seed (integer up to 2^53), e.g. a shared map seed
// Saving the state stops save scumming: reloading replays the same upcoming values.
const save = { entities, gameTime: world.gameTime, random: world.random.getState() }; // { seed, position }
await world.load(save);Leave random out of the save if you don't care; each load then gets a new random seed.
The stream is reproducible, but when several threads draw at the same time, which thread gets which value depends on timing. A save restores what comes next, not which system draws it.
Measuring performance
PerformanceTiming watches a world and reports what running it costs. Hand it the world and it hooks itself
up to the events the world already emits - there is nothing to call per frame and nothing added to the hot
path:
const timing = new PerformanceTiming(world);
timing.on('stats-updated', (stats: PerformanceStats) => renderDebugPanel(stats));It gathers samples every frame and collapses them into a fresh timing.stats snapshot once
ticksBetweenUpdates (default 1_000) worth of elapsed time has gone by, then emits stats-updated with it.
The window is measured in whatever unit you drive world.update with, so a game running on milliseconds gets
a snapshot a second. Frames the world was paused for are skipped, since it does no work on them.
Every entry is an { avg, min, max, samples } over the window just closed:
stats.update- one wholeworld.updatecall on the thread the world lives on.stats.systems[]- per system, in run order:runis the run itself on its worker, as the worker measured it, andeventsis what handling that run's results (the events it reported onto entities, the entities it asked to be created) cost back on the calling thread. Both worker and main-thread fallback runs report these timings. ForworkBatchSize, one sample sums all batches in a complete pass: batches taking 3, 5, and 4 ms report a 12 ms run withsamples: 1. Event handling is summed across the pass too; gaps between batches are excluded. Failed preparation closes the attempt with the time spent so far.stats.events- every system's event handling added together. Workers finish on their own schedule rather than on a frame boundary, so there is no per-frame combined sample to take; these are the per-system figures summed at the end of the window, giving what a run of every system costs the main thread between them.
To coordinate reporting with an external clock, construct with { autoUpdate: false } and call
timing.flush() at each reporting boundary. It publishes and returns a snapshot of all collected update,
worker, and event samples, then starts a fresh window. This also works while paused: windows without work
report zeroes. A batched pass retains its partial totals across reporting boundaries and contributes
one sample when its final batch arrives (event timing completes when its results are handled). Clear/reload
drops unfinished passes, and the next pass starts fresh totals. Automatic reporting remains enabled by default.
getSystemStats(name) pulls one system out of the latest snapshot, reset() throws away everything collected
so far (worth doing after loading a new scene, when the samples either side are not comparable), and
destroy() unhooks it from the world.
Building
npm install
npm run build # emits dist/ (js + d.ts)
npm run type-checkWorkers invoked directly outside scheduler dispatch publish their effects immediately. Replies from before a scheduler reset are discarded.