@etherfold/browser

Index a chain in the tab, with no server and no database to provision, and publish the result as observable stores a UI can subscribe to.

It is @etherfold/core plus the four things a browser application needs on top of the engine: a place for the state to live that survives a reload, a loop that keeps indexing, a host to run that loop somewhere that is not the UI thread, and a typed port a tab holds onto it.

When you want this package

| you are | use | | --- | --- | | indexing inside a tab, client-side, over EIP-1193 | here | | indexing on a server, into libSQL, from a terminal | etherfold | | writing the processor itself | @etherfold/processor-entities | | driving the engine yourself, in a runtime with no adapter | @etherfold/core |

Where the indexer runs: three hosting shapes

Indexing is a fold over every log a contract ever emitted — on the real measured workload, 31,332 events across 1,042 blocks, at 45.6 ms per block of store writes on Chromium. So WHERE it runs is the first decision, and it is a deployment choice rather than a rewrite: a host owns the container, the store and the loop, a tab holds a typed port to it, and the three shapes differ ONLY in how that port is obtained (ADR-0082). The code you write against the port is identical across all three.

| shape | how a port is obtained | pick it when | | --- | --- | --- | | dedicated worker (the default) | dedicatedWorkerHost(() => new Worker(…)) | almost always | | SharedWorker | sharedWorkerHost(() => new SharedWorker(…)) | several tabs of one app should share one fold | | main thread | createIndexerState(…).mainThreadHost() | tests, a small backfill, or a bundler that cannot emit a worker |

The default: a dedicated worker

You write two short files. The worker entry imports the processor — a processor is code and closures, so it can never cross a postMessage — and the tab holds a port.

// indexer.worker.ts — the one file your app writes for this
import {createBrowserStateStore, hostIndexerInThisWorker} from '@etherfold/browser';
import {fromEntityProcessor, openForWriting} from '@etherfold/processor-entities';
import {myProcessor, source, provider} from './my-app.js';

hostIndexerInThisWorker({
	// versioned rows in IndexedDB: the browser default, decided on measurement (ADR-0024).
	// `openForWriting` is what makes the HOST the writer of that store, and every tab a
	// reader: building a store and claiming it are two acts (ADR-0077).
	// `signal` bounds the writer claim: a storage that never answers becomes a
	// refusal the app can render, rather than a wait with no end.
	createState: async (context, {signal}) =>
		openForWriting(await createBrowserStateStore(myProcessor.entities, {databaseName: 'my-app'}), {signal}),
	createProcessor: (store) => fromEntityProcessor(myProcessor)(store),
	provider, // an EIP-1193 provider is an object with methods, so it is built HERE
	source: {chainId: '11155111', contracts: [{abi, address: '0x…', startBlock: 3040661}]},
	config: {stream: {finality: 12}},
});

// the tab
import {connectToIndexerHost, createPortReadSurface, dedicatedWorkerHost} from '@etherfold/browser';
import {myProcessor} from './my-app.js';

const indexer = connectToIndexerHost(
	// the URL must be a literal your bundler can trace, which is what keeps the
	// processor type-checked and de-duplicated with the rest of your app
	dedicatedWorkerHost(() => new Worker(new URL('./indexer.worker.ts', import.meta.url), {type: 'module'})),
);

// TYPED reads over the port, generated from the declarations you already wrote,
// with no query runtime loaded at all
const reads = createPortReadSurface(indexer, myProcessor.entities);
render((await reads.counter.getCurrent({name: 'transfers'}))?.value ?? 0);

// status is PUSHED, not polled: the host posts when a batch LANDED or the phase moved
indexer.onProgress(({phase, blocksBehindTip}) => {
	banner.textContent = phase === 'at-tip' ? 'live' : `syncing, ${blocksBehindTip} blocks behind`;
});

// and so is the STATE MOVING, which is what tells this tab to re-read: one
// notification per block the fold applied, one per reorg, and silence at the tip
// because nothing moved. The whole reader rule is these two lines (ADR-0083).
let held: string | undefined;
indexer.onStateMoved((moved) => {
	if (moved.coherence !== held) {
		held = moved.coherence; // a reorg or a promotion: everything you hold may be stale
		return refetchEverything();
	}
	if (moved.kind === 'applied') for (const entity of moved.entities) refetch(entity);
});

The host starts folding as soon as it exists. startIndexing() / stopIndexing() turn the driver off and on from the tab (a stopped host still answers reads), reconfigure({source}) folds a new source in a generation beside the live one, and checkTxInclusion(...) answers the optimistic-update question. A worker the browser evicts is an expected event with a defined outcome: the port TELLS you (onHostDeath), rejects every call in flight with an IndexerHostDiedError rather than hanging, starts another host, and the fold resumes from the cursor.

createProgressReadable(indexer) is the small reactive wrapper over the pushed progress, for the app that just wants a progress bar.

GraphQL over the port is opt-in (ADR-0099). This package carries a generic query case and never imports a query language: pass a query handler where the host is built (hostIndexerInThisWorker({..., query: graphqlQueryHandler()}), the same on hostIndexerInThisSharedWorker, or createIndexerState(...).mainThreadHost({query})) and hold workerExecutor(indexer) in the tab, both from @etherfold/graphql/worker. A host whose entry passed none refuses indexer.query(...), and its worker bundle carries no graphql; taking it costs about 48 KiB gzipped in the worker bundle (see @etherfold/graphql's README). A closed port rejects a call with IndexerPortClosedError, beside IndexerHostDiedError for a host that died.

onStateMoved is deliberately a plain callback and nothing more: it is what every client library's invalidation API already is (invalidateQueries, refetchQueries, reexecuteOperation), and the value it hands you is the one @etherfold/core published, unchanged — so the same handler works when the notification later arrives from another tab or from a server. A tab that attaches part way through is told nothing until the fold next moves, because a notification is a thing that HAPPENED rather than a value to render: read through the surface you already hold instead.

A second window is not a stale window

A tab that is not doing the indexing has no host to ask. openStateMovedAcrossTabs carries the same notification between tabs of one profile over a BroadcastChannel, so it re-reads when the state moved instead of polling:

import {openStateMovedAcrossTabs} from '@etherfold/browser';

// every tab, whether or not it is the one indexing
const tabs = openStateMovedAcrossTabs({databaseName: 'my-app'});
tabs.onStateMoved(rerenderFromTheStore); // the SAME handler you wrote for the port

// and, in a tab that holds a host, forward what that host tells it
indexer.onStateMoved(tabs.publish);

The channel is scoped by the storage this state lives in — the databaseName you built the store with, which is the same thing the writer claim is scoped to — so two tabs of one app hear each other and two unrelated indexers on one origin never do. Pass the same value you passed createBrowserStateStore; a name invented here would be a second answer to "which store is this".

It is an ADAPTER and not a second mechanism: what crosses is the value the fold published, so a reader's rule is unchanged. Delivery is best-effort — nothing is buffered for a tab that was not listening, and a tab that missed one converges on the next, because the coherence token it carries is one that tab has not seen. A tab is never handed back its own publication, so wiring both lines above in every tab is correct rather than noisy. Which tab indexes is not decided here and there is no election, lease or heartbeat: every indexing tab publishes, every tab listens, and the writer claim is what already guarantees only one of them is writing.

A SharedWorker

The same entry point with one word changed (hostIndexerInThisSharedWorker), and sharedWorkerHost(() => new SharedWorker(url, {type: 'module', name: 'my-app-indexer'})) in the tab. It wins one store connection and no election at all, and pays for it: no devtools panel (it needs chrome://inspect), no terminate(), and every tab's reads funnel through the one instance instead of parallelising across a worker per tab. Name it — the name is half of a SharedWorker's identity, the script URL being the other half, which is what keeps two apps on one origin apart with nothing to configure.

The main thread

createIndexerState IS the main-thread host — not a second way of doing the same thing. It owns the container, opens the store for writing and runs the loop, so indexer.mainThreadHost() joins a wire to the host that is already there rather than constructing another one.

What it costs is the fold on your UI thread, which is the whole reason a dedicated worker is the default: 45.6 ms per block of store writes is jank in an app that is also trying to render. It is the right shape for tests (an in-process path is needed regardless), for a backfill small enough that nobody notices, and for a build that cannot emit a worker.

const indexer = createIndexerState({createState, createProcessor});
await indexer.init({provider, source, config: {stream: {finality: 12}}});
await indexer.startAutoIndexing();

// the SAME port surface the two worker shapes offer, over a real MessageChannel:
// what could not cross to a worker does not cross here either
const port = connectToIndexerHost(indexer.mainThreadHost(), {watch: false});

Pass {watch: false}: a host on this thread cannot die independently of the tab holding the port, so the liveness probe has nothing to find.

The hook the main-thread host publishes

The main-thread host is also a hook, and an app that runs there can subscribe to it directly instead of holding a port. You hand over the two FACTORIES rather than their results: an indexer holds any number of generations (a stream plus a fold over it), one of which is canonical and answers every read, and each folds into its own state — so the hook is what calls these, once per generation.

Say WHICH FOLD this is, if your build knows. A generation spec — this one, addGeneration's and a worker host's alike — takes an optional processorIdentity: the identity the app's ARRIVAL derived, ordinarily the sha256:<hex> of the self-contained bundle the processor came from. It is what the generation is registered under, compared for equality and rendered into messages and never parsed, so an edited handler is a different generation whether or not anybody bumped anything (ADR-0086). Leave it off and the identity is still DERIVED, never absent and never yours to state: a fold that arrived as a MODULE (a dev server's unbundled ESM, which has no bytes to hash) is named by a digest of its HANDLER SOURCES instead, so that edit moves the identity and a save that changed nothing does not, and is answered {stateDiscarded: false} rather than costing a rebuild. There is no declared version field to fall back on: ADR-0086 deleted the field itself, along with the getVersionHash() that composed it with the entity and config declarations, so nothing in the tree reads a processor version for anything. A processor whose handlers have no readable source, all bind-ed or behind a proxy, has nothing to derive from and is REFUSED before anything is registered, with both ways out named in the message.

import {createBrowserStateStore, createIndexerState} from '@etherfold/browser';
import {fromEntityProcessor, openForWriting} from '@etherfold/processor-entities';

const indexer = createIndexerState({
	// `signal` bounds the writer claim: a storage that never answers becomes a
	// refusal the app can render, rather than a wait with no end.
	createState: async (context, {signal}) =>
		openForWriting(await createBrowserStateStore(myProcessor.entities, {databaseName: 'my-app'}), {signal}),
	createProcessor: (store) => fromEntityProcessor(myProcessor)(store),
});

await indexer.init({
	provider: window.ethereum,
	source: {chainId: '11155111', contracts: [{abi, address: '0x…', startBlock: 3040661}]},
	config: {stream: {finality: 12}},
});

// `state` publishes a READ HANDLE, because the state is rows in a store rather
// than an object: ask it questions instead of being handed all of it
indexer.state.subscribe(async (view) => {
	render((await view.getCurrent<{value: number}>('counter', {name: 'transfers'}))?.value ?? 0);
});

// `syncing` publishes the cursor and the progress; `status` publishes the phase
indexer.syncing.subscribe(($syncing) => showProgress($syncing.lastSync?.syncPercentage ?? 0));

await indexer.startAutoIndexing(); // or call indexMoreAndCatchupIfNeeded() on each new head

Run the PUBLISHED bundle in production. A snapshot a build publishes is keyed to the identity of the fold that computed it, which etherfold build names by the SHA-256 of the bundle file (ADR-0095). So a deployed tab runs THAT file: give the spec processorBundle: {url} and the host (on this thread or inside the worker) fetches the bytes once, hashes them, refuses them if they still import anything, instantiates the processor from those very bytes and registers the generation under their hash. Both factories are handed the instantiated bundle as their third argument, so the store can be declared from the entities the bundle itself declares. Do not pass a processorIdentity beside it (that is refused): a hash supplied next to code the tab did not hash is the lie ADR-0086 removes. Keep the module arrival above for development, where hot updates need it.

// what bytes fetched at run time contain is not something a type checker saw
const processorOf = (bundle?: InstantiatedProcessorBundle) => bundle!.processor as EntityProcessor<MyAbi>;

const indexer = createIndexerState({
	// the bundle has arrived before either factory runs, so both are handed it
	createState: async (context, {signal}, bundle) =>
		openForWriting(await createBrowserStateStore(processorOf(bundle).entities, {databaseName: 'my-app'}), {signal}),
	createProcessor: (store, _context, bundle) => fromEntityProcessor(processorOf(bundle))(store),
	processorBundle: {url: '/processor.bundle.js'},
});

A bundle that cannot run is REFUSED before any store is claimed, with a ProcessorBundleRefusedError whose reason says why: unreachable, not-self-contained, unreadable-module, not-a-processor, or forbidden-by-policy when the Content-Security-Policy that applies (a worker's is its own script response's) allows neither data: nor blob: modules, which is decided by probing each scheme so a policy is never mistaken for damaged bytes. init rejects with it; a worker host reports it as phase: 'refused' with the refusal's fields on failure.details. loadProcessorBundle(url) is the same load, answered as data, for an app that drives it itself.

indexMore, indexToLatest and indexMoreAndCatchupIfNeeded are the manual forms; calling one of them on every newHeads message is better than a timer. .withHooks(react) turns the three observables into React hooks (useState, useSyncing, useStatus); the stores are otherwise plain subscribe + $state, so Svelte and a hand-rolled loop both work.

Those three observables are the one thing that does NOT cross a port, deliberately: reproducing a same-thread reactive triple across a boundary is either polling or duplicated state in every tab, so a hosted indexer pushes progress instead and an app builds its own wrapper over the signal.

Choosing where the state lives

createBrowserStateStore is the ONE place a backend is named, and swapping it changes nothing about the processor. It does NOT claim: a tab that only renders opens the same database, so building the store and becoming its writer are separate (wrap it in openForWriting where you index, as in the usage above).

// the default: versioned rows in IndexedDB, resumed from the cursor after a reload
const store = await createBrowserStateStore(entities, {databaseName: 'my-app'});

// the light store instead: current state as an object, history as immer reverse patches
const store = await createBrowserStateStore(entities, {
	backend: (declarations) => new PatchStateStore(declarations, {retention: 'revert-only', finalityDepth: 12}),
});

Whether a RELOAD keeps the state is a property of the backend, and it is knowable in advance. The sync cursor lives behind the storage seam and is written in the same transaction as the block it describes (ADR-0027), so a tab closed mid-index reopens consistent on @etherfold/state-store-indexeddb. @etherfold/state-store-patch is memory-only by design and reports durability: 'memory-only' in its capabilities, so an app that cares can read that at startup rather than discover it from an empty tab.

What you say you KEEP is what the tab keeps. retention defaults to unbounded, and a store that states a floor instead — {blocks: N} beside a finalityDepth, or revert-only beside one — has the versions below that floor physically deleted, because the indexing cycle schedules the prune (ADR-0022: a prune is a call a host makes and never a side effect of applying a block). Each pass is bounded (createIndexerState(..., {pruneBudget})) and the next cycle picks up what it could not finish, so a long-lived tab reclaims steadily rather than stalling on one delete. The live version of an entity survives however old it is. Size a window against the chain rather than against how much history it feels like: event-bearing blocks on the real measured stream are median 429 apart, so 64 blocks is roughly one of them.

Reorg safety and NO history is revert-only, not a small window. createBrowserStateStore(entities, {retention: 'revert-only', finalityDepth: 12}) says exactly that: superseded versions survive only as long as a reorg revert needs them (the finality depth is that floor, so revert keeps working), and the store reports capabilities.asOf === false, so an app that does need history learns it at startup rather than from a refused read mid-session. A short window is a different request that looks like the same one: retention counts BLOCK NUMBERS and never updates (ADR-0019), and with event-bearing blocks median 429 apart (max 1,226,194), {blocks: 64} typically holds exactly one of them, the tip's, and often none. That buys a store that refuses almost every historical read while looking configured for history. revert-only is a retention setting, so the IndexedDB default takes it too; the light store is revert-only by construction (ADR-0023).

openAndBootstrap in @etherfold/processor-entities starts a tab from state somebody else already computed, instead of replaying the chain from the start block. Open the store through it (or through openSnapshotAware) on EVERY boot, not just the one that installs a snapshot: a bootstrapped store has no history below the snapshot's block and must keep saying so.

Reconfiguring without an outage

Both axes are in-place calls that report whether the state survived, so a caller holding a copy knows to replace it:

• the processor changed (a handler was edited): indexer.updateProcessor(fromEntityProcessor(next)(store)), where store is the one this generation's createState opened — keep the reference your factory built, or reopen the same databaseName. The core compares the two folds' IDENTITIES, and an author does not get to state one (ADR-0086): a processor a dev server handed the tab as a MODULE has no bytes to hash, so it is named by a derivation over its handler sources. So an edited handler takes effect with no version to bump, and a save that changed nothing is answered as {stateDiscarded: false} rather than costing a rebuild.
• the contract changed (a redeploy, a new ABI): indexer.updateIndexer({source}). The ABI is hashed into the source, so a changed one discards and re-indexes by itself. reset() as well would be a second full rebuild.

Both return a ReconfigureOutcome: {stateDiscarded} for the caller that only has to re-seed its own copy, plus sourceInvalidation, the verdict that bit was collapsed from, which names WHICH half stopped being valid (the raw log stream, the state folded out of it, or both) and FROM WHICH block. It is undefined on updateProcessor and reset, which ask no source question. examples/browser-reference indexes in a dedicated worker, so it drives the port's counterparts instead: a redeploy is reconfigure({source}), which folds a new generation beside the live one, and an edited processor reloads the worker.

A hot update reconfigures the tab it is running in

updateProcessor reconfigures the generation that is answering reads, so the rebuild it costs is time the app has nothing to render. reconfigureFromHotUpdate(indexer, {createState, createProcessor}) is the other shape for a dev loop: it registers a successor beside the live generation, which goes on answering every read while the new fold catches up, and it answers WHAT it did. Call it from the application's own hot-update handler, with the module that handler receives:

import {reconfigureFromHotUpdate} from '@etherfold/browser';

if (import.meta.hot) {
	import.meta.hot.accept('./processor.js', async (module) => {
		if (!module) return;
		const next = module.tokenProcessor;
		const report = await reconfigureFromHotUpdate(indexer, {
			// ITS OWN store: the incumbent goes on writing its own rows, and two
			// generations sharing one `databaseName` are one store.
			createState: async (context) =>
				openForWriting(
					await createBrowserStateStore(next.entities, {databaseName: `app-${context.stream}-${++saves}`}),
				),
			createProcessor: (state) => fromEntityProcessor(next)(state),
		});
		// 'registered' | 'unchanged' | 'failed'
		show(report.outcome === 'failed' ? report.message : report.outcome);
	});
}

This package subscribes to nothing, and contains no reference to import.meta.hot or any other bundler HMR global. Noticing a change is the application's job — the same rule the server side follows, where whatever watches a file stays outside the process — and the bundler is already the watcher. So a build with no HMR is unaffected by construction rather than by a guard, and because this is a free function rather than a method, the production build that eliminates the app's own if (import.meta.hot) block drops it too.

In a worker, the worker takes the update. Under Vite a module worker is an HMR client of its own, so the worker ENTRY's import.meta.hot.accept is handed the edited module (the measurement), and the host hostIndexerInThisWorker / hostIndexerInThisSharedWorker returned takes it: host.reconfigureFromHotUpdate({createState, createProcessor}) is this same function run against the worker's indexer, with the same three outcomes. The processor is built inside the worker, so its identity is derived there. The tab did not make that call, so it reads the outcome off the progress push, as progress.hotUpdate ({count, report}). Any module in the tab that imports the processor module has to accept the update too (an empty callback is enough), or Vite turns the save into a page reload.

It answers ReconfigureReport, which is the shape the server's POST /{indexer}/admin/upload answers when a bundle is uploaded to a running etherfold node, so the two arrivals share one contract rather than two (ADR-0085). registered names the generation now folding beside the live one; unchanged is a SUCCESS and says plainly that the handler sources are the ones already running; failed carries the reason and leaves the tab exactly as it was — same generations, same pointer, still folding, still answering — which is the ordinary case in an editing loop, since a developer saves mid-edit. A burst stays bounded with nothing to do on the caller's side: the successor slot holds at most one, so a newer save replaces the pending one (ADR-0084). There is deliberately no {force}, because forcing means registering a generation beside one of the same name and the name is what a generation IS; updateProcessor(next, {force: true}) is the in-place verb for that case.

What that derivation can and cannot see, because both halves are real. It is taken over the handler SOURCE TEXT, so it survives reformatting and re-ordering, and it does NOT survive minification or a change of transpiler — which is why it names a module a DEV SERVER handed the tab, and never a deployed build. A production app arrives as a self-contained bundle and is named by the SHA-256 of those octets, so the same code has a different identity as a module than as a bundle: a dev iteration and a deployed build are different generations either way. In the other direction it does not MOVE for a change the source text does not carry — an edited helper the handler imports, an entity declaration that changed, behaviour decided by a captured value — and updateProcessor(next, {force: true}) is what an integrator who knows better passes. Nothing an application supplies is ever taken as an identity here; a value nobody can check against the code it claims to name is the author-declared identity ADR-0086 deletes.

Two more things a browser app tends to need

• checkTxInclusion(...) answers whether the state you are about to render already accounts for a transaction. An app laying an OPTIMISTIC update over indexed state needs it: applied on top of state that already contains it, a non-idempotent update is counted twice. Its own receipt cannot answer that, because a reorg can re-include the same transaction in a different block. It is on the hook and on the port, and the verdict crosses WHOLE — a status AND the basis for it, because unknown means keep the optimistic update where an honest absent means the fold has looked.
• keepStreamOnIndexedDB(name), passed as createIndexerState(..., {keepStream}), caches the raw fetched logs so a state rebuild replays from IndexedDB instead of re-fetching every log. It is an append-only run of segments: a save costs its batch and not the history, and an inconsistent stream is cleared and re-fetched rather than repaired.

A second tab takes the store: this one becomes a reader

Every mutation is checked against a claim inside the transaction that writes it (ADR-0075), so a tab that loses the race writes NOTHING. What it does about that is demote itself to a reader: it stops fetching, drops the in-memory cursor that is now a lie, and goes on answering reads from the store the other tab is writing, so the data on screen stays correct (ADR-0078).

It is a state change and not an error: syncing.demotion carries {reason, reading} (beside streamSeed, deliberately not inside error), the auto-index loop stops and is not re-armed, and an advance answers undefined — which means demoted and means nothing else. indexer.demoteToReader('lease-lost') is the same path for an app that elects one indexing tab itself; it is not the inverse of promote, which moves the canonical pointer between generations. Getting the write duty back is dispose() plus a fresh init over a store built fresh, because a store that lost is never re-claimed.

Tests

pnpm --filter @etherfold/browser test (vitest, on fake-indexeddb) and pnpm --filter @etherfold/browser test:browser (playwright, in a real engine).

The three hosting shapes are held to ONE behaviour suite rather than to three test files that agree: browser/hostingShapes.ts holds the cases as data, browser/threeHostingShapes.spec.ts runs the list against all three in a real browser (which is the only place a Worker and a SharedWorker exist), and test/theThreeHostingShapesRunOneImplementation.test.ts runs the same list against the main-thread shape on every commit.