@kolny/pheromone-engine
v0.1.1
Published
Reference implementation of the KOLNY allocation mathematics: realized performance to pheromone to capped capital weights, in deterministic integer fixed point.
Maintainers
Readme
@kolny/pheromone-engine
Reference implementation of the KOLNY allocation mathematics.
KOLNY is an autonomous agent fund on Solana. Foragers are agents that each run capital, and it is their realized performance, not a backtest, that becomes the pheromone deciding how the next epoch's capital is split. Pheromone evaporates, so a trail that stops earning fades out on its own.
Realized performance becomes a bounded pheromone deposit, pheromone evaporates every epoch, and the resulting trail vector is normalized, capped and thinned into per-forager capital weights. The allocation specification is the specification; this package implements it and nothing else.
npm i @kolny/pheromone-engineThe program this mirrors is deployed on devnet only, and it is an unaudited test deployment rather than production. This package itself is pure arithmetic and touches no network and no chain.
| | | |---|---| | Site | https://kolny.fi | | Source | https://github.com/kolny-labs/kolny | | Read API | https://api.kolny.fi/docs |
The package is an untrusted proposer and a mirror. It never moves capital. The Anchor program performs the same state transition on-chain and checks the result before anything is funded (architecture section 5, option C). Its outputs drive the Trail Board, the allocation preview and the indexer.
import { DEFAULT_ALLOCATION_PARAMS, runEpoch } from "@kolny/pheromone-engine";
const outcome = runEpoch(
[
{ foragerId: "forager-01", tau: 1_705_148n, status: "active" },
{ foragerId: "forager-02", tau: 602_625n, status: "active" },
],
[
{ foragerId: "forager-01", realizedReturnBps: 1_200n, realizedDrawdownBps: 300n },
{ foragerId: "forager-02", realizedReturnBps: 700n, realizedDrawdownBps: 1_100n },
],
10_000_000_000n, // deployable value under colony, base units
DEFAULT_ALLOCATION_PARAMS,
);
outcome.updates; // per-forager perf, deposit, retained, next tau
outcome.allocation.weights; // target weight and capital per forager
outcome.allocation.undeployedCapitalBaseUnits; // what stays in the vaultThe model in four lines
perf_f(e) = r_f(e) - lambda * DD_f(e) risk-adjusted realized return
D_f(e) = Q * tanh(perf_f(e) / s) bounded, signed deposit
tau_f(e+1) = max(0, (1 - rho) * tau_f(e) + D) evaporate, then deposit
w_f = normalize -> cap -> drop -> renormalizeThis is Ant System (Dorigo, Maniezzo and Colorni, 1996) with the terms reinterpreted: an edge is a forager, tour quality is risk-adjusted realized performance, evaporation is the time decay that stops capital chasing stale winners, and pheromone is the allocation weight score.
Two deliberate departures from textbook ACO, both from the specification:
- The deposit is signed. A losing epoch erodes a trail faster than passive evaporation, where classic ACO would only withhold reinforcement.
- The deposit is bounded by
Q. One enormous epoch, whether a lucky fat tail or a manipulation attempt, cannot dominate a trail.
Note on rho: this package uses the modern convention where rho is the
evaporation rate and (1 - rho) is persistence, so larger rho means faster
forgetting. The 1996 paper used the opposite convention.
Why integers, not floats
Every number in the allocation path is a bigint. There is no number
arithmetic anywhere between a realized commit and a capital figure.
The reason is architecture principle 5: the allocation has to be reproducible. The Anchor program computes the same update on-chain in integer fixed point, because Solana programs have no floating point. If this mirror used IEEE-754 doubles, the two would agree on the first few epochs and then drift, because pheromone is a recurrence: each epoch's rounding error is carried into the next one and amplified by the normalization. A drift of one part in 1e15 eventually flips a weight across the cap or the drop threshold, and at that point the off-chain Trail Board and the on-chain allocation disagree about who gets paid. That is not a display bug, it is the audit trail breaking.
Three conventions make the two implementations agree exactly. The Anchor side must match all three:
- Scales.
FP6(1e6) for pheromone, deposits and weights.BPS(1e4) for every rate committed on-chain: returns, drawdown, config ratios. Raw amounts stay in the base asset's own units. - Multiply before divide. Every ratio is
(a * b) / cthroughmulDiv, nevera * (b / c). Intermediates are wide enough that this cannot overflow:i128on the Rust side, arbitrary precision here. - Truncation toward zero. Rust's
i128 /and JavaScript'sBigInt /both truncate toward zero, including for negative operands, so no rounding-mode flag has to be agreed on. Deposits are signed, so this case is real.
bigint was chosen over a decimal library because it is the only numeric type
that maps one-to-one onto the on-chain integers with no library-specific
rounding policy in between. A decimal library would add a second rounding
contract to keep in sync, which is the exact failure this design avoids.
Fixed-point tanh
tanhFp6 is integer-only and short enough to transcribe into Rust without
drift. This is the contract the on-chain implementation has to match, so it
is written out step by step:
input x, FP6 signed output tanh(x), FP6 signed, |result| <= 1e6 - 1
working scale W = 1e12 (i128 on the Rust side)
1. if x == 0 return 0; remember the sign and work on |x|.
2. clamp |x| to 8 * 1e6. tanh(8) differs from 1 by 2.3e-7, below FP6.
3. u = |x| * 1e6 promote FP6 -> FP12
k = 0; while u >= W/4 { u /= 2; k += 1 } k <= 5
4. u2 = u*u / W
acc = (-17 * W) / 315
acc = (2 * W) / 15 + acc*u2 / W
acc = -(W / 3) + acc*u2 / W
acc = W + acc*u2 / W
t = u * acc / W tanh(u) through the 7th order
5. repeat k times: t = (2 * t * W * W) / (W*W + t*t) double-angle
6. r = (t + 500000) / 1000000 FP12 -> FP6, round half up
if r > 999999 { r = 999999 } keeps |D| < Q strictly
7. reapply the sign.Every division truncates toward zero, which i128 / and BigInt / both do.
Widest intermediate is step 5's numerator at about 2e36, inside i128.
Measured worst-case error against Math.tanh across the whole domain is under
1e-6, and the result is exact for every deposit argument in the
specification's worked example.
A lookup table was rejected here and, on these measurements, removed from the program too. The Anchor side originally used a 33-entry table with linear interpolation. No such table reproduces the specification's own section 10.1 deposit column at the three decimals the document prints, at any domain:
| Table domain | Step | Worst error | Reproduces the 10.1 deposit table |
|---|---|---|---|
| [0, 1.5] | 0.0469 | 1.8e-4 | X |
| [0, 2] | 0.0625 | 2.4e-4 | X |
| [0, 3] | 0.0938 | 8.4e-4 | X |
| [0, 4] | 0.1250 | 1.4e-3 | X |
| [0, 8] | 0.2500 | 3.8e-3 | X |
| range-reduce plus series | -- | < 1e-6 | O |
The error is not cosmetic. Carried through three epochs of the worked example it moves forager B's epoch-3 capital by up to 450 base units out of 900,000, which is 5 basis points of the pool going to the wrong forager:
accurate tanh B = 257562.6 (exact)
33-entry [0, 2] B = 257589.1 +26.5
33-entry [0, 3] B = 257541.4 -21.2
33-entry [0, 4] B = 257725.5 +162.9
33-entry [0, 8] B = 258012.6 +450.0The series method above is about twenty lines of integer Rust with no table to transcribe, so it is both more accurate and less exposed to copying error. The program now uses it, transcribed step for step from the block above.
The cap is solved as a scalar, not iterated
Section 7 step 2 describes the concentration cap as a loop: trim anything above
w_max, redistribute the excess proportionally over whoever is still under it,
repeat. The capped vector that loop converges to is characterized exactly by a
single scalar,
w_f = min(w_max, tau_f / K) with sum_f w_f = 1and computeWeights solves for that instead of iterating, which is also what
the Anchor program does. Two reasons:
- No termination to disagree about. The iterative form is correct in both languages, but its stopping condition is the one place two correct implementations can still part on the last digit. There is nothing to terminate in the scalar form.
- One pass. At most
floor(1 / w_max)foragers can be capped at once, so the capped set is found by walking the pheromone ranking once. That is what keeps the on-chain settlement crank affordable.
No shared divisor is formed anywhere. The state is the level
(capped_count, remaining, rest_sum). The capped set is chosen by the exact
integer test tau_k * remaining < w_max * rest_sum, which involves no division
at all, and every target is then one division of its own.
That is the whole point, and it is not the same as rounding a shared K. One
divisor shared by every uncapped forager biases all of their targets in the same
direction at once. Rounding it down over-allocates outright -- the Anchor
side hit this, producing 900,049 against a 900,000 pool. Rounding it up
stops the over-allocation but still disagrees with the exact answer, by up to 12
base units in the near-infeasible regime, because the bias is shared rather than
removed. Dividing once per forager has no shared bias to correct, so the sum is
bounded above by construction:
sum_f floor(remaining * tau_f / rest_sum) <= remaining. There is no rounding
direction to choose and no guard to get wrong.
The strict < matches the on-chain test, which caps when
remaining_bps * tau >= w_max * rest_sum. The scales agree exactly because the
effective cap is bps-quantized, so remaining is always 100 * remaining_bps
and both sides scale by the same factor. At exact equality the two conventions
provably coincide, and a test constructs boundary vectors to pin that rather
than leave it as an argument.
test/waterfill-k.test.ts checks the scalar result against a literal iterative
implementation written from the specification text in exact rational arithmetic,
over a deterministic sweep of vectors from 1 to 24 foragers, and separately
drives the capped set right up against the pool at the forager counts where the
on-chain rounding bug was reachable, asserting the sum never exceeds the target.
Dust is left where the chain leaves it
Proportional integer splits always leave a residue. The tempting fix is to hand the leftover units to whoever was rounded down hardest so the parts re-add to the total exactly. This package deliberately does not do that in the allocation path.
The Anchor program truncates and cannot redistribute a remainder, so up to one base unit per forager stays unplaced. An off-chain mirror that "corrected" the sum would produce a different vector from the chain on precisely the inputs where the residue is non-zero, and the Trail Board would then show an allocation the chain never made. Agreeing with the chain matters more than a tidy total.
So distributeTruncating computes floor(total * share_i / sum) per part and
stops.
Capital comes from pheromone, not from the weight column
The weight column is itself a truncated FP6 quantity, so deriving capital from it would round twice: once into the weight, once out of it. The first of those roundings has no counterpart on-chain, and it is worth a base unit or two on inputs where the weight truncation bites. Capital is therefore computed directly:
capped: capital_f = floor(pool * w_max / 1e6)
uncapped: capital_f = floor(pool * remaining * tau_f / (1e6 * rest_sum))One multiplication chain and one division each, truncated toward zero. The
weight column remains a faithful report of the same split and is what the Trail
Board renders, but no capital figure is derived from it. The widest intermediate
is pool * remaining * tau, around 1e24 for realistic inputs, so the Rust side
needs the u128 path here.
test/waterfill-k.test.ts checks each uncapped result against the exact
rational target computed independently in bigint arithmetic, and separately
asserts that the discarded via-weight path really does disagree somewhere in the
sweep, so the distinction stays a tested property rather than a claim.
What still holds absolutely:
- Nothing is created. Every part is floored, so
sum(parts) <= totalfor every input. - Nothing disappears. The shortfall is reported as
undeployedWeightFp6andundeployedCapitalBaseUnits; placed plus reserve always re-adds to the whole. - The dust is bounded by one base unit per forager holding a share.
In the section 10 worked example this shows up as epoch 3 placing 899,999 of the 900,000 pool, with one base unit reported as reserve.
distributeProportionally, the conserving largest-remainder variant, is still
exported for off-chain-only splits that have no on-chain counterpart. It must
not be used in the allocation path.
Interpretations
Two places where the specification leaves a choice to the implementer. Both are
marked INTERPRETATION in src/weights.ts.
The cap and drop steps are iterated, not run once. Section 7 lists
normalize, cap, drop, renormalize as a single pass. One pass is not enough for
two reasons. Renormalizing freed weight onto the survivors can push one of them
back over w_max, which breaks the invariant the same section states.
And the feasibility relaxation of section 7.1 depends on the active count, so
after a demotion the cap has to be recomputed for the smaller set; without
that, capital the smaller set could legally hold gets reported as un-deployed
when it did not have to be. Each round removes at least one forager, so the
loop is bounded by the forager count.
feasibilityMarginFp6 defaults to 0.01 of weight. Section 7.1 relaxes the
cap to 1/N_active + margin without fixing the margin. At exactly 1/N the cap
forces every weight equal and pheromone can no longer express any preference,
which is the degenerate state the section warns about, so the margin cannot be
zero. One percentage point of weight is small enough not to loosen the
concentration control meaningfully and large enough to leave the signal room.
It is a config field, not a constant, so a deployment can retune it.
The relaxation is transcribed from the chain, not derived here. Section 7.1
says the cap relaxes to 1/N_active + margin and fixes neither the margin nor a
ceiling. The on-chain effective_max_weight_bps resolves both, and this mirror
copies it exactly:
even = 10000 / N basis points, floor division
relaxed = min(even + capRelaxMarginBps, maxRelaxedWMaxBps)
cap = max(w_max_bps, relaxed)Three details each cost real base units before they were aligned, so none of
them is incidental: the cap is quantized to basis points because the chain
holds it in a u16; even floors; and there is no feasibility gate, so
the relaxation applies whenever 1/N + margin exceeds w_max, including counts
where the cap was already feasible.
| Active foragers | Effective cap | Placed | Reported reserve | |---|---|---|---| | 1 | 100% | 100% | 0 | | 2 | 51% | 100% | 0 | | 3 | 34.33% | 100% | 0 | | 5 | 21% | 100% | 0 | | 6 or more | 20% | 100% | 0 |
A concentration concern, recorded rather than diverged from. At one active
forager the ceiling lets a single agent hold the whole main pool, and
risk-spec.md section 2 states the blast-radius containment in terms of w_max
("cannot exceed its own allocation weight, capped at w_max, default 20
percent"). That sentence does not hold at N = 1.
This package still mirrors the chain. A predicted amount that differs from the
paid amount is the worse failure, and a mirror that quietly applied a stricter
cap would produce exactly that. maxRelaxedWMaxBps exists so the ceiling can be
lowered, but it has to be lowered on both sides in the same change. A test pins
the current mirrored behaviour so a unilateral move on either side fails loudly.
Known boundaries
w_drop implicitly caps the active-forager count. With N equally
weighted foragers every weight is 1/N. Once 1/N < w_drop, the drop rule
would demote the entire main pool. At the default w_drop = 0.03 that happens
at 34 active foragers. maxActiveForagersUnderDrop reports the limit, and
computeWeights refuses to demote every active forager at once, setting
dropSuppressed on the result instead. An empty main pool is a configuration
error, not an allocation outcome, and it is surfaced rather than silently
applied.
At N = 1 / w_max the cap overrides pheromone entirely. Section 7.1 states
this. It has a second-order effect worth knowing: since the drop test runs on
post-cap weights, water-filling in that regime can lift a nearly dead trail up
to the cap before the drop test ever sees it. The behaviour is faithful to the
specification, and the remedy is the one the specification already gives, which
is to set w_max comfortably above 1 / target_forager_count.
The worked example uses example parameters
Section 10 runs on rho 0.20, Q 1.0, s 0.10, w_max 0.35, w_drop 0.03. Those are
illustrative, chosen so five foragers can demonstrate the cap. They are not
the section 11 production defaults (rho 0.16, w_max 0.20), and
DEFAULT_ALLOCATION_PARAMS carries the production set. test/worked-example.ts
overrides them explicitly at the top of the file so the two can never be
confused.
Findings against the specification, and how they landed
Both were raised from this package, confirmed independently, and are now fixed in the allocation specification. They are kept here because the tests that found them are still in place as regression guards.
1. Section 10.1 re-rounded an intermediate. Forager E's epoch-3 pheromone
was printed as 1.087, from a worked step reading
0.8 * 0.603 = 0.483. It is 0.4824, so 0.482. At full precision the value
is 0.8 * 0.602625 + 0.604368 = 1.086468. Corrected to 1.086.
2. Section 10.2's epoch-3 B and E were briefly changed to the wrong values.
A revision published B 257.5k and E 163.6k; the correct figures are B
257.6k and E 163.5k, and the table is back to them. Once A is capped the
split is fully determined:
capital_f = 900000 * 0.65 * tau_f / (tau_B + tau_C + tau_E)
tau_B = 1.711392482 tau_C = 1.089212869 tau_E = 1.086467521
K = 3.887072872 / 0.65 = 5.980112110
B = 257562.6017 C = 163925.2850 E = 163512.1133computed at 60 significant digits with no floating point, and agreed on by this package's integer engine, a separate rational-arithmetic reference and a plain double-precision script. Truncated to base units the program stores B 257562, C 163925, E 163512, placing 899,999 with one base unit of dust in reserve.
The check that settles it without reference to any tanh implementation is the
gap between C and E:
tau_C - tau_E = 0.8 * (2 * tanh 0.2) + tanh 0.3 - tanh 0.7 = 0.002745348
gap = 0.002745348 / 5.980112110 * 900000 = 413.2 base units413 is asserted directly. The withdrawn pair implied 297, which no tanh
consistent with section 10.1 can produce.
3. The lookup-table tanh was removed on measurement. The Anchor program
originally specified a 33-entry table with linear interpolation. No such table
reproduces section 10.1's own deposit column, at any domain, and the error moved
a forager's epoch-3 capital by up to 450 base units out of 900,000. The program
now uses the integer series documented above, transcribed step for step, and
reproduces all ten published deposit values.
Everything else in sections 10.1 and 10.2 reproduces exactly.
Cross-implementation agreement
packages/anchor-program/README.md is the authority on the fixed-point
contract. This package is aligned to it on every point that moves a base unit:
| Contract | Here |
|---|---|
| tanh by integer range reduction plus 7th-order odd series | Same seven steps, same working scale |
| Widen, multiply, then divide | mulDiv everywhere |
| Truncate toward zero | BigInt /, which matches i128 / |
| Never form a shared divisor K | Capped set by integer comparison; one division per forager |
| Never double-round a weight | Capital from pheromone, not from the weight column |
| Dust stays in the vault | distributeTruncating; no residue redistribution |
| effective_max_weight_bps | Transcribed, including the bps quantization and the BPS_DENOM clamp |
The worked example agrees to the base unit: A 315000, B 257562, C 163925, D 0, E 163512, placing 899,999 of 900,000.
Cross-check fixture
crosscheck/vectors.json is a language-neutral conformance fixture: the same
file is read by this package's test/crosscheck.test.ts and by the Anchor
side's runner, and both must reproduce every expected integer with no
tolerance. It covers tanh, the pheromone update and the full allocation, and
includes the cases that actually broke -- the lookup-table error, the
w_max 3500 / tau 4837 over-allocation, the cap boundary, the concentration
edge at one active forager, and vectors where several base units of dust remain.
Both sides passing their own suites is not the same as the two agreeing; this
is the thing that checks the second claim. Contract, format and regeneration
rules are in crosscheck/README.md.
npm run build && npm run crosscheck:generate # regenerate, guarded by anchors.jsonLayout
| File | Contents |
|---|---|
| src/fixed-point.ts | Scales, mulDiv, tanhFp6, largest-remainder distribution |
| src/params.ts | Section 11 defaults and ranges, rhoBpsForHalfLife, validation |
| src/pheromone.ts | perf, deposit, evaporation, per-epoch update |
| src/weights.ts | Pool split, water-filling cap, drop, renormalize |
| src/rebalance.ts | No-trade band and turnover cap |
| src/epoch.ts | runEpoch and previewAllocation |
| src/display.ts | Display-only shares. Never an amount |
| crosscheck/ | Shared conformance fixture and its generator |
Development
npm install
npm run typecheck # tsc --noEmit
npm test # vitest run
npm run build # emits dist/The suite covers the fixed-point primitives, the pheromone update, the weight pipeline, the rebalance controls, and the full five-forager three-epoch walkthrough from section 10 asserted number for number. That last file is the load-bearing one: if the document and the code disagree, it fails.
