solana-llm

On-chain LLM inference on Solana. Run GPT-2 (TinyStories-1M) entirely within blockchain transactions.

Break Solana: Edition 2 — by @STACCoverflow

What is this?

A 1-million-parameter GPT-2 model (TinyStories-1M) that runs entirely on-chain on Solana. Every matrix multiplication, every attention head, every layer norm — all executed as Solana transactions. Each token requires ~191 transactions and ~148.5M compute units, consuming roughly 25% of Solana's network capacity for 1 minute per sentence.

This is a sequel to the original Clockwork exploit that demonstrated Solana's compute limits.

Install

npm install -g solana-llm

Usage

Basic Inference

# Using a config file (localnet)
solana-llm -c ./preload_config.json "Once upon a time"

# Using explicit keys (mainnet)
solana-llm -r https://api.mainnet-beta.solana.com \
  -p <PROGRAM_ID> --shard0 <SHARD0> --shard1 <SHARD1> \
  "Once upon a time"

# With priority fees for mainnet
solana-llm -c config.json --cu-price 10000 "There was a"

# JSON output
solana-llm -c config.json --json "Once upon a time"

SAY THE MAGIC WORD (Game)

# Check pot balance
solana-llm-game -c config.json --pot-info

# Play the game (costs 1 SOL entry fee)
solana-llm-game -c config.json "Tell me about magic"

Game Rules:

1. Pay 1 SOL entry fee (goes to prize pot)
2. Choose a prompt (max 32 tokens)
3. The on-chain LLM generates text from your prompt
4. If ANY generated token is " magic" (token 5536), you WIN THE ENTIRE POT
5. Session rent (~1 SOL) also goes to the pot when you close
6. The pot grows with every play until someone wins

Architecture

| Component | Details | |-----------|---------| | Model | TinyStories-1M (GPT-2/GPT-Neo) | | Parameters | 1M (8 layers, 64 hidden dim, 16 heads) | | Weights | F16 embeddings, F32 transformer, INT8 lm_head | | Storage | ~10.78 MB across 2 shard accounts | | Per Token | ~191 transactions, ~148.5M CU | | Network Impact | ~25% of Solana capacity per sentence |

Instruction Set

Each position in each layer requires 15 fine-grained instructions to stay within Solana's 1.4M CU limit:

| Instruction | Purpose | ~CU | |-------------|---------|-----| | LN1_Q | LayerNorm + Q Projection | 895K | | K_PROJ | K Projection | 850K | | V_PROJ | V Projection | 880K | | ATTN | Attention Scores | 80K | | O_PROJ | Output Projection + Residual | 885K | | LN2 | LayerNorm 2 | 56K | | FFN_UP x4 | FFN Up-projection (chunked) | 993K | | FFN_DOWN x4 | FFN Down-projection (chunked) | 880K | | FFN_RESIDUAL | Residual + Advance Layer | 12K |

Config File Format

The preload_config.json file contains the deployed account addresses:

{
  "program_id": "<PROGRAM_PUBKEY>",
  "shard_pubkeys": ["<SHARD0_PUBKEY>", "<SHARD1_PUBKEY>"],
  "state_pubkey": "<STATE_PUBKEY>",
  "worker_pubkeys": ["<W0>", "<W1>", "<W2>", "<W3>"]
}

Programmatic Usage

const { decodeToken, encodePrompt, sendIx, createSession, IX, MAGIC_TOKEN } = require('solana-llm');

// Decode a GPT-2 token ID to text
console.log(decodeToken(5536)); // " magic"

// Encode text to token IDs
console.log(encodePrompt('Once upon a time')); // [7454, 2402, 257, 640]

// Check if a token is the magic word
if (tokenId === MAGIC_TOKEN) {
  console.log('WINNER!');
}

Cost Analysis

| Metric | Value | |--------|-------| | Storage rent | ~78.65 SOL | | Prefill (4 tokens) | 584 txs, 0.003 SOL | | Per generated token | ~191 txs, 0.001 SOL | | Full sentence | ~1,700 txs, 0.009 SOL | | Game entry fee | 1 SOL |

Links

• Token: CLWeikxiw8pC9JEtZt14fqDzYfXF7uVwLuvnJPkrE7av
• Trade: axiom.trade/@raycc
• Twitter: @STACCoverflow

Disclaimer

I have no idea if this actually works on mainnet. It probably shouldn't. Just vibes, math, and an unreasonable number of transactions. Thank you for participating.

License

MIT