@solixdb/core

Production-grade Solana blockchain indexer core engine.

Features

Multiple Indexing Modes

• Polling Mode: Fetch historical blocks via RPC
• Streaming Mode: Real-time updates via WebSocket
• Hybrid Mode: Historical + live data (coming soon)

High Performance

• Adaptive batch sizing
• Concurrent block processing
• Efficient buffering and batching

Flexible Querying

• Fluent query builder API
• Filter by slot, time, program, account
• Pagination and streaming support

Production Ready

• Comprehensive error handling
• Automatic retry with exponential backoff
• Health checks and monitoring
• Detailed logging

Installation

pnpm add @solixdb/core

Quick Start

import { SolanaIndexer } from "@solixdb/core";
import { JSONStorage } from "@solixdb/storage-adapters";
import { DefaultProvider } from "@solixdb/rpc-providers";

// 1. Create storage adapter
const storage = new JSONStorage({
  directory: "./data",
});

// 2. Create RPC provider
const rpc = new DefaultProvider({
  endpoint: "https://api.mainnet-beta.solana.com",
});

// 3. Create and configure indexer
const indexer = new SolanaIndexer(
  {
    mode: "polling",
    startSlot: 100000,
    endSlot: 100100,
    batchSize: 10,
    commitment: "confirmed",
  },
  storage,
  rpc
);

// 4. Start indexing
await indexer.start();

// 5. Query indexed data
const blocks = await indexer.query.getRecentBlocks(10);
console.log("Recent blocks:", blocks);

// 6. Stop when done
await indexer.stop();

Configuration

interface IndexerConfig {
  // Indexing mode
  mode: "polling" | "streaming" | "hybrid";

  // Slot range (optional)
  startSlot?: number;
  endSlot?: number;

  // Performance tuning
  batchSize?: number; // Default: 10
  pollingInterval?: number; // Default: 1000ms
  commitment?: "processed" | "confirmed" | "finalized";

  // Retry configuration
  maxRetries?: number; // Default: 3
  retryDelay?: number; // Default: 1000ms

  // Filtering
  programIds?: string[]; // Only index these programs
  accountFilters?: AccountFilter[]; // Custom account filters

  // Logging
  logLevel?: "debug" | "info" | "warn" | "error";
}

Usage Examples

Historical Indexing

Index a specific range of slots:

const indexer = SolanaIndexer.forHistoricalData(
  100000, // start slot
  200000, // end slot
  storage,
  rpc,
  {
    batchSize: 20,
    logLevel: "info",
  }
);

await indexer.start();

// Wait for completion
await indexer.waitForCompletion();

Live Indexing

Stream real-time blockchain data:

const indexer = SolanaIndexer.forLiveData(storage, rpc, {
  mode: "streaming",
  logLevel: "info",
});

await indexer.start();

// Runs continuously, call stop() when done

Program-Specific Indexing

Index only specific programs:

const indexer = new SolanaIndexer(
  {
    mode: "polling",
    startSlot: 100000,
    programIds: [
      "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA", // SPL Token
      "JUP6LkbZbjS1jKKwapdHNy74zcZ3tLUZoi5QNyVTaV4", // Jupiter
    ],
  },
  storage,
  rpc
);

await indexer.start();

Resume from Last Indexed Slot

Automatically resume from where you left off:

const indexer = SolanaIndexer.fromLatest(storage, rpc);

// Starts from the last indexed slot in storage
await indexer.start();

Querying Data

Using Query Builder

import { QueryBuilder } from "@solixdb/core";

// Get recent blocks
const query = QueryBuilder.create().orderBySlot("desc").limit(10).build();

const result = await indexer.query.queryBlocks(query);

Quick Queries

// Recent blocks
const blocks = await indexer.query.getRecentBlocks(10);

// Blocks in range
const rangeBlocks = await indexer.query.getBlocksInRange(100, 200);

// Transactions for a program
const txs = await indexer.query.getTransactionsForProgram(
  "TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"
);

// Transactions for an account
const accountTxs = await indexer.query.getTransactionsForAccount(
  "YourAccountPublicKey"
);

// Single transaction
const tx = await indexer.query.getTransaction("signature...");

// Single block
const block = await indexer.query.getBlock(123456);

Streaming Large Result Sets

// Stream blocks (for large datasets)
for await (const block of indexer.query.streamBlocks(query)) {
  console.log("Block:", block.slot);
  // Process block...
}

// Stream transactions
for await (const tx of indexer.query.streamTransactions(query)) {
  console.log("Transaction:", tx.signature);
  // Process transaction...
}

Monitoring

Get Statistics

const stats = indexer.getStats();
console.log({
  status: stats.status,
  currentSlot: stats.currentSlot,
  processedBlocks: stats.processedBlocks,
  processedTransactions: stats.processedTransactions,
  blocksPerSecond: stats.blocksPerSecond,
  transactionsPerSecond: stats.transactionsPerSecond,
});

Progress Tracking (Historical Mode)

setInterval(() => {
  const progress = indexer.getProgress();
  if (progress !== null) {
    console.log(`Progress: ${progress.toFixed(2)}%`);
  }
}, 5000);

Health Checks

const isHealthy = await indexer.healthCheck();
if (!isHealthy) {
  console.error("Indexer health check failed!");
}

Advanced Usage

Custom Processing

import { BlockProcessor } from "@solixdb/core";

const processor = new BlockProcessor({
  includeTransactions: true,
  includeInstructions: true,
  includeAccounts: false,
  programFilters: ["YourProgramId"],
});

processor.configure({
  includeRewards: true,
});

Custom Retry Logic

import { retry } from "@solixdb/core";

const data = await retry(
  async () => {
    // Your async operation
    return await fetchSomeData();
  },
  {
    maxRetries: 5,
    initialDelay: 2000,
    maxDelay: 30000,
    backoffMultiplier: 2,
  },
  "fetchSomeData"
);

Custom Logging

import { createLogger } from "@solixdb/core";

const logger = createLogger({
  level: "debug",
  prefix: "MyIndexer",
  enableTimestamp: true,
  enableColors: true,
});

logger.info("Indexer started");
logger.debug("Processing block", { slot: 12345 });
logger.error("Error occurred", error);

Architecture

SolanaIndexer
├── IndexerEngine (orchestrates everything)
│   ├── IndexerState (tracks progress)
│   ├── IndexerConfig (configuration)
│   ├── BlockProcessor (processes blocks)
│   │   ├── TransactionProcessor
│   │   └── AccountProcessor
│   ├── RPCPoller (polling mode)
│   │   └── PollingStrategy
│   └── StreamManager (streaming mode)
│       ├── WebSocketStream
│       └── StreamBuffer
├── Storage Adapter (persists data)
└── RPC Provider (fetches data)

Error Handling

import { IndexerError, RPCError, StorageError } from "@solixdb/core";

try {
  await indexer.start();
} catch (error) {
  if (error instanceof IndexerError) {
    console.error("Indexer error:", error.message, error.code);
  } else if (error instanceof RPCError) {
    console.error("RPC error:", error.message);
  } else if (error instanceof StorageError) {
    console.error("Storage error:", error.message);
  }
}

Performance Tips

1. Batch Size: Larger batches = higher throughput, but more memory usage
2. Commitment Level: Use 'confirmed' for a balance of speed and finality
3. Program Filters: Filter early to reduce processing overhead
4. Adaptive Polling: Enable for automatic performance tuning
5. Storage Choice: Use PostgreSQL/ClickHouse for production workloads

License

MIT

Support

• Documentation: https://docs.solixdb.xyz
• GitHub Issues: https://github.com/SolixDB/solixdb/issues