sol-tx-tools

A comprehensive TypeScript library and CLI for working with Solana transaction signatures and data. Features rate-limited RPC calls, streaming capabilities, and robust error handling.

Installation

npm install @nvana-dharma/sol-tx-tools
# or
pnpm add @nvana-dharma/sol-tx-tools

Features

• 🚀 Efficient signature fetching with automatic pagination
• 📡 Real-time signature streaming for monitoring accounts
• ⚡ Rate-limited transaction fetching with configurable RPS
• 🔄 Automatic retry logic with exponential backoff
• 💪 TypeScript support with full type definitions
• 🛠️ CLI tools for command-line usage
• 🎯 Order preservation for batch operations

CLI Usage

The package includes a powerful CLI tool for working with Solana signatures.

Global Options

• -r, --rpc <url> - RPC endpoint URL (required)
• -c, --commitment <level> - Commitment level: confirmed or finalized (default: finalized)

Commands

get-sigs - Fetch Historical Signatures

Get all signatures for an account with optional bounds:

# Get all signatures for an account
sol-tx-tools --rpc https://api.mainnet-beta.solana.com get-sigs \
  --account EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v

# Get signatures within a range
sol-tx-tools --rpc <RPC_URL> get-sigs \
  --account <ADDRESS> \
  --lookforward <NEWEST_SIG> \
  --lookback <OLDEST_SIG>

# Filter out failed transactions
sol-tx-tools --rpc <RPC_URL> get-sigs \
  --account <ADDRESS> \
  --filter-failed

Options:

• -a, --account <address> - Account address (required)
• -b, --lookback <signature> - Stop before this signature (exclusive oldest boundary)
• -f, --lookforward <signature> - Start from this signature (newest boundary)
• --filter-failed - Filter out failed transactions

stream-sigs - Stream Real-time Signatures

Monitor an account for new transactions in real-time:

# Stream new signatures as they arrive
sol-tx-tools --rpc https://api.mainnet-beta.solana.com stream-sigs \
  --account <ADDRESS>

# With custom poll interval (in seconds)
sol-tx-tools --rpc <RPC_URL> stream-sigs \
  --account <ADDRESS> \
  --poll-interval 5

# With verbose logging
sol-tx-tools --rpc <RPC_URL> stream-sigs \
  --account <ADDRESS> \
  --verbose

Options:

• -a, --account <address> - Account address to monitor (required)
• -p, --poll-interval <seconds> - Poll interval in seconds (default: 10)
• --filter-failed - Filter out failed transactions
• -v, --verbose - Enable verbose debug logging

The stream runs indefinitely until interrupted (Ctrl+C). It starts from the most recent signature and only looks forward for new transactions.

get-tx - Fetch Transaction Data

Fetch full transaction data for signatures provided via stdin. Reads signatures continuously from stdin and fetches transactions with rate limiting:

# Fetch transactions for specific signatures
echo "signature1" | sol-tx-tools --rpc <RPC_URL> get-tx

# Pipe signatures from get-sigs to get-tx
sol-tx-tools --rpc <RPC_URL> get-sigs -a <ADDRESS> | \
  sol-tx-tools --rpc <RPC_URL> get-tx --rps 20

# Stream transactions in real-time
sol-tx-tools --rpc <RPC_URL> stream-sigs -a <ADDRESS> | \
  sol-tx-tools --rpc <RPC_URL> get-tx --verbose

# Custom rate limiting and retries
sol-tx-tools --rpc <RPC_URL> get-tx \
  --rps 50 \
  --max-retries 5 \
  --retry-delay 2000

# Output as base64 encoded
echo "sig1" | sol-tx-tools --rpc <RPC_URL> get-tx --output-format base64

Options:

• -s, --rps <number> - Requests per second (default: 10)
• --max-retries <number> - Maximum retries per signature (default: 3)
• --retry-delay <ms> - Base retry delay in milliseconds (default: 1000)
• --max-queue <number> - Maximum queue size (default: 10000)
• --output-format <format> - Output format: json or base64 (default: json)
• -v, --verbose - Enable verbose debug logging

The command reads signatures from stdin (one per line) and outputs JSON for each transaction. Results are returned in the same order as the input signatures. The command will continue reading until stdin is closed or interrupted.

Piping and Integration

All commands output signatures to stdout (one per line), making them perfect for piping:

# Save to file
sol-tx-tools --rpc <RPC> get-sigs -a <ADDR> > signatures.txt

# Process each signature
sol-tx-tools --rpc <RPC> stream-sigs -a <ADDR> | while read sig; do
  echo "Processing $sig"
  # Your processing logic here
done

# Count signatures
sol-tx-tools --rpc <RPC> get-sigs -a <ADDR> | wc -l

Library Usage

Connection Provider Setup

The library uses an abstraction layer (IConnectionProvider) for RPC operations, allowing you to use simple connections, RPC pools, or custom implementations:

import { SimpleConnectionProvider, RpcPoolConnectionProvider } from '@nvana-dharma/sol-tx-tools'
import { Connection } from '@solana/web3.js'

// Simple connection provider (with built-in retry logic)
const connection = new Connection('https://api.mainnet-beta.solana.com')
const provider = new SimpleConnectionProvider(connection, 'finalized')

// Or use with RPC pool for better reliability (requires @nvana-dharma/rpc-pooler)
import { ExponentialBackoffRpcPool } from '@nvana-dharma/rpc-pooler'

const pool = new ExponentialBackoffRpcPool({
  urls: ['rpc1', 'rpc2', 'rpc3'],
  maxRetries: 3
})
const poolProvider = new RpcPoolConnectionProvider(pool, 'finalized')

Fetching Signatures

import { getSignaturesForAccountAddress, SimpleConnectionProvider } from '@nvana-dharma/sol-tx-tools'
import { PublicKey, Connection } from '@solana/web3.js'

// Create connection provider
const connection = new Connection('https://api.mainnet-beta.solana.com')
const provider = new SimpleConnectionProvider(connection, 'finalized')

// Fetch all signatures for an account
for await (const sig of getSignaturesForAccountAddress({
  accountAddress: new PublicKey('...'),
  connectionProvider: provider,
  commitment: 'finalized',
  filterOutFailed: true
})) {
  console.log(`Signature: ${sig.signature}, Slot: ${sig.slot}`)
}

// Fetch with bounds
for await (const sig of getSignaturesForAccountAddress({
  accountAddress: new PublicKey('...'),
  connectionProvider: provider,
  lookForwardLimit: 'newestSig123', // Start from here
  lookBackLimit: 'oldestSig456'     // Stop before this (exclusive)
})) {
  // Process signatures
}

Streaming Signatures

import { streamSignaturesWithSignalHandling, SignatureStreamer, SimpleConnectionProvider } from '@nvana-dharma/sol-tx-tools'
import { PublicKey, Connection } from '@solana/web3.js'

// Create connection provider
const connection = new Connection('https://api.mainnet-beta.solana.com')
const provider = new SimpleConnectionProvider(connection, 'finalized')

// Simple streaming with automatic signal handling
for await (const sig of streamSignaturesWithSignalHandling({
  accountAddress: new PublicKey('...'),
  connectionProvider: provider,
  pollIntervalMs: 5000,
  filterOutFailed: true,
  onError: (err) => console.error('Error:', err),
  onDebug: (msg) => console.log('Debug:', msg)
})) {
  console.log(`New transaction: ${sig.signature}`)
}

// Using the SignatureStreamer class for more control
const streamer = new SignatureStreamer({
  accountAddress: new PublicKey('...'),
  connectionProvider: provider,
  pollIntervalMs: 5000
})

for await (const sig of streamer.start()) {
  console.log(`New signature: ${sig.signature}`)

  // Stop streaming based on condition
  if (someCondition) {
    streamer.stop()
  }
}

Fetching Transactions with Rate Limiting (Batch)

For fetching a fixed list of transactions with rate limiting:

import { getTransactionsBySigs, SimpleConnectionProvider } from '@nvana-dharma/sol-tx-tools'
import { Connection } from '@solana/web3.js'

const connection = new Connection('https://api.mainnet-beta.solana.com')
const provider = new SimpleConnectionProvider(connection, 'finalized')
const signatures = ['sig1', 'sig2', 'sig3', ...]

// Fetch transactions with rate limiting (10 RPS default)
for await (const result of getTransactionsBySigs(provider, signatures)) {
  if (result.ok) {
    console.log('Transaction:', result.value)
  } else {
    console.error('Failed to fetch:', result.signature, result.error)
  }
}

// Custom rate limit
for await (const result of getTransactionsBySigs(provider, signatures, 'finalized', 20)) {
  // Process at 20 requests per second
}

Fetching Transactions with Producer-Consumer Pattern

The TransactionFetcher class provides a producer-consumer pattern for fetching transactions, perfect for integration with message queues, event streams, or other data sources:

import { TransactionFetcher } from '@nvana-dharma/sol-tx-tools'
import { Connection } from '@solana/web3.js'

const connection = new Connection('https://api.mainnet-beta.solana.com')

// Create a fetcher instance
const fetcher = new TransactionFetcher({
  provider,
  commitment: 'finalized',
  requestsPerSecond: 20,
  maxQueueSize: 10000,
  batchSize: 100,
  onError: (sig, error) => {
    console.error(`Failed to fetch ${sig}:`, error)
  },
  onDebug: (msg) => console.log('Debug:', msg)
})

// Start consuming transactions (in a separate async context)
const consumer = (async () => {
  for await (const result of fetcher.fetch()) {
    if (result.transaction) {
      console.log(`Got transaction: ${result.signature}`)
      // Process transaction data
    }
  }
})()

// Producer: enqueue signatures as they arrive
// For example, from a Kafka consumer:
kafkaConsumer.on('message', (message) => {
  const signature = message.value.toString()

  // Enqueue returns false if queue is full
  if (!fetcher.enqueue(signature)) {
    // Handle backpressure
    console.warn('Queue full, waiting...')
  }
})

// When done, stop the fetcher
fetcher.stop() // Will process remaining queue before stopping
await consumer // Wait for all transactions to be processed

Fetching Transactions from stdin

import { fetchTransactionsFromStdin, SimpleConnectionProvider } from '@nvana-dharma/sol-tx-tools'
import { Connection } from '@solana/web3.js'

const connection = new Connection('https://api.mainnet-beta.solana.com')
const provider = new SimpleConnectionProvider(connection, 'finalized')

// Read signatures from stdin and fetch transactions
for await (const result of fetchTransactionsFromStdin(provider, {
  commitment: 'finalized',
  requestsPerSecond: 20,
  maxRetries: 5
})) {
  if (result.transaction) {
    console.log('Transaction:', result.transaction)
  } else {
    console.error('Error:', result.error)
  }
}

Generic Rate Limiting

Use the rate limiter for any async operations:

import { rateLimitedMapOrdered } from '@nvana-dharma/sol-tx-tools'

const urls = ['url1', 'url2', 'url3', ...]

// Rate limit any async operation
for await (const result of rateLimitedMapOrdered(
  urls,
  async (url) => fetch(url),
  {
    requestsPerSecond: 5,
    maxRetries: 3,
    retryDelayMs: 1000
  }
)) {
  if (result.ok) {
    console.log('Success:', result.value)
  } else {
    console.error('Failed:', result.error)
  }
}

Utility Functions

import {
  calculateExponentialBackoff,
  sleep,
  isRetryableError,
  withRetry
} from '@nvana-dharma/sol-tx-tools'

// Calculate backoff delay
const delay = calculateExponentialBackoff(attemptNumber, 1000, 30000)

// Sleep with promise
await sleep(1000)

// Check if error is retryable
if (isRetryableError(error)) {
  // Retry logic
}

// Wrap any function with retry logic
const result = await withRetry(
  async () => someUnreliableOperation(),
  {
    maxRetries: 5,
    baseDelayMs: 500,
    onRetry: (attempt, error) => {
      console.log(`Retry ${attempt + 1}:`, error)
    }
  }
)

Key Features

Automatic Retry Logic

All RPC calls include built-in retry logic with exponential backoff:

• Retries on 429, 500, 502, 503, 504 errors
• Retries on network timeouts and connection errors
• Configurable max retries and delays
• Preserves request parameters across retries

Rate Limiting

The transaction fetcher uses a token bucket algorithm for smooth rate limiting:

• Configurable requests per second
• Burst capacity support
• Order preservation
• Memory efficient streaming

Signal Handling

The streaming functionality includes graceful shutdown:

• Handles SIGINT/SIGTERM signals
• Drains queued signatures before exit
• No data loss on shutdown

API Reference

Types

interface SignatureDatum {
  signature: string
  slot: number
}

interface StreamSignaturesOptions {
  accountAddress: PublicKey
  rpcUrl: string
  commitment?: Finality
  pollIntervalMs?: number
  filterOutFailed?: boolean
  onError?: (error: Error) => void
  onDebug?: (message: string) => void
  signal?: AbortSignal
}

interface TransactionFetcherOptions {
  provider: IConnectionProvider
  commitment?: Finality
  requestsPerSecond?: number
  maxQueueSize?: number
  batchSize?: number
  maxRetries?: number
  retryDelayMs?: number
  onError?: (signature: string, error: unknown) => void
  onDebug?: (message: string) => void
}

interface TransactionResult {
  signature: string
  transaction?: VersionedTransactionResponse | null
  error?: unknown
}

type Settled<T> =
  | { ok: true; value: T; index: number }
  | { ok: false; error: unknown; index: number }

Error Handling

The library includes sophisticated error handling:

• Automatic retries on transient failures
• Exponential backoff to prevent thundering herd
• Graceful degradation on persistent errors
• Detailed error reporting

Performance

• Streaming: Minimal memory footprint with async generators
• Pagination: Efficient handling of large signature lists
• Rate Limiting: Smooth request distribution
• Parallel Processing: Sliding window for concurrent operations

Development

# Install dependencies
pnpm install

# Build the project
pnpm build

# Run tests
pnpm test

# Run the CLI locally
node build/cli.js --help

License

UNLICENSED (private package)

Contributing

This is a private package in the nvana-dharma monorepo.