Magazine

A decentralized library for Ethereum event logging, syncing, and querying with pluggable storage backends — built on ethers.js with support for GunDB and Plexus.

Magazine gives you a complete pipeline: sync events from any EVM chain, store them in a decentralized peer-to-peer network, transform them into structured data models, and query them with a fluent API. Choose your storage backend — Gun for decentralized graph storage or Plexus for WASM-based mesh networking — or write your own adapter.

Quick Start

npm install magazine

With Gun (default, synchronous)

import Magazine from 'magazine';

const magazine = new Magazine({
  name: 'My Token Tracker',
  address: '0x1234567890123456789012345678901234567890',
  abi: [
    'event Transfer(address indexed from, address indexed to, uint256 value)',
    'event Approval(address indexed owner, address indexed spender, uint256 value)'
  ],
  provider: 'https://mainnet.infura.io/v3/YOUR_PROJECT_ID',
  networkId: 1,
  gun: { peers: ['https://gun-manhattan.herokuapp.com/gun'] }
});

With Plexus (async factory)

const magazine = await Magazine.create({
  name: 'My Token Tracker',
  address: '0x1234567890123456789012345678901234567890',
  abi: [...],
  provider: 'https://mainnet.infura.io/v3/YOUR_PROJECT_ID',
  networkId: 1,
  storage: {
    type: 'plexus',
    plexus: {
      seed: '64-char-hex-seed...',
      peers: ['wss://peer1.example.com'],
      scopePrefix: 'my-app'
    }
  }
});

Sync and query

// Sync events from the blockchain
await magazine.syncEvents();

// Listen to events in real-time
const unsub = magazine.on('Transfer', (event) => {
  console.log(`Transfer: ${event.args.value} from ${event.args.from}`);
});

// Cleanup when done
await magazine.destroy();

Features

• Pluggable Storage — abstract adapter layer with built-in Gun and Plexus backends, or bring your own
• Event Syncing — automatic, manual, range-based, transaction-specific, and real-time WebSocket sync
• Data Models — schema-based storage with validation, CRUD, bulk operations, lifecycle hooks, and schema migration
• Advanced Querying — MongoDB-style operators ($gt, $in, $regex, $or, $and, ...), fluent QueryBuilder, async iteration
• Aggregation Pipeline — group, bucket (time-series), sum, avg, count, min, max, sort
• Network Resilience — multi-provider failover, token-bucket rate limiting, LRU caching
• Compression — gzip, JSON, and optimized strategies for efficient storage
• Decentralized Storage — GunDB and Plexus integration for peer-to-peer data sharing
• Debug Tools — health checks, performance metrics (p95/p99), structured logging with file output

Documentation

| Guide | Description | |-------|-------------| | Getting Started | Installation, first sync, first query | | Architecture | Component diagram, data flow, storage model | | Configuration | Full config reference with all options |

Guides

| Guide | Description | |-------|-------------| | Event Syncing | Auto-sync, manual, range, WebSocket real-time | | Data Models | Schemas, CRUD, hooks, bulk ops, transformers | | Querying | QueryBuilder, operators, logical combinators | | Aggregation | Pipelines, grouping, time-series bucketing | | Schema Migration | Versioned schemas, migration paths | | Pagination | Offset, cursor, async iteration | | Network Resilience | Failover, rate limiting, caching | | Debugging | Logger, health checks, performance metrics | | Compression & Pub/Sub | Storage optimization, publishing |

API Reference

| Class | Description | |-------|-------------| | Magazine | Main entry point — syncing, querying, models | | DataModel | Schema-based CRUD with hooks and bulk ops | | AggregationPipeline | Analytics and time-series aggregation | | SchemaMigration | Document version migration | | QueryBuilder | Fluent query API for events | | EventFilter | Parameter matching and filtering | | NetworkManager | Provider failover, caching, rate limiting | | DataCompressor | Compression strategies | | Paginator | Offset and cursor-based pagination | | Logger | Structured logging | | EventSyncer | Blockchain sync engine | | DebugUtils | Health checks, performance reports | | Errors | ValidationError, ModelError |

Examples

The examples/ directory contains runnable scripts:

| Example | What it covers | |---------|---------------| | quick-start.js | Minimal setup, sync, query | | data-transformation.js | Models, schemas, event transformers | | filtering-pagination.js | Pagination, QueryBuilder, async iteration | | query-builder-advanced.js | Complex queries, $or/$and/$nor, regex | | aggregation-pipeline.js | Grouping, time-series bucketing, analytics | | lifecycle-hooks.js | Pre/post save, update, delete hooks | | schema-migration.js | Versioned schemas, migration paths | | bulk-operations.js | insertMany, updateMany, deleteMany | | realtime-sync.js | WebSocket real-time event streaming | | multi-provider-failover.js | Provider rotation and caching | | metadata-compression.js | Event enrichment, compression strategies | | debug-config.js | Config validation, health checks, debug mode | | Decentralized USDC Indexer | Full production app with peer coordination |

Named Exports

For advanced usage, all components are individually importable:

import Magazine, {
  // Storage adapters
  StorageAdapter,       // Abstract base class for custom adapters
  GunAdapter,           // GunDB adapter (default)
  PlexusAdapter,        // Plexus mesh-network adapter
  createAdapter,        // Factory: returns adapter from config

  // Core components
  DataModel,
  AggregationPipeline,
  SchemaMigration,
  ValidationError,
  ModelError,
  DataTransformer,
  EventFilter,
  ConfigManager,
  NetworkManager,
  EventStore,
  EventSyncer,
  EventMetadata,
  DataCompressor,
  Paginator,
  PaginationHelper,
  Logger,
  DebugUtils
} from 'magazine';

Development

# Install dependencies
npm install

# Build (ES, UMD, CJS with source maps)
npm run build

# Unit tests (no blockchain needed)
npm run test:unit

# Full test suite (starts Hardhat node)
npm test

Requirements

• Node.js 20+ (ES modules)
• Ethereum provider (Infura, Alchemy, or local node)
• Optional: Plexus SDK (prl) — only needed if using storage.type: 'plexus'

License

MIT — see LICENSE

Contributing

See CONTRIBUTING.md for development setup and guidelines.

Links

• GitHub
• Issue Tracker
• Changelog