trusted-network-providers

A lightweight Node.js library for identifying IP addresses that belong to trusted network providers. Prevent your firewalls from blocking legitimate services like payment processors, search engine crawlers, and monitoring tools.

⚠️ v2.0.0 Breaking Changes: This version migrates from CommonJS to ES modules. Use import instead of require(). See the migration guide for details.

Quick Start

npm install @headwall/trusted-network-providers

import trustedProviders from '@headwall/trusted-network-providers';

// Load built-in providers (Googlebot, Stripe, PayPal, Cloudflare, etc.)
trustedProviders.loadDefaultProviders();

// Initialize dynamic providers
await trustedProviders.reloadAll();

// Check an IP address
const provider = trustedProviders.getTrustedProvider('66.249.66.87');
console.log(provider); // "Googlebot"

const unknown = trustedProviders.getTrustedProvider('123.123.123.123');
console.log(unknown); // null

Use Cases

• Firewall Management: Whitelist IPs for WordPress hosting networks
• Rate Limiting: Bypass rate limits for trusted crawlers
• Access Control: Allow trusted services through security layers
• Traffic Analysis: Identify and categorize legitimate traffic sources

Built-in Providers

Includes 20+ trusted providers out of the box:

• Search Engines: Googlebot, Bingbot, AhrefsBot, SemrushBot
• Advertising: Google AdsBot, AdSense (Google Special Crawlers)
• User-Triggered Fetchers: Google (Gmail image proxy, Chrome prefetch proxy, Feedfetcher, etc.)
• Payment Processors: Stripe, PayPal, Opayo
• Email Services: Outlook, Brevo, Mailgun
• CDN/Infrastructure: Cloudflare, BunnyNet
• Development Tools: GTmetrix, GetTerms, Labrika
• Social Media: FacebookBot
• E-commerce: ShipHero
• Networks: Private/Internal (RFC 1918)

Key Features

• ✅ Fast synchronous IP lookups (< 1ms typical with caching)
• ✅ Support for IPv4 and IPv6
• ✅ CIDR range matching
• ✅ Dynamic provider updates from external sources
• ✅ Provider lifecycle events (reload, error, stale detection)
• ✅ Configurable result caching with TTL
• ✅ Easy custom provider integration
• ✅ Zero configuration with sensible defaults

API

Core Functions

// Load default providers
loadDefaultProviders();

// Update all providers with dynamic data
await reloadAll();

// Check if IP is trusted (returns provider name or null)
getTrustedProvider(ipAddress);

// Check if IP is trusted (returns boolean)
isTrusted(ipAddress);

// Provider management
addProvider(provider);
deleteProvider(providerName);
hasProvider(providerName);
getAllProviders();

// Testing
await runTests();

Lifecycle & Monitoring

// Get provider status and metadata
const status = getProviderStatus('Googlebot');
// Returns: { name, state, lastUpdated, lastError }
// State: 'ready' | 'loading' | 'error' | 'stale'

// Listen to provider lifecycle events
trustedProviders.on('reload:success', ({ provider, timestamp }) => {
  console.log(`${provider} reloaded at ${new Date(timestamp)}`);
});

trustedProviders.on('reload:error', ({ provider, error }) => {
  console.error(`${provider} failed to reload:`, error);
});

trustedProviders.on('stale', ({ provider, lastUpdated, staleDuration }) => {
  console.warn(`${provider} is stale (last updated ${staleDuration}ms ago)`);
});

// Configure staleness detection (default: 24 hours)
trustedProviders.setStalenessThreshold(12 * 60 * 60 * 1000); // 12 hours

Caching & Performance

// Configure IP lookup result cache TTL (default: 1 hour)
trustedProviders.setResultCacheTTL(30 * 60 * 1000); // 30 minutes
const currentTtl = trustedProviders.getResultCacheTTL();

Logging

// Configure log level: 'silent' | 'error' | 'warn' | 'info' | 'debug'
trustedProviders.setLogLevel('info');
const level = trustedProviders.getLogLevel();

Provider State Constants

The library exports constants for checking provider states programmatically:

import trustedProviders, {
  PROVIDER_STATE_READY,
  PROVIDER_STATE_LOADING,
  PROVIDER_STATE_ERROR,
  PROVIDER_STATE_STALE,
} from '@headwall/trusted-network-providers';

const status = trustedProviders.getProviderStatus('Stripe API');
if (status.state === PROVIDER_STATE_ERROR) {
  // Handle provider failure
}

CLI

A trusted-lookup command is included for ad-hoc IP checks from the terminal.

Run it without installing, via npx:

# Check a single IP
npx -p @headwall/trusted-network-providers trusted-lookup 66.249.66.87

# Check several at once
npx -p @headwall/trusted-network-providers trusted-lookup 66.249.66.87 1.2.3.4

Or install globally to get the bare command:

npm install -g @headwall/trusted-network-providers
trusted-lookup 66.249.90.77

It prints the matching provider for each IP (or "not trusted"), and exits 0 when every IP is trusted or 1 if any are untrusted — handy for scripts and CI:

if npx -p @headwall/trusted-network-providers trusted-lookup "$ip"; then
  echo "$ip is trusted"
fi

Documentation

• Providers - Built-in provider reference and custom provider guide
• Security - Security features and production recommendations
• DNS Security Guide - DNS/SPF provider security considerations
• Migration Guide - Upgrading from v1.x to v2.x

Example: Custom Provider

import trustedProviders from '@headwall/trusted-network-providers';

trustedProviders.addProvider({
  name: 'My Custom Network',
  testAddresses: ['10.0.0.1'],
  ipv4: {
    addresses: ['10.0.0.1'],
    ranges: ['10.0.0.0/24'],
  },
  ipv6: {
    addresses: [],
    ranges: [],
  },
});

Example: Express.js Middleware

import express from 'express';
import trustedProviders from '@headwall/trusted-network-providers';

const app = express();

app.use((req, res, next) => {
  const provider = trustedProviders.getTrustedProvider(req.ip);
  if (provider) {
    req.trustedProvider = provider;
    // Skip rate limiting, logging, etc.
  }
  next();
});

Example: Long-Running Service with Monitoring

import trustedProviders from '@headwall/trusted-network-providers';

// Configure logging for production
trustedProviders.setLogLevel('warn');

// Load providers
trustedProviders.loadDefaultProviders();
await trustedProviders.reloadAll();

// Monitor provider health
trustedProviders.on('stale', ({ provider, lastUpdated }) => {
  console.warn(`Provider ${provider} is stale, triggering reload...`);
  trustedProviders.reloadAll();
});

trustedProviders.on('reload:error', ({ provider, error }) => {
  // Alert your monitoring system
  console.error(`Critical: ${provider} reload failed:`, error.message);
});

// Check provider health periodically
setInterval(
  () => {
    const googlebot = trustedProviders.getProviderStatus('Googlebot');
    if (googlebot.state === 'error' || googlebot.state === 'stale') {
      console.warn(`Googlebot health check failed: ${googlebot.state}`);
    }
  },
  60 * 60 * 1000
); // Every hour

Maintenance

Update bundled IP assets before releases:

./scripts/update-assets.sh

License

MIT © Paul Faulkner

Contributing

Issues and pull requests welcome on GitHub.