@bernierllc/validators-secret-patterns

Primitive validator for detecting hardcoded secrets and sensitive patterns in code - API keys, tokens, passwords, private keys, database credentials, and high-entropy strings.

Installation

npm install @bernierllc/validators-secret-patterns

Features

• API Key Detection - AWS, GitHub, Google, Stripe, SendGrid, Twilio, Slack, NPM, and more
• High-Entropy String Detection - Identifies potential secrets based on entropy analysis
• JWT Token Detection - Detects hardcoded JSON Web Tokens
• Private Key Detection - RSA, SSH, PGP, EC, OpenSSH, and other private key formats
• Password Pattern Detection - Identifies hardcoded passwords in various formats
• Database Connection Strings - PostgreSQL, MySQL, MongoDB, Redis, SQL Server, Oracle
• Custom Pattern Support - Add your own secret detection patterns
• Configurable Detection - Enable/disable specific checks as needed
• High-Precision Matching - Minimizes false positives with smart filtering

Usage

Basic Validation

import { validateSecretPatterns } from '@bernierllc/validators-secret-patterns';

const code = `
  const awsKey = "AKIAIOSFODNN7EXAMPLE";
  const password = "MySecretP@ssw0rd123";
`;

const problems = await validateSecretPatterns(code);

if (problems.length > 0) {
  problems.forEach(problem => {
    console.log(`${problem.severity}: ${problem.message}`);
    console.log(`  at line ${problem.location?.line}, column ${problem.location?.column}`);
  });
}

With Custom Options

import { validateSecretPatterns } from '@bernierllc/validators-secret-patterns';

const problems = await validateSecretPatterns(sourceCode, {
  // Entropy detection settings
  minEntropyScore: 4.5,      // Higher = stricter (0-8 scale)
  minEntropyLength: 20,       // Minimum length for entropy analysis

  // Enable/disable specific checks
  checkApiKeys: true,
  checkEntropy: true,
  checkJwtTokens: true,
  checkPrivateKeys: true,
  checkPasswords: true,
  checkDatabaseStrings: true,

  // Custom patterns
  customPatterns: [
    {
      pattern: /CUSTOM-SECRET-\d+/g,
      name: 'Custom Secret Format',
      severity: 'error'
    }
  ],

  // Exclude known safe patterns (e.g., test data)
  excludePatterns: [
    /example\.com/,
    /test-key-\d+/
  ]
});

Quick Secret Check

import { hasSecrets } from '@bernierllc/validators-secret-patterns';

if (await hasSecrets(code)) {
  console.log('⚠️  Secrets detected in code!');
}

Get Secret Counts

import { getSecretCounts } from '@bernierllc/validators-secret-patterns';

const counts = await getSecretCounts(code);
console.log(`Found ${counts.total} secrets:`);
console.log(`  - ${counts.apiKeys} API keys`);
console.log(`  - ${counts.passwords} passwords`);
console.log(`  - ${counts.privateKeys} private keys`);
console.log(`  - ${counts.databaseStrings} database connections`);
console.log(`  - ${counts.jwtTokens} JWT tokens`);
console.log(`  - ${counts.entropy} high-entropy strings`);

Using Individual Rules

import { apiKeysRule, passwordsRule } from '@bernierllc/validators-secret-patterns';
import { createRuleContext } from '@bernierllc/validators-core';

const problems = [];
const context = createRuleContext(
  'secret-patterns/api-keys',
  { excludePatterns: [/test-key/] },
  utils,
  {},
  (p) => problems.push(p)
);

const validator = apiKeysRule.create(context);
await validator(sourceCode);

console.log(problems);

API Reference

validateSecretPatterns(content, options?, utils?)

Validates content for hardcoded secrets and sensitive patterns.

Parameters:

• content (string): Content to validate (source code, configuration, etc.)
• options (object, optional): Validation options
  • minEntropyScore (number, default: 4.5): Minimum entropy score (0-8)
  • minEntropyLength (number, default: 20): Minimum length for entropy analysis
  • checkApiKeys (boolean, default: true): Check for API keys
  • checkEntropy (boolean, default: true): Check for high-entropy strings
  • checkJwtTokens (boolean, default: true): Check for JWT tokens
  • checkPrivateKeys (boolean, default: true): Check for private keys
  • checkPasswords (boolean, default: true): Check for passwords
  • checkDatabaseStrings (boolean, default: true): Check for database connections
  • customPatterns (array, optional): Custom patterns to detect
  • excludePatterns (array, optional): Patterns to exclude/allowlist
• utils (SharedUtils, optional): Shared utilities from validators-utils

Returns: Promise<Problem[]> - Array of validation problems

hasSecrets(content, options?)

Quick check if content contains any secrets.

Parameters:

• content (string): Content to check
• options (object, optional): Same as validateSecretPatterns

Returns: Promise<boolean> - true if secrets detected

getSecretCounts(content, options?)

Get count of secrets by type.

Parameters:

• content (string): Content to analyze
• options (object, optional): Same as validateSecretPatterns

Returns: Promise<object> - Object with counts by type:

• apiKeys (number)
• entropy (number)
• jwtTokens (number)
• privateKeys (number)
• passwords (number)
• databaseStrings (number)
• custom (number)
• total (number)

Validation Rules

secret-patterns/api-keys

Detects hardcoded API keys and tokens from various services.

Detects:

• AWS Access Keys (AKIA..., ASIA...)
• AWS Secret Keys
• GitHub Personal Access Tokens (ghp_, gho_, ghr_)
• Google API Keys (AIza...)
• Google OAuth Tokens (ya29.)
• Stripe API Keys (sk_live, pk_live)
• SendGrid API Keys (SG....)
• Twilio API Keys (SK...)
• Slack Tokens (xoxb-, xoxp-)
• Slack Webhooks
• Mailgun API Keys
• Mailchimp API Keys
• NPM Tokens (npm_...)
• Azure Storage Keys
• Heroku API Keys (UUID format)
• Generic API key patterns

Example:

// ❌ Detected
const awsKey = "AKIAIOSFODNN7EXAMPLE";
const githubToken = "ghp_1234567890abcdefghijklmnopqrstuv";
const stripeKey = "sk_live_1234567890abcdefghijklmn";

secret-patterns/entropy

Detects high-entropy strings that may indicate hardcoded secrets.

Algorithm:

• Calculates Shannon entropy (0-8 scale)
• Requires mix of character types (uppercase, lowercase, numbers, special chars)
• Filters out common false positives (UUIDs, URLs, package names, hex colors)

Configuration:

• minEntropyScore (default: 4.5): Minimum entropy threshold
• minEntropyLength (default: 20): Minimum string length

Example:

// ⚠️  Detected (high entropy)
const secret = "aB3dEfGhIjKlMnOpQrStUvWxYz123456789";

// ✅ Not detected (UUID format)
const id = "550e8400-e29b-41d4-a716-446655440000";

// ✅ Not detected (URL)
const url = "https://api.example.com/v1/users";

secret-patterns/jwt-tokens

Detects hardcoded JSON Web Tokens (JWT).

Detection:

• Validates three base64url segments separated by dots
• Decodes header to verify JWT format
• Extracts algorithm and token type

Example:

// ❌ Detected
const token = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c";

secret-patterns/private-keys

Detects hardcoded private keys in various formats.

Detects:

• RSA Private Keys
• Generic Private Keys (PKCS#8)
• EC Private Keys (Elliptic Curve)
• DSA Private Keys
• OpenSSH Private Keys
• PGP/GPG Private Keys
• Encrypted Private Keys

Example:

// ❌ Detected
const privateKey = `-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEA0Z3VS5JJcds3xvn6bBHs6
-----END RSA PRIVATE KEY-----`;

secret-patterns/passwords

Detects hardcoded passwords in various contexts.

Patterns:

• Variable assignments (password = "...")
• JSON/config files ("password": "...")
• Connection strings (password=...)
• Basic authentication headers

False Positive Filtering:

• Excludes common placeholders (password, your_password)
• Excludes environment variable references (${PASSWORD})
• Excludes single words and simple patterns
• Excludes example/test data

Example:

// ❌ Detected
const password = "MySecretP@ssw0rd123";
{"password": "SecurePass123!"}
password=MySecretPass123

// ✅ Not detected (placeholders)
const password = "password";
const password = "${PASSWORD}";
const password = "example";

secret-patterns/database-strings

Detects database connection strings with embedded credentials.

Detects:

• PostgreSQL (postgresql://user:pass@host/db)
• MySQL (mysql://user:pass@host/db)
• MongoDB (mongodb://user:pass@host/db)
• Redis (redis://user:pass@host)
• SQL Server (Server=...;Password=...)
• Oracle (Data Source=...;Password=...)
• ODBC (DRIVER=...;PWD=...)
• Generic database URLs

False Positive Filtering:

• Excludes placeholder usernames/passwords
• Excludes environment variable references
• Excludes example domains (example.com, test.com)

Example:

// ❌ Detected
const db = "postgresql://admin:secretpass123@localhost:5432/mydb";
const mongo = "mongodb://user:pass123@dbhost:27017/database";
Server=myserver;Database=mydb;Password=sqlpass123;

// ✅ Not detected (placeholders)
const db = "postgresql://username:password@localhost:5432/db";
const db = "postgresql://user:[email protected]:5432/db";

Problem Structure

Each validation problem includes:

interface Problem {
  ruleId: string;           // e.g., "secret-patterns/api-keys"
  message: string;          // Human-readable description
  severity: 'error' | 'warn' | 'info' | 'off';
  domain: 'security';       // Validation domain
  location?: {              // Optional location information
    line?: number;
    column?: number;
  };
  suggestion?: string;      // How to fix the issue
  fixable?: boolean;        // Whether issue is auto-fixable
  evidence?: {              // Supporting evidence
    snippet?: string;       // Code snippet (with masking for sensitive data)
    context?: Record<string, unknown>;  // Additional context
  };
  tags?: string[];          // Additional categorization
}

Examples

Pre-commit Hook Integration

import { validateSecretPatterns } from '@bernierllc/validators-secret-patterns';
import * as fs from 'fs';
import * as path from 'path';

async function checkForSecrets(filePaths: string[]): Promise<boolean> {
  let hasIssues = false;

  for (const filePath of filePaths) {
    const content = fs.readFileSync(filePath, 'utf-8');
    const problems = await validateSecretPatterns(content);

    if (problems.length > 0) {
      console.error(`❌ Secrets detected in ${filePath}:`);
      problems.forEach(p => {
        console.error(`  ${p.severity}: ${p.message} (line ${p.location?.line})`);
      });
      hasIssues = true;
    }
  }

  return !hasIssues;
}

// Usage in pre-commit hook
const changedFiles = process.argv.slice(2);
const passed = await checkForSecrets(changedFiles);
process.exit(passed ? 0 : 1);

CI/CD Pipeline Integration

import { validateSecretPatterns, getSecretCounts } from '@bernierllc/validators-secret-patterns';
import * as glob from 'glob';
import * as fs from 'fs';

async function scanRepository(): Promise<void> {
  const files = glob.sync('**/*.{ts,js,json,env}', {
    ignore: ['node_modules/**', 'dist/**', '.git/**']
  });

  let totalSecrets = 0;

  for (const file of files) {
    const content = fs.readFileSync(file, 'utf-8');
    const counts = await getSecretCounts(content);

    if (counts.total > 0) {
      console.log(`⚠️  ${file}: ${counts.total} secrets`);
      totalSecrets += counts.total;
    }
  }

  if (totalSecrets > 0) {
    console.error(`\n❌ FAIL: Found ${totalSecrets} hardcoded secrets`);
    process.exit(1);
  } else {
    console.log('\n✅ PASS: No secrets detected');
  }
}

scanRepository().catch(console.error);

IDE Plugin Integration

import { validateSecretPatterns } from '@bernierllc/validators-secret-patterns';

async function lintDocument(document: TextDocument): Promise<Diagnostic[]> {
  const problems = await validateSecretPatterns(document.getText(), {
    // Adjust sensitivity for IDE (fewer false positives)
    minEntropyScore: 5.0,
    checkEntropy: false  // Disable in IDE to reduce noise
  });

  return problems.map(problem => ({
    severity: problem.severity === 'error'
      ? DiagnosticSeverity.Error
      : DiagnosticSeverity.Warning,
    range: {
      start: { line: problem.location?.line || 0, character: problem.location?.column || 0 },
      end: { line: problem.location?.line || 0, character: (problem.location?.column || 0) + 10 }
    },
    message: problem.message,
    source: 'secret-patterns',
    code: problem.ruleId
  }));
}

Custom Pattern Detection

import { validateSecretPatterns } from '@bernierllc/validators-secret-patterns';

const problems = await validateSecretPatterns(code, {
  customPatterns: [
    // Organization-specific secret format
    {
      pattern: /ACME-API-[A-Z0-9]{32}/g,
      name: 'ACME Corp API Key',
      severity: 'error'
    },
    // Internal service tokens
    {
      pattern: /internal_token_[a-f0-9]{40}/g,
      name: 'Internal Service Token',
      severity: 'error'
    },
    // Legacy credential format
    {
      pattern: /legacy_cred=[A-Za-z0-9+/=]{24,}/g,
      name: 'Legacy Credential',
      severity: 'warn'
    }
  ]
});

Allowlist/Exclude Patterns

import { validateSecretPatterns } from '@bernierllc/validators-secret-patterns';

const problems = await validateSecretPatterns(code, {
  excludePatterns: [
    // Test/example keys (documented in README)
    /AKIAIOSFODNN7EXAMPLE/,
    /test-key-[a-z0-9]+/i,

    // Known public demo tokens
    /demo-token-12345/,

    // Development environment keys (non-production)
    /dev-api-key-/i
  ]
});

Integration with Validators Ecosystem

This package follows the BernierLLC Validators Principles:

• Pure Validation Functions - Returns structured problems, never throws
• Atomic Primitives - Each rule has single responsibility
• Tool-Agnostic - Works in CLI, web apps, CI/CD, IDE plugins
• Evidence-Rich Results - Provides actionable debugging information
• Composable - Can be combined with other validators

Used By

• @bernierllc/validators-security - Comprehensive security validation
• @bernierllc/validators-code-quality - Code quality and security checks
• Custom security scanning tools

Integration Status

Logger Integration

Status: not-applicable

This is a primitive validator that returns structured problem arrays. Logging is handled by the consumer application. The validator uses @bernierllc/validators-core for structured problem reporting, which can be integrated with @bernierllc/logger by the consuming application if needed.

Docs-Suite Integration

Status: ready

• Full TypeDoc API documentation generated
• Markdown documentation exported
• Format: markdown, typedoc, jsdoc

NeverHub Integration

Status: not-applicable

This is a primitive validator with no service discovery requirements. The validator uses @bernierllc/neverhub-adapter pattern through the validators-core framework, but direct NeverHub integration (detectNeverHub) is not needed for atomic validation primitives. Service orchestration and discovery are handled at higher levels (domain validators, validator runners).

Performance

• Sub-50ms validation for typical source files (< 10KB)
• Parallel execution of independent rules
• Smart filtering to minimize false positives
• Memory efficient - Streams through content without full AST

Security Best Practices

Prevention Strategies

1. Use Environment Variables

   // ✅ Good
   const apiKey = process.env.API_KEY;
   
   // ❌ Bad
   const apiKey = "AKIAIOSFODNN7EXAMPLE";

2. Use Secret Management Systems

   • AWS Secrets Manager
   • Azure Key Vault
   • HashiCorp Vault
   • Google Secret Manager

3. Implement Pre-commit Hooks

   • Scan all changed files before commit
   • Block commits containing secrets
   • Provide clear remediation guidance

4. CI/CD Integration

   • Scan entire repository on each build
   • Fail builds if secrets detected
   • Generate security reports

5. Developer Training

   • Educate team about secret detection
   • Document proper secret management
   • Regular security awareness training

Remediation Steps

If secrets are detected:

1. Rotate the Secret - Immediately invalidate and regenerate
2. Remove from History - Use git filter-branch or BFG Repo-Cleaner
3. Update Code - Replace with environment variable or secret management
4. Review Access Logs - Check if compromised secret was accessed
5. Document Incident - Record what happened and how it was resolved

Testing

This package includes comprehensive test coverage:

npm test              # Run tests in watch mode
npm run test:run      # Run tests once
npm run test:coverage # Run tests with coverage report

Coverage Requirements:

• Branches: 90%+
• Functions: 90%+
• Lines: 90%+
• Statements: 90%+

See Also

• @bernierllc/validators-core - Core validator types and utilities
• @bernierllc/validators-runner - Execute validators with policies
• @bernierllc/validators-reporters - Format validation results
• @bernierllc/validators-security - Comprehensive security validation

License

Copyright (c) 2025 Bernier LLC. All rights reserved.