@tpmjs/tools-url-normalize

Normalize URLs with configurable options for consistent URL handling.

Installation

npm install @tpmjs/tools-url-normalize

Usage

import { urlNormalizeTool } from '@tpmjs/tools-url-normalize';

// Use with AI SDK
const result = await urlNormalizeTool.execute({
  url: 'HTTPS://WWW.Example.COM:443/path/?z=1&a=2#section',
  options: {
    sortParams: true,
    removeHash: true,
    lowercase: true,
    removeTrailingSlash: true,
    removeDefaultPort: true,
    removeWWW: true,
  },
});

console.log(result.normalized);
// 'https://example.com/path?a=2&z=1'

console.log(result.changes);
// [
//   { type: 'lowercase', description: '...', before: 'HTTPS:', after: 'https:' },
//   { type: 'lowercase', description: '...', before: 'WWW.Example.COM', after: 'www.example.com' },
//   { type: 'removeWWW', description: '...', before: 'www.example.com', after: 'example.com' },
//   { type: 'removeDefaultPort', description: '...', before: '443', after: '' },
//   { type: 'sortParams', description: '...', before: '?z=1&a=2', after: '?a=2&z=1' },
//   { type: 'removeHash', description: '...', before: '#section', after: '' },
// ]

Features

• Lowercase conversion: Converts protocol and hostname to lowercase
• Trailing slash removal: Removes trailing slashes from pathnames
• Query parameter sorting: Sorts query parameters alphabetically
• Hash removal: Optionally removes URL fragments
• Default port removal: Removes standard ports (80, 443, 21)
• WWW removal: Optionally removes www subdomain
• Change tracking: Reports all transformations applied

Parameters

• url (string, required): The URL to normalize
• options (object, optional): Normalization options
  • sortParams (boolean, default: true): Sort query parameters alphabetically
  • removeHash (boolean, default: false): Remove URL fragment/hash
  • lowercase (boolean, default: true): Convert protocol and hostname to lowercase
  • removeTrailingSlash (boolean, default: true): Remove trailing slash from pathname
  • removeDefaultPort (boolean, default: true): Remove default ports
  • removeWWW (boolean, default: false): Remove www subdomain

Returns

{
  normalized: string;
  original: string;
  changes: Array<{
    type: string;
    description: string;
    before: string;
    after: string;
  }>;
  metadata: {
    protocol: string;
    hostname: string;
    pathname: string;
    hasQueryParams: boolean;
    hasHash: boolean;
  };
}

Use Cases

• URL deduplication in web crawlers
• Canonical URL generation for SEO
• Link comparison and matching
• Database indexing of URLs
• Analytics tracking consistency

Example with AI Agent

import { openai } from '@ai-sdk/openai';
import { generateText } from 'ai';
import { urlNormalizeTool } from '@tpmjs/tools-url-normalize';

const result = await generateText({
  model: openai('gpt-4'),
  tools: {
    urlNormalize: urlNormalizeTool,
  },
  prompt: 'Normalize this URL and remove the hash: https://Example.com/PATH/?b=2&a=1#top',
});

License

MIT