Overview

URL and query implementation for JS. Like built-in URL but actually usable. Features:

• Somewhat aligned with URL API.
• Almost everything is optional. In particular:
  • .protocol is optional.
  • .pathname is optional.
• Various common-sense shortcuts.
  • Fluent builder-style API.
  • Support for correctly joining/appending URL paths.
  • Support for traditional "query dictionaries" like {key: ['val']}.
  • Support for patching/merging queries.
• Better compatibility with custom URL schemes.
• Less information loss.
  • No magic defaults, fallbacks, automatic appending, or automatic prepending.
  • .pathname is preserved from input exactly as-is.
  • Empty .origin is '', not 'null'.
• Stricter validation of input types and string formats.
  • Nil is considered '', not 'null' or 'undefined'.
  • Accidental stringification of junk like '[object Object]' is forbidden and causes exceptions.
  • Query keys must be strings. Nil keys are considered missing.
  • Invalid inputs for various URL components cause exceptions instead of being silently converted to garbage, truncated, or ignored.
• Subclassable.
  • Can subclass Search and override it for your Url variant.
  • Can override any getter, setter, or method.
  • Compatible with proxies and Object.create.
  • No "illegal invocation" exceptions.
• No special cases for "known" URL schemes.
• Search is Map<string, string[]> as it should be.
• Automatically stringable as it should be.
• Decent test coverage.
• Decent benchmark coverage.
• Tuned for #performance.
• Browser compatibility: evergreen, Safari 11+.
• Tiny, dependency-free, single file, native module.

TOC

• #Overview
• #Why
• #Perf
• #Usage
• #API
  • #function url
  • #function search
  • #class Url
  • #class Search
  • #Undocumented
• #Limitations
• #License
• #Misc

Why

The JS built-in URL implementation is insane. I have no other words for it.

Various issues:

• Requires .protocol. WTF. In real app code, both on client and server, many URLs are relative to website origin, without a protocol.
  • This alone can force app authors to either avoid URL, or use hacks involving a fake protocol like file:.
• Empty .origin is 'null' rather than ''. WTF.
  • Even worse: .origin is 'null' for any custom scheme. It works only for a small special-cased whitelist.
• Unwanted garbage by default:
  • Forces empty .pathname for some schemes to be '/' rather than ''.
    • But only for some schemes!
  • Non-empty .hash starts with #, which is often undesirable.
  • Non-empty .search starts with ?, which is often undesirable.
  • I always end up with utility functions for stripping this away.
  • But non-empty .port doesn't start with : because lolgic!
• URL property setters and URLSearchParams methods stringify nil values as some junk rather than ''. null becomes 'null', undefined becomes 'undefined'. In JS, where nil is an automatic fallback for a missing value, this is asinine. Nil should be considered ''.
• No support for appending path segments, which is an extremely common use case. WTF.
  • new URL(<path>, <base>) is not good enough. It requires <base> to have an origin (real website links often don't), and works only if path and base begin/end with the right amount of slashes, forcing app authors to write utility functions for stripping/appending/prepending slashes.
• Made-up component .protocol is unusable.
  • The URI standard defines "scheme" which does not include : or //. The JS URL lacks .scheme; its .protocol includes : but not //, which is the worst possible choice.
  • The lack of // makes it impossible to programmatically differentiate protocols like http:// from protocols like mailto: without a special-case whitelist, which is of course not exposed by this implementation. URLs are a general-purpose structured data format which is extensible, and custom protocols are frequently used. Special-case whitelists should not be required for using your API, or at the very least they must be exposed.
  • The no-less-atrocious Go net/url.URL correctly uses a "scheme" field without :, but makes the same mistake of hiding the knowledge of whether the original string had // in its protocol.
• URLSearchParams is nearly unusable:
  • Garbage inputs → garbage outputs. Nil is converted to 'null' or 'undefined'. Various non-stringable objects are converted to '[object Object]'. This insanity has to stop.
  • Lacks support for traditional "query dictionaries" which are extremely popular in actual apps.
  • Lacks support for patching and merging. Can be emulated by spreading .entries() into constructors which is bulky and inefficient.
  • Lacks various common-sense methods: .setAll, .appendAll, .clear.
  • Can't override url.searchParams with a custom subclass.
  • Instead of being a normal Map<string, string[]>, its iteration methods are bizarre and made-up just for this. Nobody needs this weirdness. This just makes things slower and more surprising.
• Many operations are much slower than possible.

Perf

• Checked with benchmarks.
• Uses various optimizations such as lazy query parsing, string caching, structural copying instead of reparsing.
• Most operations seem to perform significantly better than corresponding built-ins in Deno 1.17 / V8 9.7+.

Usage

In browsers and Deno, import by URL:

import * as u from 'https://cdn.jsdelivr.net/npm/@mitranim/[email protected]/ur.mjs'

When using Node or NPM-oriented bundlers like Esbuild:

npm i -E @mitranim/ur

Example parsing:

const url = u.url(`https://example.com/path?key=val#hash`)

url.pathname         // '/path'
url.search           // 'key=val'
url.hash             // 'hash'
url.query.get(`key`) // 'val'
url.query.dict()     // {key: 'val'}
url.query.dictAll()  // {key: ['val']}

Example segmented path:

u.url(`https://example.com`).setPath(`/api/msgs`, 123, `get`) + ``
// 'https://example.com/api/msgs/123/get'

Example without scheme/protocol:

u.url(`/api`).addPath(`msgs`, 123, `get`) + ``
// '/api/msgs/123/get'

Example query dict support:

u.url(`/profile`).addQuery({action: `edit`}) + ``
// `'/profile?action=edit'

API

function url

Same as #new Url but syntactically shorter.

function search

Same as #new Search but syntactically shorter.

class Url

Like URL but much better. See #Overview for some differences.

type UrlLike       = string | Url | URL | Location
type StrLike       = boolean | number | string
type SearchDictLax = Record<string, string | string[]>
type SearchLike    = string | Search | URLSearchParams | SearchDictLax

class Url {
  constructor(src?: UrlLike)

  // All of the following are getter/setters.
  // Many are covariant with each other.
  scheme:       string // Without ':' or '//'.
  slash:        string // Either '' or '//'.
  username:     string // Without '@'.
  password:     string // Without ':' or '@'.
  hostname:     string
  port:         string
  pathname:     string
  search:       string // Without leading '?'.
  searchParams: Search
  query:        Search
  hash:         string // Without leading '#'.
  protocol:     string
  host:         string
  origin:       string
  href:         string

  // All of the following set the corresponding property,
  // mutating and returning the same `Url` reference.
  // Passing nil clears the corresponding property.
  setScheme       (val?: string): Url
  setSlash        (val?: string): Url
  setUsername     (val?: string): Url
  setPassword     (val?: string): Url
  setHostname     (val?: string): Url
  setPort         (val?: number | string): Url
  setPathname     (val?: string): Url
  setSearch       (val?: string): Url
  setSearchParams (val?: SearchLike): Url
  setQuery        (val?: SearchLike): Url
  setHash         (val?: string): Url
  setHashExact    (val?: string): Url
  setProtocol     (val?: string): Url
  setHost         (val?: string): Url
  setOrigin       (val?: string): Url
  setHref         (val?: string): Url

  // All of these return a clone with the corresponding property updated.
  withScheme       (val?: string): Url
  withSlash        (val?: string): Url
  withUsername     (val?: string): Url
  withPassword     (val?: string): Url
  withHostname     (val?: string): Url
  withPort         (val?: number | string): Url
  withPathname     (val?: string): Url
  withSearch       (val?: string): Url
  withSearchParams (val?: SearchLike): Url
  withQuery        (val?: SearchLike): Url
  withHash         (val?: string): Url
  withHashExact    (val?: string): Url
  withProtocol     (val?: string): Url
  withHost         (val?: string): Url
  withOrigin       (val?: string): Url
  withHref         (val?: string): Url

  // Replace `.pathname` with slash-separated segments.
  // Empty or non-stringable segments cause an exception.
  setPath(...vals: StrLike[]): Url

  // Like `.setPath` but appends to an existing path.
  addPath(...vals: StrLike[]): Url

  // Reinitializes the `Url` object from the input.
  // Mutates and returns the same reference.
  // Passing nil is equivalent to `.clear`.
  mut(src?: UrlLike): Url

  // Clears all properties. Mutates and returns the same reference.
  clear(): Url

  // Returns a cloned version.
  // Future mutations are not shared.
  // Cheaper than reparsing.
  clone(): Url

  // Converts to built-in `URL`, for compatibility with APIs that require it.
  toURL(): URL

  // Same as `.href`. Enables automatic JS stringification.
  toString(): string

  // Enables automatic JSON string encoding.
  // As a special case, empty url is considered null.
  toJSON(): string | null

  // All of these are equivalent to `.toString()`. This object may be considered
  // a primitive/scalar, equivalent to a string in some contexts.
  valueOf(): string
  [Symbol.toPrimitive](hint?: string): string

  // Class used internally for instantiating `.searchParams`.
  // Can override in subclass.
  get Search(): {new(): Search}

  // Shortcut for `new this(val).setPath(...vals)`.
  static join(val: UrlLike, ...vals: StrLike[]): Url
}

Warning: this library does not support parsing bare-domain URLs like example.com without a scheme. They cannot be syntactically distinguished from a bare pathname, which is a more important use case. However, Url does provide a shortcut for generating a string like this:

u.url(`https://example.com/path`).hostPath() === `example.com/path`
u.url(`scheme://host:123/path?key=val#hash`).hostPath() === `host:123/path`

class Search

Like URLSearchParams but much better. See #Overview for some differences.

type StrLike          = boolean | number | string
type SearchDictLax    = Record<string, string | string[]>
type SearchDictSingle = Record<string, string>
type SearchDictMulti  = Record<string, string[]>
type SearchLike       = string | Search | URLSearchParams | SearchDictLax

class Search extends Map<string, string[]> {
  constructor(src?: SearchLike)

  // Similar to the corresponding methods of `URLSearchParams`,
  // but with stricter input validation. In addition, instead of
  // returning void, they return the same reference for chaining.
  // A nil key is considered missing, and the operation is a nop.
  // A nil val is considered to be ''.
  has(key?: string): boolean
  get(key?: string): string | undefined
  getAll(key?: string): string[]
  set(key?: string, val?: StrLike): Search
  append(key?: string, val?: StrLike): Search
  delete(key?: string): boolean

  // Common-sense methods missing from `URLSearchParams`.
  // Names and signatures are self-explanatory.
  setAll(key?: string, vals?: StrLike[]): Search
  setAny(key?: string, val?: StrLike | StrLike[]): Search
  appendAll(key?: string, vals?: StrLike[]): Search
  appendAny(key?: string, val?: StrLike | StrLike[]): Search

  // Reinitializes the `Search` object from the input.
  // Mutates and returns the same reference.
  // Passing nil is equivalent to `.clear`.
  mut(src?: SearchLike): Search

  // Appends the input's content to the current `Search` object.
  // Mutates and returns the same reference.
  add(src?: SearchLike): Search

  // Combination of `.get` and type conversion.
  // Nil if property is missing or can't be converted.
  bool(key?: string): boolean | undefined
  int(key?: string): number | undefined
  fin(key?: string): number | undefined

  // Conversion to a traditional "query dictionary".
  dict(): SearchDictSingle
  dictAll(): SearchDictMulti

  // Returns a cloned version.
  // Future mutations are not shared.
  // Cheaper than reparsing.
  clone(): Search

  // Converts to built-in search params.
  // Note that `new URLSearchParams(<u.Search>)` should be avoided.
  toURLSearchParams(): URLSearchParams

  // Same as `.toString` but prepends '?' when non-empty.
  toStringFull(): string

  // Encodes to a string like 'key=val'.
  // Enables automatic JS stringification.
  // Uses caching: if not mutated between calls, this is nearly free.
  toString(): string

  // Enables automatic JSON string encoding.
  // As a special case, empty url is considered null.
  toJSON(): string | null
}

Warning: while Search is mostly compatible with URLSearchParams, it has different iteration methods. The iteration methods of URLSearchParams are something bizarre and made-up just for this type:

[...new URLSearchParams(`one=two&one=three&four=five`)]
// [[`one`, `two`], [`one`, `three`], [`four`, `five`]]

Meanwhile Search is Map<string, string[]>:

[...new u.Search(`one=two&one=three&four=five`)]
// [[`one`, [`two`, `three`]], [`four`, [`five`]]]

The following works properly:

new u.Search(new URLSearchParams(`one=two&one=three&four=five`))
new u.Search(`one=two&one=three&four=five`).toURLSearchParams()

But the following does not work properly and should be avoided:

new URLSearchParams(new u.Search(`one=two&one=three&four=five`))

Undocumented

Some APIs are exported but undocumented to avoid bloating the docs. Check the source files and look for export.

Limitations

• Url lacks support for optional base URL. Constructor takes only 1 value.
• Search iterates as Map<string, string[]>, not as URLSearchParams.

License

https://unlicense.org

Misc

I'm receptive to suggestions. If this library almost satisfies you but needs changes, open an issue or chat me up. Contacts: https://mitranim.com/#contacts