link-harvester

📖 中文文档 (Chinese README)

Features

• Extracts Markdown links, Markdown images, HTML <a> anchors, and HTML <img> tags
• Classifies each link's target: external page, external resource, local resource, in-page anchor, or other
• Fluent pipeline API: gather → filter / detect → classify
• Built-in detect(Accessible) — checks whether local files actually exist on disk
• Built-in detect(ExternalRefs) — finds other Markdown files in the same base directory that reference the same local asset
• Automatic pipeline optimisation: duplicate detect ops are deduplicated; consecutive filter ops are merged

Requirements

• Node.js ≥ 18.0.0

Installation

npm install link-harvester

Quick Start

import { LinkHarvester, LinkType, LinkTarget, DetectType, REST_KEY } from 'link-harvester';
import { resolve } from 'node:path';

const harvester = new LinkHarvester({
  base: resolve('./docs'),   // absolute path to the root directory
  filePath: 'guide/intro.md' // relative (or absolute) path to the target file
});

// Gather all links from the file
const links = await harvester.gather();
console.log(links);

API

new LinkHarvester({ base, filePath })

Creates a harvester instance for a single Markdown file.

| Option | Type | Description | |---|---|---| | base | string | Absolute path to the root directory. Used to resolve relative file paths and to scope cross-reference scanning. | | filePath | string | Path to the target Markdown file. May be relative to base or absolute (must be inside base). |

Throws Error when:

• base is not a string, not an absolute path, or does not exist
• filePath is not a string, does not exist, or is outside base

.gather() → LinkDataPipeline

Reads and parses the target file. Must be the first call in every pipeline chain. Returns a LinkDataPipeline for further operations.

const links = await harvester.gather();
// → ExtractedLink[]

.filter(predicate) → LinkDataPipeline

Keeps only links for which predicate(link) returns true. Multiple .filter() calls are merged into a single AND check.

const links = await harvester
  .gather()
  .filter(l => l.linkTarget === LinkTarget.LocalResource);

Throws TypeError if predicate is not a function.

.filterBy(type) → LinkDataPipeline

Shorthand for filtering by a LinkType or LinkTarget enum value.

// By link type
await harvester.gather().filterBy(LinkType.MarkdownImage);

// By link target
await harvester.gather().filterBy(LinkTarget.ExternalPage);

Throws TypeError for any unrecognised value.

.detect(detectType) → LinkDataPipeline

Runs a detection pass that enriches each ExtractedLink with additional fields. Only affects LocalResource links.

| detectType | Effect | |---|---| | DetectType.Accessible | Sets link.accessible: boolean — true if the file exists and is readable. | | DetectType.ExternalRefs | Sets link.externalRefs: string[] — relative paths of other .md/.markdown files in base that reference the same local asset. |

import { DetectType } from 'link-harvester';

// Check file accessibility
const links = await harvester
  .gather()
  .detect(DetectType.Accessible);

// Find cross-file references
const links = await harvester
  .gather()
  .detect(DetectType.ExternalRefs);

Throws TypeError for an unrecognised detectType.

.classify(buckets) → ThenPipeline

Partitions links into named buckets. Returns a ThenPipeline (thenable, no further chaining).

Each value in buckets must be either:

• a predicate function (link: ExtractedLink) => boolean, or
• the REST_KEY constant ("rest") — collects every link not matched by any named bucket.

At most one bucket may use REST_KEY.

import { REST_KEY } from 'link-harvester';

const result = await harvester
  .gather()
  .classify({
    images: l => l.type === LinkType.MarkdownImage || l.type === LinkType.HtmlImage,
    pages:  l => l.linkTarget === LinkTarget.ExternalPage,
    rest:   REST_KEY,
  });

console.log(result.images); // ExtractedLink[]
console.log(result.pages);  // ExtractedLink[]
console.log(result.rest);   // ExtractedLink[]  (everything else)

Throws TypeError when:

• buckets is not a plain object
• buckets is empty
• more than one bucket uses REST_KEY
• any bucket value is neither a function nor REST_KEY

Pipeline Composition

Operations can be freely composed before .classify() terminates the chain.

gather()
  → .filter()       zero or more, merged into a single AND predicate
  → .detect()       zero or more, deduplicated automatically
  → .filterBy()     shorthand for .filter() by type or target
  → .classify()     optional terminal — switches result to Record<string, ExtractedLink[]>

Examples

Filter then classify

const result = await harvester
  .gather()
  .filter(l => l.type === LinkType.MarkdownImage || l.type === LinkType.MarkdownLink)
  .classify({
    images: l => l.type === LinkType.MarkdownImage,
    rest: REST_KEY,
  });

Detect accessibility then filter broken links

const broken = await harvester
  .gather()
  .detect(DetectType.Accessible)
  .filter(l => l.linkTarget === LinkTarget.LocalResource && l.accessible === false);

Find shared assets referenced by other files

const shared = await harvester
  .gather()
  .filter(l => l.linkTarget === LinkTarget.LocalResource)
  .detect(DetectType.ExternalRefs)
  .filter(l => l.externalRefs.length > 0);

Full pipeline: filter → detect → classify

const result = await harvester
  .gather()
  .filter(l => l.linkTarget === LinkTarget.LocalResource)
  .detect(DetectType.ExternalRefs)
  .classify({
    shared:  l => l.externalRefs.length > 0,
    private: REST_KEY,
  });

Data Types

ExtractedLink

All extracted links share these fields:

| Field | Type | Description | |---|---|---| | type | LinkType | Syntax type of the link | | linkTarget | LinkTarget | Classified target of the URL | | url | string | The raw URL string | | syntax | string | The full matched syntax fragment | | line | number | 1-based line number in the source file | | accessible | boolean \| undefined | Set by detect(Accessible) on LocalResource links | | externalRefs | string[] \| undefined | Set by detect(ExternalRefs) on LocalResource links |

Markdown-specific subtypes add:

| Subtype | Extra field | Description | |---|---|---| | MarkdownLink | text: string | Link display text | | MarkdownImageLink | alt: string | Image alt text |

LinkType

| Value | Syntax | |---|---| | LinkType.MarkdownLink | [text](url) | | LinkType.MarkdownImage | ![alt](url) | | LinkType.HtmlImage | <img src="url"> | | LinkType.HtmlAnchor | <a href="url"> |

LinkTarget

| Value | Condition | |---|---| | LinkTarget.ExternalPage | http(s):// URL without a resource file extension | | LinkTarget.ExternalResource | http(s):// URL with a resource file extension (image, pdf, zip, …) | | LinkTarget.LocalResource | Relative path with no scheme | | LinkTarget.InPageAnchor | Starts with # | | LinkTarget.Other | mailto:, ftp:, empty, or other schemes |

DetectType

| Value | Description | |---|---| | DetectType.Accessible | Check if the local file is readable | | DetectType.ExternalRefs | Find other Markdown files that reference the same local asset |

Low-level API: extractLinks

The underlying parser is also exported for direct use:

import { extractLinks } from 'link-harvester';

const links = await extractLinks('/absolute/path/to/file.md');
// → ExtractedLink[]

License

MIT © isaaxite