Novel Crawler

A powerful tool to download online novels and save them as EPUB or JSON files.

Installation

# Using npm
npm install -g novel-crawler-vn

# Using npx (no installation required)
npx novel-crawler-vn

Quick Start

npx novel-crawler-vn "https://your-novel-url" 

Features

• Downloads novels from various Vietnamese and English novel sites
• Supports EPUB and JSON output formats
• Concurrent chapter downloading
• Cloudflare bypass support via FlareSolverr
• Split large books into multiple parts
• Progress tracking and status updates

Command Line Usage

npx novel-crawler-vn [novelUrl] [options]

Options

| Option | Alias | Description | Default | |--------|-------|-------------|---------| | --output | -o | Output directory path | "." | | --format | -f | Output format (epub or json) | "epub" | | --concurrency | -c | Number of concurrent downloads | 5 | | --flaresolverr | --fs | FlareSolverr API URL | - | | --chapterPerPart | --cpp | Chapters per split part | 0 | | --list | -l | List supported sites | - | | --help | - | Show help | - | | --version | - | Show version | - |

Supported Sites

View all supported sites using:

npx novel-crawler-vn --list

Currently supports:

• TruyenYY
• TruyenConnect
• SSTruyen
• DTruyen.net
• TruyenFull
• TienVuc
• TruyenChuFull
• TangThuVien
• DocTruyenLN
• LightNovelWorld
• MetruyenCV
• NetTruyen

Programmatic Usage

Installation

# Using npm
npm install novel-crawler-vn

# Using yarn
yarn add novel-crawler-vn

# Using pnpm
pnpm add novel-crawler-vn

Basic Usage

import { crawler } from 'novel-crawler-vn';

await crawler({
  novelUrl: 'https://example.com/novel/my-novel'
});

Advanced Usage

import { CrawlerManager, OutputFormat } from 'novel-crawler-vn';

const manager = new CrawlerManager({
  processConcurrency: 2,    // Parallel novel processing
  crawlerConcurrency: 5,    // Parallel chapter downloads
  format: OutputFormat.EPUB,
  chaptersPerPart: 50,      // Split into parts
  flareSolverrServer: 'http://localhost:8191/v1'  // Optional
});

// Download novel
await manager.downloadNovel('https://example.com/novel/my-novel');

// Get supported sites
const sites = manager.getSupportedCrawlers();

// Monitor progress
manager.$subscribe.on('qUpdate', () => {
  const status = manager.getQueueStatus();
  console.log('Progress:', status);
});