scrutari
v0.1.0
Published
A website crawler gathers URLs with the same origin on a site. Available as both a CLI tool and a JavaScript library.
Downloads
7
Maintainers
Readme
WIP !!!!
Scrutari
Scrutari is a Latin deponent verb meaning "to search," "probe," "examine carefully," or "investigate".
A method to compile a list of URLs based on robot files, sitemaps, and page crawling.
✨ Features
- Dependency-light
- Plays nicely with existing workflows
📦 Installation
npm install scrutari
Usage
CLI
Simple output
URL=[website] scrutari
# URL=https://www.sunsweet.com scrutari
Options
Verbose
URL=[website] scrutari -v
Output
URL=[website] scrutari -o
Piping to another function
URL=[website] scrutari | [script]
Importing
ES Module
import scrutari from "scrutari";
CommonJs Module
const scrutari = require("scrutari");
