@metreeca/pipe-url
v0.11.1
Published
URL processing tasks for @metreeca/pipe.
Maintainers
Readme
@metreeca/pipe-url
URL processing tasks for @metreeca/pipe.
Provides ready-made @metreeca/flow steps for working with resources identified by URLs in pipelines run by the @metreeca/gear job executor.
Installation
npm install @metreeca/gear # the job executor
npm install @metreeca/pipe-url # this package[!IMPORTANT]
Node.js 22 or later is required.
[!WARNING]
TypeScript consumers must use
"moduleResolution": "nodenext"/"node16"/"bundler"intsconfig.json. The legacy"node"resolver is not supported.
Usage
| Task | Description |
|--------------------|--------------------------------|
| fetch() | Resource fetcher |
| crawl() | URL graph walker and harvester |
Support
- open an issue to report a problem or to suggest a new feature
- start a discussion to ask a how-to question or to share an idea
License
This project is licensed under the Apache 2.0 License – see LICENSE file for details.
