@chronocide/robots-txt
v0.0.4
Published
Tiny robots.txt parser
Readme
robots-txt is a tiny TypeScript robots.txt parser and matcher. It's based on Google's robotstxt project.
Installation
npm i @chronocide/robots-txtUsage
import robots from '@chronocide/robots-txt';
const allowed = robots(`
user-agent: FooBot
disallow: /
`);
allowed('FooBot')('/'); // False