sqlite-robotstxt NPM Package

sqlite-robotstxt is distributed on npm for Node.js developers. To install on supported platforms, simply run:

npm install sqlite-robotstxt

The sqlite-robotstxt package is meant to be used with Node SQLite clients like better-sqlite3 and node-sqlite3. For better-sqlite3, call .loadExtension() on your database object, passing in getLoadablePath().

import Database from "better-sqlite3";
import * as sqlite_robotstxt from "sqlite-robotstxt";

const db = new Database(":memory:");

db.loadExtension(sqlite_robotstxt.getLoadablePath());

const version = db.prepare("select robotstxt_version()").pluck().get();
console.log(version); // "v0.2.0"

For node-sqlite3, call the similarly named .loadExtension() method on your database object, and pass in getLoadablePath().

import sqlite3 from "sqlite3";
import * as sqlite_robotstxt from "sqlite-robotstxt";

const db = new sqlite3.Database(":memory:");

db.loadExtension(sqlite_robotstxt.getLoadablePath());

db.get("select robotstxt_version()", (err, row) => {
  console.log(row); // {robotstxt_version(): "v0.2.0"}
});

See the full API Reference for the Node API, and docs.md for documentation on the sqlite-robotstxt SQL API.

Supported Platforms

Since the underlying robotstxt0 SQLite extension is pre-compiled, the sqlite-robotstxt NPM package only works on a few "platforms" (operating systems + CPU architectures). These platforms include:

• darwin-x64 (MacOS x86_64)
• darwin-arm64 (MacOS M1 and M2 chips)
• win32-x64 (Windows x86_64)
• linux-x64 (Linux x86_64)

To see which platform your machine is, check the process.arch and process.platform values like so:

$ node -e 'console.log([process.platform, process.arch])'
[ 'darwin', 'x64' ]

When the sqlite-robotstxt NPM package is installed, the correct pre-compiled extension for your operating system and CPU architecture will be downloaded from the optional dependencies, with platform-specific packages like sqlite-robotstxt-darwin-x64. This will be automatically, there's no need to directly install those packages.

More platforms may be supported in the future. Consider supporting my work if you'd like to see more operating systems and CPU architectures supported in sqlite-robotstxt.

API Reference

# getLoadablePath <>

Returns the full path to where the sqlite-robotstxt should be installed, based on the sqlite-robotstxt's package.json optional dependencies and the host's operating system and architecture.

This path can be directly passed into better-sqlite3's .loadExtension().

import Database from "better-sqlite3";
import * as sqlite_robotstxt from "sqlite-robotstxt";

const db = new Database(":memory:");
db.loadExtension(sqlite_robotstxt.getLoadablePath());

It can also be used in node-sqlite3's .loadExtension().

import sqlite3 from "sqlite3";
import * as sqlite_robotstxt from "sqlite-robotstxt";

const db = new sqlite3.Database(":memory:");
db.loadExtension(sqlite_robotstxt.getLoadablePath());

This function throws an Error in two different cases. The first case is when sqlite-robotstxt is installed and run on an unsupported platform. The second case is when the platform-specific optional dependency is not installed. If you reach this, ensure you aren't using --no-optional flag, and file an issue if you are stuck.

The db.loadExtension() function may also throw an Error if the compiled extension is incompatible with your SQLite connection for any reason, including missing system packages, outdated glib versions, or other misconfigurations. If you reach this, please file an issue.