SnapCrawl Express Middleware

This middleware intercepts requests from known bots (Googlebot, ChatGPT, Bing, etc.) and serves pre-rendered HTML content from SnapCrawl's SSR API. This ensures optimal SEO for JavaScript-heavy applications like React, Vue, Angular SPAs.

Features

• Detects popular bots via User-Agent
• Detects desktop vs mobile crawlers
• Skips rendering for static assets (like .js, .png, .css, etc.)
• Fetches HTML from SnapCrawl API and serves to the bot

Free Credits, Billing & Your API Secret

🔑 Get your API secret

1. Create an account (or sign in): snapcrawl.io
2. Go to your Dashboard → Secret keys
3. Copy your Secret Key

Installation

npm install snapcrawl-express

Usage

import express from "express";
import snapcrawlMiddleware from "snapcrawl-express";

const app = express();

app.use(
  snapcrawlMiddleware({
    secret: "your-snapcrawl-api-secret",
  })
);

app.listen(3000);

How It Works

1. Middleware checks the User-Agent header to identify if it's a bot.
2. Skips requests for static assets like .js, .jpg, etc.
3. If it's a bot requesting an HTML route, it sends a request to SnapCrawl's SSR API.
4. The API returns the pre-rendered HTML and serves it directly to the bot.

List of Known Bots

A sample of detected bots includes:

• Googlebot
• Bingbot
• GPTBot
• FacebookExternalHit
• TwitterBot
• ChatGPT-User
• DiscordBot
• and many more...

Options

| Option | Type | Required | Description | | -------- | ------ | -------- | ------------------------------ | | secret | String | ✅ | Your SnapCrawl API secret key. |