PS-Remix-Crawlers

🚀 Getting Started

Installation

npm i @pallasoft/ps-remix-crawlers

Usage

Runtime Generation

1. Create a sitemap[.]xml route

import { routes } from '@remix-run/dev/server-build';
import { LoaderFunctionArgs } from '@remix-run/node';

import { createSEOGenerator } from '@pallasoft/ps-remix-crawlers';

export const loader = async ({ request }: LoaderFunctionArgs) => {
	const url = new URL(request.url);
	const { sitemap } = createSEOGenerator({
		siteUrl: url.origin,
	});

	return sitemap(request, routes);
};

2. Create a robots[.]txt route

import { LoaderFunctionArgs } from '@remix-run/node';

import { createSEOGenerator } from '@pallasoft/ps-remix-crawlers';

export const loader = async ({ request }: LoaderFunctionArgs) => {
	const url = new URL(request.url);
	const { robots } = createSEOGenerator({
		siteUrl: url.origin,
	});

	return robots();
};

📝 Guides

Generate Sitemap for Dynamic Routes

import type { SitemapFunction } from '@pallasoft/ps-remix-crawlers';

export const sitemap: SitemapFunction = ({ config, request }) => {
	const posts = await getPosts();

	return posts.map((post) => ({
		loc: `/posts/${post.slug}`,
		lastmod: post.updatedAt,
		exclude: post.isDraft, // exclude this entry
		// acts only in this loc
		alternateRefs: [
			{
				href: `${config.siteUrl}/en/posts/${post.slug}`,
				absolute: true,
				hreflang: 'en',
			},
			{
				href: `${config.siteUrl}/es`,
				hreflang: 'es',
			},
		],
	}));
};

Exclude Route

import type { SitemapFunction } from 'remix-sitemap';

export const sitemap: SitemapFunction = () => ({
	exclude: true,
});

Caching

createSEOGenerator({
	siteUrl: 'https://example.com',
	headers: {
		'Cache-Control': 'max-age=3600',
	},
});

Google: News, Image and Video

Url set can contain additional sitemaps defined by google. These are Google news, image or video. You can add these sitemaps in sitemap function by adding the news, images or videos property.

export const sitemap: SitemapFunction = () => ({
	images: [{ loc: 'https://example.com/example.jpg' }],
	news: [
		{
			title: 'Random News',
			date: '2023-03-15',
			publication: {
				name: 'The Example Times',
				language: 'en',
			},
		},
	],
});

📖 API Reference

Config

• siteUrl: the website base url
• autoLastmod = true: (optional) Add <lastmod /> property with the current date.
• priority = 0.7: (optional) default priority for all entries.
• changefreq = 'daily': (optional) default changefreq for all entries.
• format = false: (optional) format the sitemap for a better view.
• alternateRefs = []: (optional) default multi language support by unique url for all entries.
• generateRobotsTxt = false: (optional) generate robots.txt file.
• robotsTxtOptions: (optional) options for generating robots.txt details.
• rateLimit: (optional) limits the number of sitemap functions that can execute at once.

Runtime only

• headers = {}: (optional) headers for the sitemap and robots.txt response.
• cache: (optional) cache the sitemap details.

Build Time only

• generateIndexSitemap = true: (optional) generate index sitemap.
• sitemapBaseFileName = 'sitemap': (optional) the name of the generated sitemap file before the file extension.
• outDir = 'public': (optional) the directory to create the sitemaps files.
• size = 50000: (optional) max size of the sitemap.

RobotsTxtOptions

• policies = []: (optional) policies for generating robots.txt.
• additionalSitemaps = []: (optional) add additionals sitemaps to robots.txt.