@thejob/sitemap-client

Typed client for thejob-sitemap-service: the URL enumeration behind the sitemap*.xml routes.

import { SitemapClient } from '@thejob/sitemap-client';

const sitemap = new SitemapClient({ baseUrl: process.env.SITEMAP_SERVICE_URL! });

Auth

None. Every route on this service is public, because crawlers and SSR sitemap routes are the callers. apiKey and token are accepted so the constructor matches the other clients, and for callers sitting behind a gateway that does require a header.

The API

| method | endpoint | |---|---| | countUrls(type) | GET /sitemap/count | | listUrls(type, query?) | GET /sitemap/urls | | health() | GET /health |

type is one of job, company, institute, group (SITEMAP_TYPES). An unknown type throws in-process rather than reaching the service: it would 400 there, and a caller looping over types would log it and still emit a sitemap silently missing a whole content class.

Walking a type

Prefer the cursor, not page. Offset paging re-scans and re-skips on every call, so enumerating hundreds of thousands of jobs gets slower the further it goes, and a document inserted mid-walk shifts the offsets and can duplicate or skip a row.

let cursor: string | undefined;
do {
  const { results, nextCursor } = await sitemap.listUrls('job', { limit: 1000, cursor });
  for (const entry of results) emit(entry);
  cursor = nextCursor ?? undefined;
} while (cursor);

nextCursor comes back null on the last page, which is the only reliable stop signal.

Two things that bite

• limit is clamped service-side to 1..5000. Asking for more is not an error and does not return more, so code that assumes its own limit was honoured will under-enumerate. Read results.length.
• updatedAt / createdAt are epoch milliseconds, and either can be absent. A <lastmod> builder has to handle undefined rather than formatting NaN.

Counts are filtered, not collection sizes

countUrls counts the same filter listUrls enumerates, so the two agree. Each type carries its own crawlability rules: jobs are published and from roughly the last three months, pages are published, groups are active + public + discoverable.

Development

pnpm build      # tsc -p tsconfig.json
pnpm test       # vitest run

The test suite reads the service's own route files and asserts this client has a method for every endpoint they register, so an endpoint added without a client method fails here.