@glyphmoe/sdk

SDK for building Glyph extensions. Provides HTTP helpers, HTML parsing, rate limiting, retries, cookie management, and the source factory.

Install

npm install @glyphmoe/sdk cheerio

cheerio is a peer dependency (used for HTML parsing).

Quick Start

import { createSource, get, load, buildUrl, RateLimit } from '@glyphmoe/sdk'

const BASE = 'https://example-novels.com'

export default createSource({
  id: 'example',
  name: 'Example Novels',
  baseUrl: BASE,
  icon: `${BASE}/favicon.ico`,
  language: 'en',
  rateLimit: RateLimit.balanced,

  async searchNovels(query, page) {
    const html = await get(buildUrl(BASE, '/search', { q: query, page }))
    const $ = load(html)
    const items = $('div.result').map((_, el) => ({
      id: $(el).find('a').attr('href')!,
      title: $(el).find('h3').text().trim(),
      url: $(el).find('a').attr('href')!,
      cover: $(el).find('img').attr('src'),
    })).get()
    return { items, hasNextPage: $('a.next').length > 0 }
  },

  async fetchNovelDetails(novelUrl) {
    const html = await get(novelUrl)
    const $ = load(html)
    return {
      id: novelUrl, title: $('h1').text().trim(), url: novelUrl,
      chapters: $('ul.chapters a').map((i, el) => ({
        id: $(el).attr('href')!, title: $(el).text().trim(),
        number: i + 1, url: $(el).attr('href')!,
      })).get(),
    }
  },

  async fetchChapterContent(chapterUrl) {
    const $ = load(await get(chapterUrl))
    return $('div.content').html() ?? ''
  },
})

API

HTTP

• get(url, headers?) — GET request, returns body string
• post(url, body?, headers?) — POST request
• json<T>(url, headers?) — GET + parse JSON
• postJSON<T>(url, data, headers?) — POST JSON + parse response
• request(req) / requestFull(req) — full control

Rate Limiting

// Via createSource (recommended):
rateLimit: RateLimit.strict   // 1 req/s
rateLimit: RateLimit.balanced // 3 req/s (recommended)
rateLimit: RateLimit.loose    // 10 req/s

// Or manually:
import { setRateLimit } from '@glyphmoe/sdk'
setRateLimit({ requests: 5, perSeconds: 2 })

Timeout

// Via createSource:
timeout: 60000  // 60s (default: 30s)

// Or manually:
import { setRequestTimeout } from '@glyphmoe/sdk'
setRequestTimeout(60000)

HTML Parsing

import { load } from '@glyphmoe/sdk' // re-export of cheerio.load
const $ = load(html)

Cookies

import { getCookies, setCookie } from '@glyphmoe/sdk'
const cookies = await getCookies('https://example.com')
setCookie('https://example.com', 'token', 'abc')

Content Rating

import { getMaxContentRating, isRatingAllowed } from '@glyphmoe/sdk'
const rating = await getMaxContentRating() // 'everyone'|'teen'|'mature'|'adult'

Pagination

import { fetchAllPages } from '@glyphmoe/sdk'
const all = await fetchAllPages(async (page) => {
  const items = await fetchPage(page)
  return { items, hasNextPage: items.length === 20 }
})

// With error recovery (returns partial results on failure):
const chapters = await fetchAllPages(fetcher, 1, 200, (error, page) => {
  console.warn(`Page ${page} failed:`, error)
  return 'stop' // or 'skip' to continue to next page
})

Testing

The SDK provides a mock system so extensions can be tested without real HTTP requests.

Setup

Your test setup file needs to provide the Application global that the SDK calls at runtime. The SDK ships a ready-made mock for this:

// test-setup.js (or .ts)
import { findMock, isMockEnabled } from '@glyphmoe/sdk/test-runtime'

globalThis.Application = {
  async scheduleRequest(request) {
    const mock = findMock(request.url)
    if (mock) {
      return [{ status: mock.status ?? 200, headers: mock.headers ?? {} }, mock.body]
    }
    if (isMockEnabled()) {
      throw new Error(`No mock registered for: ${request.url}`)
    }
    // Fallback to real fetch for integration tests
    const resp = await fetch(request.url, {
      method: request.method,
      headers: request.headers,
      body: request.body ?? undefined,
    })
    return [{ status: resp.status, headers: Object.fromEntries(resp.headers.entries()) }, await resp.text()]
  },
  async getDefaultUserAgent() { return 'Test/1.0' },
  async getCookies() { return {} },
  setCookie() {},
  async getMaxContentRating() { return 'adult' },
}

Then point your test runner at it (e.g. in vitest.config.ts):

export default defineConfig({
  test: { setupFiles: ['./test-setup.js'] },
})

Mocking requests

import { mockRequest, clearMocks } from '@glyphmoe/sdk/testing'

beforeEach(() => clearMocks())

mockRequest('https://example.com/search?q=test', {
  body: '<html>...</html>',
})

// Wildcard patterns
mockRequest('https://example.com/api/*', { body: '{}' }, { pattern: true })

Docs

Full documentation at glyph.moe/docs.

License

MIT