Recipe Scrapers

Recipe Scrapers is a TypeScript library for extracting structured recipe data from HTML. It supports Schema.org, OpenGraph, and site-specific extraction, then normalizes and validates the result as a consistent recipe object.

The project name preserves its lineage with the Python recipe-scrapers library that inspired this port.

Features

• Extract and normalize recipe data from supplied HTML
• Support Schema.org, OpenGraph, and site-specific page structures
• Select host-specific extraction logic from the supplied recipe URL
• Built with TypeScript for better developer experience
• Comprehensive test coverage

Installation

Add the recipe-scrapers package and its peer dependencies.

npm install recipe-scrapers cheerio zod
# or
yarn add recipe-scrapers cheerio zod
# or
pnpm add recipe-scrapers cheerio zod
# or
bun add recipe-scrapers cheerio zod

Schema and utility subpath consumers do not need Cheerio:

npm install recipe-scrapers zod

Ingredient parsing is optional. Install its peer dependency only when enabling the parseIngredients option:

npm install parse-ingredient

Usage

Basic Usage

import { getScraper, scrapeRecipe } from 'recipe-scrapers'

// Acquire the HTML using your own HTTP client, browser, cache, or fixture.
const html = `<html>The recipe-page HTML to parse...</html>`
// The URL selects the extraction implementation; it is not fetched.
const url = 'https://recipes.example/recipes/tomato-soup'

// Use generic Schema.org extraction for this unregistered example host.
// Without wildMode, getScraper throws when a host is not registered.
const MyScraper = getScraper(url, { wildMode: true })
const scraper = new MyScraper(html, url, /* { ...options } */)

// Get the recipe data
const rawRecipe = await scraper.toRecipeObject()

// Get the schema validated recipe data
const validatedRecipe = await scraper.parse()

// One-shot helper (wild mode is enabled by default)
const parsed = await scrapeRecipe(html, url)

// One-shot helper with a safe parse result
const safeResult = await scrapeRecipe(html, url, { safeParse: true })

// Opt in to recipe notes when supported by the source HTML
const recipeWithNotes = await scrapeRecipe(html, url, { parseNotes: true })

// Supply a fallback when the source does not specify a yield
const recipeWithFallbackYield = await scrapeRecipe(html, url, {
  fallbackYield: 'Yield not specified',
})

// Supply an author fallback based on the extracted website name
const recipeWithFallbackAuthor = await scrapeRecipe(html, url, {
  fallbackAuthor: (siteName) => siteName ?? 'Unknown publisher',
})

Safe Parse Error Shape

When safeParse: true is used, failures return a structured error object:

type SafeParseError = {
  type: 'validation' | 'extraction'
  code:
    | 'validation_failed'
    | 'extractor_not_found'
    | 'extraction_runtime_error'
    | 'extraction_failed'
  issues: Array<{
    message: string
    path?: PropertyKey[]
    dotPath?: string | null
  }>
  cause?: unknown
  context?: {
    field?: string
    source?: string
  }
}

This makes it easy to branch in UI code:

const result = await scrapeRecipe(html, url, { safeParse: true })

if (!result.success) {
  if (result.error.code === 'extractor_not_found') {
    // missing required field (result.error.context?.field)
  } else if (result.error.code === 'extraction_runtime_error') {
    // plugin/site extractor crashed (result.error.context?.source)
  } else if (result.error.code === 'validation_failed') {
    // schema validation failed after extraction
  }
}

Recipe Evidence

Use inspectRecipeEvidence when you need to distinguish missing recipe evidence from an incomplete or failed recipe extraction. Inspection is opt-in and does not change scrapeRecipe, parse, or safeParse behavior.

import { inspectRecipeEvidence } from 'recipe-scrapers'

const evidence = await inspectRecipeEvidence(html, url)

if (evidence.status === 'detected') {
  console.log('Recipe content was detected.')
} else if (evidence.status === 'not-detected') {
  console.log('No recognized recipe content was found in the supplied HTML.')
} else if (evidence.reasons.includes('malformed-structured-data')) {
  console.log('Recipe metadata was present but could not be inspected reliably.')
} else {
  console.log('Some recipe evidence was found, but it was incomplete.')
}

The result reports whether a structured Recipe entity, non-empty ingredients, and non-empty instructions were found. uncertain results include one or more of these reasons:

• partial-evidence
• malformed-structured-data
• extractor-runtime-failure

Evidence follows the same effective merged Recipe representation and selected site-specific extractors as normal extraction. It never turns partial data into a successful Recipe Object.

not-detected means only that the supplied HTML contained no evidence the library recognizes. Plain-text recipes, dynamically rendered content, and access/interstitial pages can fall outside detection coverage; inspection does not diagnose the live page or make another network request.

The first inspection on a scraper instance evaluates only the ingredient and instruction extraction paths needed for evidence and caches its result. Subsequent calls on that instance return the cached result. Inspection does not populate the full recipe cache, so calling parse() or safeParse() afterward performs the normal extraction pipeline independently.

Validation Schema

By default, recipe data is validated with the built-in Zod schema.

Browser code that only needs schemas, recipe types, or safe-parse helpers should use the browser-safe recipe-scrapers/schema subpath. It does not import Cheerio or the scraping runtime.

import {
  RECIPE_SCHEMA_VERSION,
  RecipeObjectSchema,
  type RecipeObject,
} from 'recipe-scrapers/schema'

const result = RecipeObjectSchema.safeParse(candidate)

Browser code that only needs general utilities should use the dedicated browser-safe utilities subpath:

import { parseYields } from 'recipe-scrapers/utils'

You can also validate with any Standard Schema compatible schema (for example Valibot).

import { scrapeRecipe } from 'recipe-scrapers'

// Example: a Standard Schema-compatible schema from another library
import { RecipeSchema as ValibotRecipeSchema } from './valibot-recipe-schema'

const result = await scrapeRecipe(html, url, {
  safeParse: true,
  schema: ValibotRecipeSchema,
})

Options

interface ScraperOptions {
  /**
   * Additional extractors to be used by the scraper.
   * These extractors will be added to the default set of extractors.
   * Extractors are applied according to their priority.
   * Higher priority extractors will run first.
   * @default []
   */
  extraExtractors?: ExtractorPlugin[]
  /**
   * Additional post-processors to be used by the scraper.
   * These post-processors will be added to the default set of post-processors.
   * Post-processors are applied after all extractors have run.
   * Post-processors are also applied according to their priority.
   * Higher priority post-processors will run first.
   * @default []
   */
  extraPostProcessors?: PostProcessorPlugin[]
  /**
   * Non-empty value to use when no extractor finds a non-empty author.
   * A function receives the extracted website name, or `null` when unavailable.
   * Extracted author values always take precedence.
   * When omitted, a missing author remains an extraction failure.
   */
  fallbackAuthor?: string | ((siteName: string | null) => string)
  /**
   * Non-empty value to use when no extractor can find a recipe yield.
   * Extracted yield values always take precedence.
   * When omitted, a missing yield remains an extraction failure.
   */
  fallbackYield?: string
  /**
   * Whether link extraction is enabled.
   * @default false
   */
  linksEnabled?: boolean
  /**
   * Logging level for the scraper.
   * This controls the verbosity of logs produced by the scraper.
   * @default LogLevel.WARN
   */
  logLevel?: LogLevel
  /**
   * Enable ingredient parsing using the parse-ingredient library.
   * When enabled, each ingredient item will include a `parsed` field
   * containing structured data (quantity, unit, description, etc.).
   * Can be `true` for defaults or an options object.
   * Set `descriptionMeasurements: true` to include quantity/unit pairs
   * embedded in ingredient descriptions.
   * @see https://github.com/jakeboone02/parse-ingredient
   * @default false
   */
  parseIngredients?: boolean | IngredientParserOptions
  /**
   * Enable recipe note parsing from supported HTML recipe blocks.
   * When enabled, recipes may include a `notes` field containing
   * grouped note items when the source markup supports it.
   * @default false
   */
  parseNotes?: boolean
  /**
   * Standard Schema-compatible schema used for validation.
   * Useful when validating with libraries such as Valibot.
   */
  schema?: StandardSchemaV1<unknown, RecipeObject>
}

Recipe Notes

Recipe notes are opt-in and currently extracted from supported WP Recipe Maker HTML note blocks.

const recipe = await scrapeRecipe(html, url, {
  parseNotes: true,
})

console.log(recipe.notes)
// [
//   {
//     name: null,
//     items: [
//       { value: 'Store in an airtight container for up to 3 weeks.' },
//     ],
//   },
// ]

When note parsing is disabled, or when no supported note block is found, the notes field is omitted.

Supported Sites

This library supports recipe extraction for various popular cooking websites. It uses the supplied URL to select the appropriate extraction implementation, but it does not visit that URL.

Supported hosts are registered in src/scrapers/_index.ts, split between custom scrapers and Schema.org-only hosts.

Acquiring HTML Responsibly

This library only processes HTML supplied by the caller. If you acquire HTML from a third-party website, you are responsible for complying with that site's terms, robots directives, copyright restrictions, and applicable law.

Development

Documentation

Project policy documents

• Contributing guide
• Governance

Architecture documents

• Architecture overview
• Ingredients architecture

AI-assisted development

If you use AI coding agents on this repo, install Matt Pocock's agent skills so the agent can follow the repo's issue, triage, and domain-documentation conventions:

npx skills@latest add mattpocock/skills

The repo-specific configuration lives in AGENTS.md, CONTEXT.md, and docs/agents/.

Prerequisites

• Bun (latest version)

Setup

# Clone the repository
git clone https://github.com/recipe-scrapers/recipe-scrapers.git
cd recipe-scrapers

# Install dependencies
bun install

# Run tests
bun test

# Build the project
bun run build

Scripts

• bun run build - Build the library for distribution
• bun test - Run the test suite
• bun test:coverage - Run tests with a coverage report
• bun fetch-test-data - Fetch test data from the original Python repository
• bun lint - Run linting and type checking
• bun lint:fix - Fix linting issues automatically

Adding New Scrapers

1. Fetch test data from the original Python repository

   bun fetch-test-data

2. Convert the data into the expected JSON format (i.e. the RecipeObject interface)

   bun process-test-data <host>

3. Choose the scraper type:

   • Schema.org-only host (no site-specific extraction needed): add the hostname to SCHEMA_ORG_ONLY_HOSTS in src/scrapers/_index.ts
   • Custom scraper (site-specific extraction needed): create a new scraper class extending AbstractScraper

4. If using a custom scraper, add it to customScraperClasses in src/scrapers/_index.ts

5. Add optional host aliases to scraperAliases in src/scrapers/_index.ts when needed

6. Run tests to ensure the extraction works as expected

7. Update documentation as needed, using CONTEXT.md for the project's domain language

import { AbstractScraper } from './abstract-scraper'
import type { RecipeFields } from '@/types/recipe.interface'

export class NewSiteScraper extends AbstractScraper {
  static host() {
    return 'recipes.example'
  }

  extractors = {
    ingredients: this.extractIngredients.bind(this),
  }

  protected extractIngredients(): RecipeFields['ingredients'] {
    const items = this.$('.ingredient')
      .map((_, el) => this.$(el).text().trim())
      .get()

    return [
      {
        name: null,
        items: items.map((value) => ({ value })),
      },
    ]
  }
  
  // ... implement other extraction methods
}

Testing

The project uses test data from the original Python recipe-scrapers repository to ensure compatibility and accuracy. Tests are written using Bun's built-in test runner.

# Run all tests
bun test

# Run tests with coverage
bun test:coverage

Acknowledgments

• Original recipe-scrapers Python library by hhursev
• Schema.org Recipe specification
• Cheerio for HTML parsing
• Zod for schema validation
• Standard Schema for schema interoperability
• parse-ingredient for ingredient parsing

Contributing

Please read CONTRIBUTING.md before opening a pull request.

Project direction and maintainer decision rules are documented in GOVERNANCE.md.

Contributors

License

This project is licensed under the MIT License - see the LICENSE file for details.