puppeteer-extra-plugin-stealth [

A plugin for puppeteer-extra and playwright-extra to prevent detection.

Install

npm install @zorilla/puppeteer-extra-plugin-stealth

If this is your first puppeteer-extra plugin here's everything you need:

npm install puppeteer @zorilla/puppeteer-extra @zorilla/puppeteer-extra-plugin-stealth

Usage

// puppeteer-extra is a drop-in replacement for puppeteer,
// it augments the installed puppeteer with plugin functionality
import puppeteer from '@zorilla/puppeteer-extra'

// add stealth plugin and use defaults (all evasion techniques)
import StealthPlugin from '@zorilla/puppeteer-extra-plugin-stealth'
puppeteer.use(StealthPlugin())

// puppeteer usage as normal
puppeteer.launch({ headless: true }).then(async browser => {
  console.log('Running tests..')
  const page = await browser.newPage()
  await page.goto('https://bot.sannysoft.com')
  await page.waitForTimeout(5000)
  await page.screenshot({ path: 'testresult.png', fullPage: true })
  await browser.close()
  console.log(`All done, check the screenshot. ✨`)
})

Please check out the puppeteer-extra package to learn more about puppeteer-extra (Firefox usage, other Plugins, etc).

Status

• ✅ puppeteer-extra with stealth passes all public bot tests.

Please note: I consider this a friendly competition in a rather interesting cat and mouse game. If the other team (👋) wants to detect headless chromium there are still ways to do that (at least I noticed a few, which I'll tackle in future updates).

It's probably impossible to prevent all ways to detect headless chromium, but it should be possible to make it so difficult that it becomes cost-prohibitive or triggers too many false-positives to be feasible.

If something new comes up or you experience a problem, please do your homework and create a PR in a respectful way (this is Github, not reddit) or I might not be motivated to help. :)

Test results (red is bad)

Vanilla puppeteer without stealth 😢

Puppeteer with stealth plugin 💯

Note: The MQ_SCREEN test is broken on their page (will fail in regular Chrome as well).

Tests have been done using this test site and these scripts in test/stealth/.

Improved reCAPTCHA v3 scores

Using stealth also seems to help with maintaining a normal reCAPTCHA v3 score.

Note: The official test is to be taken with a grain of salt, as the score is calculated individually per site and multiple other factors (past behaviour, IP address, etc). Based on anecdotal observations it still seems to work as a rough indicator.

Tip: Have a look at the recaptcha plugin if you have issues with reCAPTCHAs.

API

Table of Contents

• puppeteer-extra-plugin-stealth [
  • Install
  • Usage
  • Status
  • Changelog
    • v2.4.7
    • v2.4.2 / v2.4.1
    • v2.4.0
  • Test results (red is bad)
    • Vanilla puppeteer without stealth 😢
    • Puppeteer with stealth plugin 💯
    • Improved reCAPTCHA v3 scores
  • API
    • Table of Contents
    • class: StealthPlugin
      • Purpose
      • Modularity
      • Contributing
      • Kudos
      • .availableEvasions
      • .enabledEvasions
    • defaultExport(opts?)
  • License

class: StealthPlugin

• opts Object? Options (optional, default {})
  • opts.enabledEvasions Set<string>? Specify which evasions to use (by default all)

Extends: PuppeteerExtraPlugin

Stealth mode: Applies various techniques to make detection of headless puppeteer harder. 💯

Purpose

There are a couple of ways the use of puppeteer can easily be detected by a target website. The addition of HeadlessChrome to the user-agent being only the most obvious one.

The goal of this plugin is to be the definite companion to puppeteer to avoid detection, applying new techniques as they surface.

As this cat & mouse game is in it's infancy and fast-paced the plugin is kept as flexibile as possible, to support quick testing and iterations.

Modularity

This plugin uses puppeteer-extra's dependency system to only require code mods for evasions that have been enabled, to keep things modular and efficient.

The stealth plugin is a convenience wrapper that requires multiple evasion techniques automatically and comes with defaults. You could also bypass the main module and require specific evasion plugins yourself, if you whish to do so (as they're standalone puppeteer-extra plugins):

// bypass main module and import a specific stealth plugin directly:
import WebGLVendor from '@zorilla/puppeteer-extra-plugin-stealth/evasions/webgl.vendor'
puppeteer.use(WebGLVendor())

Contributing

PRs are welcome, if you want to add a new evasion technique I suggest you look at the template to kickstart things.

Kudos

Thanks to Evan Sangaline and Paul Irish for kickstarting the discussion!

Example:

import puppeteer from '@zorilla/puppeteer-extra'
import StealthPlugin from '@zorilla/puppeteer-extra-plugin-stealth'

// Enable stealth plugin with all evasions
puppeteer.use(StealthPlugin())
;(async () => {
  // Launch the browser in headless mode and set up a page.
  const browser = await puppeteer.launch({
    args: ['--no-sandbox'],
    headless: true
  })
  const page = await browser.newPage()

  // Navigate to the page that will perform the tests.
  const testUrl =
    'https://intoli.com/blog/' +
    'not-possible-to-block-chrome-headless/chrome-headless-test.html'
  await page.goto(testUrl)

  // Save a screenshot of the results.
  const screenshotPath = '/tmp/headless-test-result.png'
  await page.screenshot({ path: screenshotPath })
  console.log('have a look at the screenshot:', screenshotPath)

  await browser.close()
})()

.availableEvasions

Type: Set<string>

Get all available evasions.

Please look into the evasions directory for an up to date list.

Example:

import StealthPlugin from '@zorilla/puppeteer-extra-plugin-stealth'

const pluginStealth = StealthPlugin()
console.log(pluginStealth.availableEvasions) // => Set { 'chrome.app', 'chrome.runtime', 'navigator.webdriver', ... }
puppeteer.use(pluginStealth)

.enabledEvasions

Type: Set<string>

Get all enabled evasions.

Enabled evasions can be configured either through opts or by modifying this property.

Example:

// Remove specific evasion from enabled ones dynamically
import StealthPlugin from '@zorilla/puppeteer-extra-plugin-stealth'

const pluginStealth = StealthPlugin()
pluginStealth.enabledEvasions.delete('chrome.runtime')
puppeteer.use(pluginStealth)

defaultExport(opts?)

• opts Object? Options
  • opts.enabledEvasions Set<string>? Specify which evasions to use (by default all)

Default export, PuppeteerExtraStealthPlugin

License

MIT