@hanivanrizky/nestjs-browser-action

⚠️ Status: Experimental

This project is currently in experimental stage and intended for personal use only. The API is subject to change, and production use is not recommended.

A NestJS module that provides Puppeteer-based browser automation with configurable options, connection pooling, and data cleansing capabilities.

Features

• (・_・) Browser Automation: Declarative workflow-based browser automation
• (☆^O^☆) Data Scraping: Single and multi-element scraping with CSS/XPath selectors
• (>_>) Connection Pooling: Efficient browser instance management
• (♡˙︶˙♡) Cookie Persistence: Save/load browser sessions for authentication
• (｡•̀ᴗ-)✧ Data Cleansing: 14 built-in transformation pipes
• (°_°)! Shadow DOM: Support for web components
• (^_^) Type-Safe: Full TypeScript support with generics
• (^^) Remote Chrome: Connect to remote Chrome instances via CDP (browserURL/browserWSEndpoint)

Installation

From npm

npm install @hanivanrizky/nestjs-browser-action
# or
yarn add @hanivanrizky/nestjs-browser-action

From GitHub

npm install https://github.com/Hanivan/nestjs-browser-action.git
# or
yarn add https://github.com/Hanivan/nestjs-browser-action.git
# or using SSH
yarn add [email protected]:Hanivan/nestjs-browser-action.git

Quick Start

1. Configure Module

import { Module } from '@nestjs/common';
import { BrowserActionModule } from '@hanivanrizky/nestjs-browser-action';

@Module({
  imports: [
    BrowserActionModule.forRoot({
      pool: { min: 2, max: 10 },
      cookies: { enabled: true },
    }),
  ],
})
export class AppModule {}

2. Inject Service

import { Injectable } from '@nestjs/common';
import { ActionHelpersService } from '@hanivanrizky/nestjs-browser-action';

@Injectable()
export class MyService {
  constructor(
    private readonly actionHelpers: ActionHelpersService,
  ) {}

  async scrapeData() {
    const result = await this.actionHelpers.scrape(
      'https://example.com',
      {
        title: 'h1',
        description: 'meta[name="description"]@content',
      }
    );

    console.log(result.title);       // "Example Domain"
    console.log(result.description); // "This domain is for use in..."
  }
}

Documentation

(^_^) Method Documentation

| Method | Description | |--------|-------------| | scrape() | Extract single elements | | scrapeAll() | Extract multiple elements | | scrapeWithActions() | Workflow-based automation | | scrapeAllWithWorkflow() | Workflow with multi-element | | takeScreenshot() | Capture screenshots | | generatePDF() | Generate PDFs | | Browser & Page Control | Low-level control |

(☆^O^☆) Feature Guides

| Feature | Description | |---------|-------------| | Cookie Management | Session persistence | | Pipe System | Data transformation | | Workflow Actions | All action types reference |

(^_^) API Reference

• API Reference - Complete API documentation
• Configuration - All options
• Types - TypeScript interfaces

Quick Examples

Simple Scraping

const data = await this.actionHelpers.scrape('https://example.com', {
  title: 'h1',
  price: '.price',
});

Multi-Element Scraping

const data = await this.actionHelpers.scrapeAll('https://example.com', {
  titles: '.card h2',
  links: '.card a',
});

Workflow Automation

const workflow = {
  version: '1.0' as const,
  actions: [
    { action: 'navigate' as const, value: 'https://example.com' },
    { id: 'title', action: 'extract' as const, target: { type: 'css' as const, value: 'h1' } },
    { action: 'click' as const, target: { type: 'css' as const, value: '#button' } },
  ],
};

const result = await this.actionHelpers.scrapeWithActions(workflow);

With Data Cleansing

import { CleansingType } from '@hanivanrizky/nestjs-browser-action';

const data = await this.actionHelpers.scrape('https://example.com', {
  price: '.price',
}, {
  pipes: {
    price: [
      { type: CleansingType.REMOVE_CURRENCY_SYMBOL },
      { type: CleansingType.TO_NUMBER },
    ],
  },
});

Cookie Persistence

const workflow = {
  version: '1.0' as const,
  actions: [
    { action: 'loadCookies' as const, value: 'user-session', onError: 'skip' as const },
    { action: 'navigate' as const, value: 'https://example.com/dashboard' },
    { action: 'saveCookies' as const, value: 'user-session', options: { overwrite: true } },
  ],
};

Remote Chrome Connection

Connect to remote Chrome instances via Chrome DevTools Protocol (CDP):

BrowserActionModule.forRoot({
  remote: {
    browserURL: 'http://localhost:9222',  // Or use browserWSEndpoint
    retryMax: 3,                          // Connection retry attempts
    retryDelay: 1000,                     // Delay between retries (ms)
  },
  pool: { min: 2, max: 5 },
})

Using browserWSEndpoint:

BrowserActionModule.forRoot({
  remote: {
    browserWSEndpoint: 'ws://localhost:9222/devtools/page/abc123',
  },
})

Remote-first priority: When both remote and launchOptions are provided, remote connection takes precedence.

See: Remote Chrome Configuration for details.

Services

| Service | Description | |---------|-------------| | ActionHelpersService | High-level automation methods (scrape, screenshot, PDF, workflows) | | BrowserManagerService | Browser pool management | | PageService | Page lifecycle and navigation | | CookieService | Cookie persistence | | CleansingService | Data cleansing with pipes |

Configuration

Basic Configuration

BrowserActionModule.forRoot({
  pool: {
    min: 2,
    max: 10,
    idleTimeoutMs: 30000,
    strategy: 'round-robin',
  },
  cookies: {
    enabled: true,
    cookiesDir: './cookies',
  },
  logLevel: 'log',
})

All Options

See Configuration Reference for complete options.

Type Safety

Full TypeScript support with generics:

// Type-safe selectors
interface ProductSelectors {
  title: string;
  price: number;
}

const result = await this.actionHelpers.scrape<ProductSelectors>(url, {
  title: 'h1',
  price: '.price',
});

// Type-safe workflow results
const workflow = await this.actionHelpers.scrapeWithActions<{
  title: string;
  price: number;
}>(url, workflow);

Development

Scripts

# Build
yarn build

# Run tests
yarn test

# Lint code
yarn lint

# Format code
yarn format

Git Hooks

• Pre-commit: Runs ESLint
• Pre-push: Runs build and tests

License

MIT

Support

For issues and questions, please use GitHub Issues.

Examples

Check out the test project for complete examples: test-browser-action

Documentation:

• Methods - Method-specific guides
• Features - Feature guides
• API Reference - Complete API
• Workflow Actions - Action reference