@lamartinecabral/web-search

A TypeScript/ESM client for web search and page-content extraction in RAG workflows. It exposes one API and selects an available backend automatically.

Requirements

• Node.js 24 or newer
• Network access to the selected backend
• Google Chrome only when the Puppeteer backend is used

Installation

npm install @lamartinecabral/web-search

The package is ESM-only, so import it from an ES module:

import { getWebSearchClient } from "@lamartinecabral/web-search";

Usage

const client = await getWebSearchClient({
  tavily: { apiKey: process.env.TAVILY_API_KEY },
});

const results = await client.webSearch(
  "latest developments in retrieval augmented generation",
);

for (const result of results) {
  console.log(result.title, result.url);
  console.log(result.snippet);
}

if (results[0]) {
  const page = await client.webFetch(results[0].url);
  console.log(page.title);
  console.log(page.content);
}

Create the client once and reuse it. webSearch returns SearchResult[]:

{
  title: string;
  url: string;
  snippet: string;
}

webFetch returns a FetchResult:

{
  title: string;
  content: string;
}

The returned title can be empty for providers that do not supply one. Both methods reject with an error when the selected backend cannot complete the operation.

Backend selection

getWebSearchClient checks providers in this order:

1. Ollama, when ollama.apiKey is set
2. Tavily, when tavily.apiKey is set
3. Puppeteer with Brave Search, when puppeteer.chromePath is configured and the executable is available
4. DuckDuckGo HTML search and native fetch, when DuckDuckGo is reachable

Only the first matching backend is used. If none is available, the function throws Web search feature is not available.

Ollama

const client = await getWebSearchClient({
  ollama: { apiKey: process.env.OLLAMA_API_KEY },
});

Uses Ollama's hosted web_search and web_fetch APIs.

To get an API key, sign in to or create an account on Ollama, then create a key in API key settings.

Tavily

const client = await getWebSearchClient({
  tavily: { apiKey: process.env.TAVILY_API_KEY },
});

Uses Tavily's Search and Extract APIs.

To get an API key, sign in to or create an account on the Tavily Platform. Find or create a key in the dashboard's API Keys section.

Puppeteer

The Puppeteer backend uses puppeteer-core to open a visible Chrome window, search Brave Search, and extract content from the requested page. Set puppeteer.chromePath to the Chrome executable's path, or use "default". With "default", the backend uses the path in the CHROME_PATH environment variable when set; otherwise, it uses this platform-specific default:

• macOS: /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
• Windows: C:\Program Files\Google\Chrome\Application\chrome.exe
• Linux: /usr/bin/google-chrome

The Puppeteer backend is skipped when puppeteer.chromePath is omitted.

For example, to use the platform default (or CHROME_PATH):

const web = await getWebSearchClient({
  puppeteer: { chromePath: "default" },
});

To use a different Chrome installation, provide its executable path instead:

const web = await getWebSearchClient({
  puppeteer: { chromePath: "path/to/chrome" },
});

DuckDuckGo fallback

When Puppeteer is not configured or Chrome is unavailable, the fallback uses DuckDuckGo's HTML endpoint for search and native fetch plus content extraction for page fetching. It depends on DuckDuckGo being reachable from the host.

Warning: The native fetch fallback has a high chance of being blocked by bot-detection systems. A hosted provider such as Ollama or Tavily, or the Puppeteer backend, is recommended for more reliable access.

Development

npm install
npm run typecheck
npm run lint
npm run build

Use npm run lint:fix to apply Biome fixes. npm run build emits JavaScript and declaration files to dist/; prepack runs the same build automatically. The project currently has no test script.

License

ISC