npm package discovery and stats viewer.

Discover Tips

  • General search

    [free text search, go nuts!]

  • Package details

    pkg:[package-name]

  • User packages

    @[username]

Sponsor

Optimize Toolset

I’ve always been into building performant and accessible sites, but lately I’ve been taking it extremely seriously. So much so that I’ve been building a tool to help me optimize and monitor the sites that I build to make sure that I’m making an attempt to offer the best experience to those who visit them. If you’re into performant, accessible and SEO friendly sites, you might like it too! You can check it out at Optimize Toolset.

About

Hi, 👋, I’m Ryan Hefner  and I built this site for me, and you! The goal of this site was to provide an easy way for me to check the stats on my npm packages, both for prioritizing issues and updates, and to give me a little kick in the pants to keep up on stuff.

As I was building it, I realized that I was actually using the tool to build the tool, and figured I might as well put this out there and hopefully others will find it to be a fast and useful way to search and browse npm packages as I have.

If you’re interested in other things I’m working on, follow me on Twitter or check out the open source projects I’ve been publishing on GitHub.

I am also working on a Twitter bot for this site to tweet the most popular, newest, random packages from npm. Please follow that account now and it will start sending out packages soon–ish.

Open Software & Tools

This site wouldn’t be possible without the immense generosity and tireless efforts from the people who make contributions to the world and share their work via open source initiatives. Thank you 🙏

© 2026 – Pkg Stats / Ryan Hefner

@laboon85/pi-web-access

v0.9.1

Published

Web search and web fetch for the pi coding agent, built around a tiered provider fallback chain.

Readme

@laboon85/pi-web-access

Web search and web fetch for the pi coding agent, built around a tiered provider fallback chain.

Both tools share one orchestrator. A provider that fails — quota, auth, transport, timeout, malformed response, or an unusable result — hands off to the next one in line. The chain also understands cooldown: a provider marked as cooling down is demoted to the tail of its own tier without ever being dropped. The shipped build does not yet mark providers, so today the configured order always stands.

Tools

web_search

| Argument | Required | Meaning | | ------------- | -------- | ---------------------------------- | | query | yes | Search text | | max_results | no | Results to return; capped globally |

web_fetch

| Argument | Required | Meaning | | -------- | -------- | ------------------ | | url | yes | Public HTTP(S) URL |

Returns a text block for textual content, an image block for image/png, image/jpeg, image/gif and image/webp (image/jpg is accepted as an alias), and a content-type error for anything else. What textual HTML becomes — GFM markdown or the untouched body — is decided by the fetch provider chain, not by the call.

Providers

| Provider | Search | Fetch | Key | | ------------- | ------ | ----- | ----------------------------------------------- | | local | — | yes | none (HTML → GFM markdown, in-process) | | pandoc | — | yes | none (pandoc on PATH; fidelity upgrade) | | raw | — | yes | none (body handed back untouched) | | exa-free | yes | — | none (Exa hosted MCP, rate limited per IP) | | bing | yes | — | none (HTML results) | | searxng | yes | — | none (SEARXNG_URL) | | bocha | yes | — | BOCHA_API_KEY | | tavily | yes | yes | TAVILY_API_KEY (fetch side is Extract) | | exa | yes | — | EXA_API_KEY | | brave | yes | — | BRAVE_SEARCH_API_KEY | | jina | yes | yes | optional JINA_API_KEY (keyless on fetch) | | firecrawl | yes | yes | optional FIRECRAWL_API_KEY (keyless on fetch) | | scrapfly | — | yes | SCRAPFLY_API_KEY | | scrapingant | — | yes | SCRAPINGANT_API_KEY | | scrapingbee | — | yes | SCRAPINGBEE_API_KEY | | scraperapi | — | yes | SCRAPERAPI_API_KEY | | zenrows | — | yes | ZENROWS_API_KEY |

Defaults with no configuration at all: search uses exa-free, fetch uses local.

The three keyless fetch providers read the URL themselves — no third party sees it — and differ only in what HTML becomes:

  • local renders markdown in-process (DOM parse, GFM tables, headings, links resolved against the final URL). Always available; the floor of every chain.
  • pandoc converts with the pandoc binary for fidelity the in-process converter cannot reach, complex tables especially. The binary must already be on PATH — this package never downloads or installs one — and a missing or failing binary is an ordinary provider failure the chain falls back from.
  • raw hands back the body exactly as the server sent it. It replaces the old raw flag on the tool: original markup is a chain decision now.

A chain that prefers fidelity and keeps raw markup reachable looks like "fetchProviders": ["pandoc", "local", "raw"].

The remote readers come prewired — endpoint, auth header or query-parameter key, and the response path that carries the markdown — so adding one to the chain is a one-word edit. Firecrawl and Jina serve a keyless tier: they work with no FIRECRAWL_API_KEY / JINA_API_KEY at all, and setting one raises the limits (their keys are optional; every other reader needs its key). Reachability is yours to judge — some networks block some of these hosts, and a reader that cannot be reached fails like any other provider while the chain moves on.

Configuration

One JSON file. $PI_PKG_CFG_DIR wins if set; otherwise <agent dir>/pi-pkg-cfg/pi-web-access/config.json, where <agent dir> is $PI_CODING_AGENT_DIR or ~/.pi/agent.

{
  // Results returned by web_search. Both the default and the ceiling: the model
  // may ask for fewer, never more. 1..20, default 10.
  "maxResults": 10,

  // Search chain, in preference order. Grouped by tier internally.
  "searchProviders": ["tavily", "exa", "firecrawl", "exa-free", "bing"],

  // Fetch chain, in preference order. Default: ["local"]. "pandoc" needs the
  // binary on PATH and falls back when it is missing; "raw" skips conversion.
  "fetchProviders": ["pandoc", "local", "raw"],

  "apiKeys": {
    "tavily": "tvly-...",
    "exa": "exa-...",
    "firecrawl": "fc-...",
  },

  // Optional per-capability tier overrides. Providers default to tier 1 except
  // the keyless search fallbacks (exa-free, bing), which default to tier 2.
  "tiers": {
    "search": { "bing": 3 },
    "fetch": { "jina": 1 },
  },

  // HTTP(S) proxy for every outbound request. Falls back to HTTPS_PROXY,
  // HTTP_PROXY, ALL_PROXY; honours NO_PROXY. SOCKS is not supported.
  "proxy": "http://127.0.0.1:7890",
}

Environment variables win over apiKeys for the keys listed above. Configuration is re-read on every call, so an edited chain takes effect immediately. Only maxResults is fixed at load time, because it is part of the tool schema.

Declaring your own provider

customProviders takes one entry per capability, so each side keeps its own rules. A search entry describes a JSON API; a fetch entry points at a remote reader.

"customProviders": {
  "search": {
    "my-engine": {
      "url": "https://api.example.com/search",
      "method": "POST",
      "auth": { "header": "authorization", "prefix": "Bearer " },
      "keyEnv": "MY_SEARCH_KEY",
      "body": { "q": "{query}", "limit": "{maxResults}" },
      "resultsPath": "data.items",
      "fields": { "title": "name", "url": "link", "snippet": "summary" },
      "tier": 1,
      "maxResults": 10
    }
  },
  "fetch": {
    "my-reader": {
      "url": "https://reader.example.com/{url}",
      "auth": { "header": "x-api-key", "prefix": "" },
      "keyEnv": "MY_READER_KEY",
      "fetch": { "textPath": "data.markdown", "titlePath": "data.metadata.title" },
      "tier": 2
    }
  }
}

Placeholders: {query}, {queryEncoded} (for a path segment), {maxResults}, {url} (percent-encoded), {urlRaw} (for a JSON body field, where encoding would corrupt the target), {apiKey} (for vendors that take the key as a query parameter; a missing key empties the slot), {baseUrl} (taken from baseUrls).

Rules, all enforced by name so one bad entry does not take the file down:

  • a name may not shadow a built-in provider of the capability it declares for — the fetch side ships firecrawl, jina, tavily, scrapfly, scrapingant, scrapingbee, scraperapi and zenrows, so those names are taken there — and the same custom name may not be declared for both capabilities
  • auth requires keyEnv; a custom provider reads the environment only, never apiKeys, so a key in this file can never be sent to an endpoint that did not ask for it
  • a search entry requires resultsPath; a POST entry requires body
  • tier is any positive integer; tiers need not be contiguous

Built-in vendors use exactly this shape in code rather than living in this file: their field mappings follow the vendors' APIs, so a renamed field is fixed by a release instead of silently rotting in every copy.

/web-access-status prints the active chain — each member's tier and key source, never the key itself; --all adds the providers outside the chain.

Notes

  • web_fetch refuses private, loopback and otherwise non-public addresses, and re-checks every redirect hop (at most five).
  • When a proxy is configured the DNS-level check is skipped for that transport: the proxy resolves names itself, so the local pre-resolution result would not describe the address actually contacted. Redirects are still re-checked.
  • Only http:// and https:// proxies work.
  • Every provider except the keyless local readers (local, pandoc, raw) sends your URL or query to a third party. Adding one to a chain is an explicit opt-in.

Development

npm install
npm run typecheck     # tsc --noEmit
npm run lint          # eslint
npm run format        # prettier --write
npm run test          # vitest, no network access
npm run build         # tsc -p tsconfig.build.json -> dist/
npm run smoke         # import dist/ and assert the default export
npm run pack:check    # assert the npm tarball contents
npm run build -- --watch   # keep dist/ fresh while using a local install

The published artifact is compiled JavaScript plus .d.ts. pi loads dist/index.js, so a local install needs a build first.

Releasing

Publishing uses npm trusted publishing: the workflow exchanges a GitHub OIDC token for a short-lived credential, so no npm token is stored in the repository, and provenance is attached automatically.

Once per package, on npmjs.com:

  • Release workflow must be registered as a GitHub Actions trusted publisher with workflow filename release.yml, and npm publish must be explicitly allowed — new configurations default to staged publishing only.
  • A new configuration has to complete its first publish within two days, and the package must already exist to be registered, so the very first release is published manually:
npm login
npm publish --access public

After that, releases are tagged and published by the workflow:

git tag v0.7.0
git push origin v0.7.0

The workflow refuses to publish when the tag and package.json disagree, and prepublishOnly re-runs the whole gate before the registry sees anything.

Acknowledgements

Portions of this project are derived from pi-web-search by byte, licensed under the MIT License. Copyright (c) 2026 byte. The full MIT text is reproduced in LICENSE.

Thanks also to the pi project for an extension surface that makes a package like this possible, and to the maintainers of every service listed under Providers for keeping a usable free tier alive.