scoutkit

Tool-call-ready, normalized data-source clients for AI agents.

Stop re-writing the same API client in every agent project. scoutkit gives you battle-tested, normalized, tool-call-ready clients for the data sources AI agents actually need — Twitter/X, YouTube, Reddit, Hacker News, Google SERP, Exa, Tavily, and TikTok (via Apify). It consolidates the provider clients the author kept rebuilding by hand across several agent projects into one framework-agnostic library, then ships first-class adapters for the Vercel AI SDK, OpenAI, Anthropic, and MCP.

Every client returns clean, well-typed shapes (no raw upstream JSON), reads credentials from a single config module, and routes through a resilient fetch with per-attempt timeouts and bounded retries — so flaky upstream APIs don't surface intermittent 502s and timeouts to your agent.

Features

• 16 ready-to-use tools across 8 sources, exposed as a single flat registry.
• Normalized return types. Each client maps the provider's response to a documented TypeScript type — no any, no raw envelopes leaking into your prompts.
• Framework-agnostic core, adapters on top. One ScoutTool shape converts to Vercel AI SDK tools, OpenAI/Anthropic tool definitions, or an MCP server.
• Runtime-agnostic. Pure fetch / URL / AbortSignal. Runs on Node >=18 and Bun, no native deps.
• Centralized, optional credentials. Keys load from .env or are set programmatically with configure(). A clear MissingKeyError is thrown only when you actually call a client without its key.
• Resilient HTTP. Per-attempt timeouts plus retry on transient 408/429/5xx and network errors, honouring Retry-After.
• Cancellation everywhere. Every public function accepts an optional AbortSignal.
• Typed errors. ApiError (provider, status, body) and MissingKeyError for predictable handling.

Install

npm i scoutkit

The adapters depend on host SDKs you probably already have. Both are optional peer dependencies — install only the ones you use:

# Vercel AI SDK adapter
npm i ai

# MCP server / adapter
npm i @modelcontextprotocol/sdk

The OpenAI and Anthropic adapters emit plain tool-definition objects and require no extra packages.

Configure

Copy the example file and fill in only the keys for the sources you use:

cp node_modules/scoutkit/.env.example .env

Every credential is optional. A client throws a clear MissingKeyError only when you call it without its key configured. Hacker News needs no key at all.

| Env var | What it is | Where to get the key | Tools that need it | | --- | --- | --- | --- | | TWITTERAPI_KEY | Twitter/X access via twitterapi.io (not the official X API) | https://twitterapi.io | twitter_search, twitter_user_tweets, twitter_list_tweets, twitter_user_info | | YOUTUBE_API_KEY | YouTube Data API v3 key | https://console.cloud.google.com (enable YouTube Data API v3) | youtube_search_channels, youtube_channel_videos, youtube_discover_channels | | SERPER_API_KEY | Google SERP via Serper | https://serper.dev | google_search | | EXA_API_KEY | Exa semantic / neural web search | https://exa.ai | exa_search, semantic_search | | TAVILY_API_KEY | Tavily web search (the semantic_search fallback) | https://tavily.com | tavily_search, semantic_search (fallback) | | APIFY_TOKEN | Apify token for the TikTok scraper / actor runner | https://apify.com → Settings → Integrations | tiktok_profiles | | REDDIT_CLIENT_ID | Reddit app-only OAuth client id | https://www.reddit.com/prefs/apps (create a script or web app) | reddit_search, reddit_subreddit_hot, reddit_thread | | REDDIT_CLIENT_SECRET | Reddit OAuth client secret | same Reddit app as above | reddit_search, reddit_subreddit_hot, reddit_thread | | REDDIT_USER_AGENT | Reddit-required User-Agent string, e.g. scoutkit/0.1 by u/you | you choose it | reddit_search, reddit_subreddit_hot, reddit_thread | | (none) | Hacker News search via Algolia | — | hackernews_search |

Back-compat aliases are also accepted: TWITTER_API_KEY for TWITTERAPI_KEY, and APIFY_API_TOKEN for APIFY_TOKEN.

Programmatic keys

Prefer to set credentials in code (tests, multi-tenant servers, bring-your-own-key)? Use configure() — anything set here wins over environment variables:

import { configure } from "scoutkit";

configure({
  exaApiKey: process.env.MY_EXA_KEY,
  serperApiKey: "...",
});

Quick start

a) Use a client directly

import { exaSearch, redditSearch } from "scoutkit";

const results = await exaSearch({ query: "open-source agent frameworks", numResults: 5 });
for (const r of results.results) {
  console.log(r.title, r.url);
}

// Every function accepts an optional AbortSignal:
const posts = await redditSearch({ query: "rust", limit: 10 }, /* … */);

b) Vercel AI SDK via toVercelTools()

import { generateText } from "ai";
import { openai } from "@ai-sdk/openai";
import { toVercelTools } from "scoutkit/adapters/vercel-ai";

const { text } = await generateText({
  model: openai("gpt-4o"),
  tools: toVercelTools(),          // all 16 tools, ready to spread in
  maxSteps: 5,
  prompt: "Find the top Hacker News discussion about Bun this week and summarize it.",
});

toVercelTools() accepts an optional subset, e.g. toVercelTools(toolGroups.reddit).

c) MCP server

Run the bundled stdio MCP server with no install:

npx scoutkit-mcp

Register it with Claude Code or Claude Desktop (claude_desktop_config.json → mcpServers):

{
  "mcpServers": {
    "scoutkit": {
      "command": "npx",
      "args": ["scoutkit-mcp"],
      "env": {
        "EXA_API_KEY": "...",
        "SERPER_API_KEY": "...",
        "REDDIT_CLIENT_ID": "...",
        "REDDIT_CLIENT_SECRET": "...",
        "REDDIT_USER_AGENT": "scoutkit/0.1 by u/you"
      }
    }
  }
}

Only set the keys for the tools you intend to use; the rest stay dormant.

OpenAI & Anthropic adapters

Both adapters turn the scoutkit registry into the provider's native tool format and give you a one-call dispatcher to run a tool the model selected.

OpenAI

import OpenAI from "openai";
import { toOpenAITools, dispatchToolCall } from "scoutkit/adapters/openai";

const client = new OpenAI();

const completion = await client.chat.completions.create({
  model: "gpt-4o",
  tools: toOpenAITools(),
  messages: [{ role: "user", content: "What are the latest YouTube videos from @veritasium?" }],
});

const call = completion.choices[0].message.tool_calls?.[0];
if (call) {
  const result = await dispatchToolCall(call); // runs the matching scoutkit client
  // feed `result` back as a tool message…
}

Anthropic

import Anthropic from "@anthropic-ai/sdk";
import { toAnthropicTools, dispatchToolUse } from "scoutkit/adapters/anthropic";

const anthropic = new Anthropic();

const message = await anthropic.messages.create({
  model: "claude-opus-4-20250514",
  max_tokens: 1024,
  tools: toAnthropicTools(),
  messages: [{ role: "user", content: "Search Twitter for chatter about scoutkit." }],
});

for (const block of message.content) {
  if (block.type === "tool_use") {
    const result = await dispatchToolUse(block); // runs the matching scoutkit client
    // return a tool_result block with `result`…
  }
}

Tools

All 16 tools, in registry order:

| Tool | Source | What it does | Key needed | Rough cost | | --- | --- | --- | --- | --- | | twitter_search | Twitter / X (twitterapi.io) | Search recent tweets by query | TWITTERAPI_KEY | Paid, per tweet returned | | twitter_user_tweets | Twitter / X (twitterapi.io) | Fetch a user's latest tweets | TWITTERAPI_KEY | Paid, per tweet returned | | twitter_list_tweets | Twitter / X (twitterapi.io) | Fetch tweets from a List | TWITTERAPI_KEY | Paid, per tweet returned | | twitter_user_info | Twitter / X (twitterapi.io) | Look up a profile (bio, follower counts) | TWITTERAPI_KEY | Paid, per request | | youtube_search_channels | YouTube Data API v3 | Search channels by keyword | YOUTUBE_API_KEY | Free within daily quota | | youtube_channel_videos | YouTube Data API v3 | List a channel's recent videos | YOUTUBE_API_KEY | Free within daily quota | | youtube_discover_channels | YouTube Data API v3 | Discover channels around a topic | YOUTUBE_API_KEY | Free within daily quota | | reddit_search | Reddit (OAuth) | Search posts across Reddit | Reddit OAuth | Free | | reddit_subreddit_hot | Reddit (OAuth) | Hot posts from a subreddit | Reddit OAuth | Free | | reddit_thread | Reddit (OAuth) | A post plus its top comments | Reddit OAuth | Free | | hackernews_search | Hacker News (Algolia) | Search HN stories & comments | none | Free | | google_search | Google SERP (Serper) | Google search results | SERPER_API_KEY | ~$0.003 / call | | exa_search | Exa | Neural / semantic web search | EXA_API_KEY | ~$0.005 / call | | tavily_search | Tavily | Web search with answer synthesis | TAVILY_API_KEY | Per Tavily plan | | semantic_search | Exa → Tavily | Semantic search via Exa, falling back to Tavily | EXA_API_KEY and/or TAVILY_API_KEY | ~$0.005 / call | | tiktok_profiles | TikTok (via Apify) | Scrape TikTok profiles & recent posts | APIFY_TOKEN | ~$0.30 / 1k posts |

Costs are approximate and depend on your provider plan and request size. Treat them as rough order-of-magnitude guidance, not a quote.

You can register all tools or a subset:

import { allTools, toolGroups, getTool } from "scoutkit";

allTools;                 // every ScoutTool, in provider order
toolGroups.reddit;        // just the Reddit tools
getTool("exa_search");    // a single tool by name

Programmatic API

Import the underlying client functions directly from scoutkit (or scoutkit/clients). Each takes a single options object and an optional AbortSignal, and returns a normalized, typed result.

Twitter / X (twitterapi.io)

• searchTweets — search recent tweets
• getUserLastTweets — a user's latest tweets
• getListTweets — tweets from a List
• getUserInfo — profile lookup

YouTube (Data API v3)

• searchChannels — search channels
• getChannels — fetch channel details by id
• getChannelVideos — a channel's recent uploads
• discoverChannels — discover channels around a topic

Reddit (app-only OAuth)

• redditSearch — search posts
• fetchHotPosts — hot posts in a subreddit
• getThread — post + top comments
• redditComments / fetchTopComments — comment helpers

Hacker News (Algolia)

• searchHN — search stories and comments

Google SERP (Serper)

• googleSearch — Google search results

Exa

• exaSearch — neural / semantic web search

Tavily

• tavilySearch — web search with answer synthesis

Semantic search

• semanticSearch — Exa with a Tavily fallback

Apify / TikTok

• scrapeTiktokProfiles — scrape TikTok profiles & posts
• runActorSync — run any Apify actor synchronously

Config

• configure, getConfig, hasConfig — manage credentials in code

How it works

Credential resolution order. For every key the config module resolves in this order: configure() override → primary env var → back-compat alias env var. .env is loaded lazily on first access (and never clobbers a var the host already set), so apps that manage process.env themselves or run on edge runtimes are unaffected. Clients never read process.env directly and never hardcode a key — they ask the config module and get a clear MissingKeyError if it's absent.

Resilient fetch. Every client routes HTTP through resilientFetch, which applies a per-attempt timeout (default 60s) and a bounded retry (default 2) on transient statuses (408/429/500/502/503/504) and network failures. Backoff is exponential and honours Retry-After. Your AbortSignal is combined with the timeout, so a hung request is aborted but a user-initiated cancellation is never retried. Non-OK responses surface as an ApiError carrying the provider, status, status text, and response body.

Normalized types. Each client maps the upstream payload onto a documented TypeScript type rather than returning raw provider JSON. That keeps tool outputs stable and prompt-friendly even when an upstream API changes its envelope. The shared ScoutTool shape (name, description, Zod inputSchema, async execute) is the single source the adapters convert into Vercel AI SDK / OpenAI / Anthropic / MCP formats.

Contributing

Issues and PRs are welcome.

npm install
npm run build       # bundle with tsup
npm run typecheck   # tsc --noEmit
npm test            # vitest
npm run mcp         # run the MCP server locally (tsx)

When adding a provider, port the upstream wire behavior faithfully (same endpoints, params, request bodies, pagination, and field mapping), expose a normalized return type, read every credential from the config module, and register the new tool in src/tools.

License

MIT © liu1700