Multi-provider AI agent toolkit for generating language responses, searching the web, extracting page content, and managing long-term memory across 10+ LLM providers.

Built on top of the Vercel AI SDK and Mastra framework, with a small registry of pre-tuned agent prompts (research, summarization, citation answering, query resolution, knowledge-graph extraction, etc.) and tool wrappers around the QwkSearch API.

Install

npm i chat-agent-toolkit

Quick start

import { writeLanguageResponse } from "chat-agent-toolkit";

const response = await writeLanguageResponse({
  provider: "groq",
  apiKey: process.env.GROQ_API_KEY,
  agent: "question",
  query: "Explain transformer attention in two sentences.",
});

console.log(response.content);

response.content is HTML by default (set html: false for raw Markdown). Agents that define an after hook also return parsed data on response.extract.

Language Intelligence Providers

| Provider | 🌍 | Top Model (Others) | 🏆 Benchmarks | 📄 Docs | 🔑 Keys | 💰 Funding | | ---------------------- | --- | --------------------------------------------- | --------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- | ------------ | | Anthropic | 🇺🇸 | Claude Mythos / Opus (Sonnet, Haiku) | 🥇 GPQA Diamond 94.6% · 🥇 SWE-bench 93.9% · 🧬 PhD reasoning | Docs | Keys | ~$60B | | OpenAI | 🇺🇸 | GPT / o3 / Codex (o1, o4, o4-mini, gpt-4o) | 🥇 AIME 2025 100% · 🥇 SWE-bench Pro · 📚 MMLU-Pro 90% | Docs | Keys | ~$180B | | Google | 🇺🇸 | Gemini Pro (Flash, Flash-Lite, Gemma) | 🥇 GPQA 94.1% · 🥇 LiveCodeBench Elo 2439 · 🌐 #1 in 6/13 Vals | Docs | Keys | Public | | xAI | 🇺🇸 | Grok Heavy (Grok-3, Grok Vision) | 🥇 AIME 2025 100% · 🧮 Math competition · ⚡ X integration | Docs | Keys | ~$45B | | Meta | 🇺🇸 | Llama Maverick / Scout (Llama 3.x, CodeLlama) | 🥇 DocVQA 94.4% · 🥇 10M token context · 📊 ChartQA 90% | Docs | Keys | Public | | NVIDIA | 🇺🇸 | Nemotron-Cascade (Llama Nemotron, Kimi) | 🥇 LCB v6 87.2% · 🏅 IMO+IOI+ICPC gold · 🧮 AIME 98.6% | Docs | Keys | Public | | Perplexity | 🇺🇸 | Sonar Reasoning Pro (Sonar Deep Research) | 🥇 Search Arena · 🔍 #1 web-grounded QA · 🌐 Real-time retrieval | Docs | Keys | ~$1B | | Groq | 🇺🇸 | (Llama, DeepSeek, Gemma, Mistral, Qwen) | ⚡ #1 inference speed · 🏎️ Fastest TTFT · 🔧 LPU hardware | Docs | Keys | ~$640M | | Mistral | 🇫🇷 | Mistral Large (Small 4, Codestral, Devstral) | 🥈 Arena Elo 1418 · 🌍 Multilingual MMLU 85.5% · 🚀 Fastest TTFT | Docs | Keys | ~$3.1B | | Together | 🇺🇸 | (Llama, Mistral, Gemma, Qwen, DeepSeek) | 🏗️ Widest open hosting · 💸 Best open-source pricing · 🔧 Fine-tuning | Docs | Keys | ~$225M | | Moonshot | 🇨🇳 | Kimi Reasoning (K2.6, K2) | 🥇 AIME open 96.1% · 🥇 MATH-500 98% · 🥇 HumanEval 99% | Docs | Keys | ~$3.9B | | Zhipu | 🇨🇳 | GLM Reasoning / GLM-4.7 (GLM-4V, CogView) | 🥇 Chatbot Arena Elo 1451 · 🥇 MMLU 96% · 🧮 AIME 95.7% | Docs | Keys | ~$1.8B | | Alibaba | 🇨🇳 | Qwen-Coder / Qwen (Qwen-VL, Qwen-Audio) | 🥇 Codeforces Elo 2056 · 💻 SWE-bench 69.6% · 🏎️ LCB 70.7% | Docs | Keys | Public | | DeepSeek | 🇨🇳 | DeepSeek (DeepSeek-Coder, DeepSeek-VL) | 🥇 IMO gold (open) · 📚 MMLU-Pro 81.2 · 🧮 AIME 87.5% | Docs | Keys | Bootstrapped | | Cloudflare | 🇺🇸 | (Llama, Mistral, Gemma, Qwen, DeepSeek) | 🌐 Edge inference · ⚡ Serverless CDN scale · 🔒 Privacy-first | Docs | Keys | Public | | Ollama | 🇺🇸 | (Llama, Mistral, Gemma, Qwen, DeepSeek) | 🖥️ #1 local inference · 🔒 Fully offline · 🆓 Free self-hosted | Docs | Keys

Built-in agents

The agent option selects a prompt template from AGENT_PROMPTS:

• question — general-purpose Q&A with chat history.
• question-research-engine — long-form, journalistic, citation-style answer.
• query-resolution — rephrases a follow-up question into standalone search queries.
• query-resolution-search — classifies a query and proposes search categories as JSON.
• summarize, summarize-bullets, summary-longtext — single-shot and reduce-style summarization.
• answer-cite-sources — answers a query against a {context} block with [n] citations.
• suggest-followups — emits 4–5 follow-up questions inside <suggestions> tags.
• remember-user — extracts factual user memories as structured JSON.
• knowledge-graph-nodes — builds a temporal knowledge graph from a document.
• results-relevance-filter — picks the most relevant URLs from a search result list.

Templates use {variableName} placeholders filled from the options object (query, article, chat_history, context, etc.).

Agent tools

AGENT_TOOLS registers callable tools that the ReAct agent attaches automatically when a prompt declares them in its tools array:

• web_search — QwkSearch metasearch over 100+ engines.
• extract_page — Mozilla Readability + Mercury extraction with PDF and YouTube support.
• generate_ai_response — proxy to QwkSearch's hosted /language endpoint.

Set QWKSEARCH_URL and QWKSEARCH_API_KEY in the environment, or pass baseURL/apiKey per-call.

Memory

import {
  SimpleMemory,
  MemoryAgent,
  DrizzleMemoryStorage,
} from "chat-agent-toolkit";

• SimpleMemory — in-memory store implementing IMemoryStorage.
• DrizzleMemoryStorage — Drizzle ORM-backed persistence; pair with createMemorySchema().
• MemoryAgent — high-level wrapper that uses the remember-user agent to extract facts from chat turns and writes them to a storage backend.

See src/memory/README.md and src/memory/ARCHITECTURE.md for details.

Mastra Framework Integration

The toolkit integrates with Mastra for advanced agent orchestration:

import { createMastraInstance } from "chat-agent-toolkit/mastra";

const mastra = createMastraInstance({
  apiKey: process.env.ANTHROPIC_API_KEY,
  model: "claude-3-5-sonnet-20241022"
});

Key features:

• Agents — define agents with tools and memory integration
• Workflows — graph-based task orchestration with .then(), .branch(), .parallel() combinators
• 40+ model routing — seamless provider switching (OpenAI, Anthropic, Groq, etc.)
• Observational memory — built-in tracing and context management
• MCP servers — register and author Model Context Protocol servers
• Standalone or embedded — deploy as a service or inside React/Next/Node apps

See src/mastra/ for integration examples and API reference.

Package layout

src/
  agents/                    # prompt + tool registry, generate function, model list
    agent-prompts.ts         # AGENT_PROMPTS, extractJSONFromLanguageReply
    agent-tools.ts           # AGENT_TOOLS (web_search, extract_page, ...)
    generate-language.ts     # writeLanguageResponse — main entry point
    generate-language-types.ts
    language-model-names.ts  # LANGUAGE_MODELS, LANGUAGE_PROVIDERS
    llm-providers.ts         # createLLMProvider — chat-model factory
    index.ts                 # barrel export
  memory/                    # SimpleMemory, MemoryAgent, Drizzle storage
  providers/                 # alternate provider abstractions (model registry)
  utils/                     # markdown-to-html, document helpers
  index.ts                   # public entry — re-exports agents + memory

Build

npm run build      # vite build
npm run make       # clean + build with extra heap
npm run test       # vitest

Vite bundles ES + CJS targets to dist/, emits .d.ts files alongside, and applies a fs/promises polyfill so the bundle works in browser and edge runtimes (Cloudflare Workers, Vercel Edge).

Resources

• Vercel AI SDK generateText docs
• Hugging Face tutorials
• Illustrated Transformer
• Building a Transformer with PyTorch
• LLM training example

Alternative Agents Frameworks

| # | Framework | Features | Stars | Language | Maker | |---:|---|---|---:|---|---| | 1 | Hermes Agent | Self-improving agent with a built-in learning loop; distills reusable skills from past runs, persists experience across sessions, "grows with you" over time | 211k | Python | Nous Research | | 2 | AutoGPT | One of the original autonomous agents; goal-driven loop that self-prompts to decompose and execute tasks. Now a low-code platform: visual block-based builder, deploy/run continuous agents, and an agent marketplace | 185k | Python/TypeScript | Significant Gravitas | | 3 | n8n | Workflow automation, native AI features, 400+ integrations, visual/code hybrid | 175k | TypeScript | n8n.io | | 4 | LangChain / LangGraph | Full ecosystem: LangChain for chains/model+tool abstractions, LangGraph (36.7k) for graph-native orchestration with explicit state, branching, loops, checkpoints, human-in-the-loop, durable execution, streaming; LangGraphJS (3.1k) mirrors it for JS runtimes | 141k | Python + TS/JS | LangChain AI | | 5 | Dify | Visual workflow builder, RAG pipelines, agent apps, deployment tooling | 130k–147k+ | TypeScript | LangGenius | | 6 | OpenHands | Autonomous dev agent — writes/edits code, runs shell commands, browses the web, executes in a sandbox; strong local secure-workflow story | 80k | Python | All-Hands-AI (formerly OpenDevin) | | 7 | MetaGPT | Simulates a software company (PM/architect/dev/QA roles), SOP-driven pipelines, generates specs, docs, and code from one prompt | 69k | Python | Foundation Agents | | 8 | Mem0 | Memory layer, not an orchestrator: extraction, consolidation, scoped retrieval, long-term user/agent memory; drops into any framework | 60k | Python/TypeScript | mem0ai | | 9 | AutoGen | Dialogue-first multi-agent conversations, async event-driven messaging, distributed collaboration, observability, code execution | 60k | Python | Microsoft | | 10 | CrewAI | Role-based "crews," task delegation and collaboration, memory, checkpointing, async flows; lean (no LangChain dependency) | 55k | Python | CrewAI Inc. / João Moura | | 11 | LlamaIndex | Retrieval-centric agents, event-driven Workflows, RAG-first data connectors, memory, structured query planning | 51k | Python | LlamaIndex | | 12 | smolagents | Minimal code-first agents that write actions as Python code, sandboxed execution, tiny surface area, model-agnostic | 28k | Python | Hugging Face | | 13 | OpenAI Agents SDK | Lightweight agent loop, handoffs between agents, guardrails, sessions, built-in tracing, sandboxed tool execution | 28k | Python/TypeScript | OpenAI | | 14 | Mastra | TS-native, batteries-included: agents, graph-based workflows (.then()/.branch()/.parallel()), 40+ model routing, observational memory, human-in-the-loop, authored MCP servers, built-in evals + observability; deploys standalone or inside React/Next/Node | 26k | TypeScript | Mastra (mastra.ai) | | 15 | Vercel AI SDK | Streaming, tool calling, structured outputs, agent loops, MCP support, provider-agnostic, first-class UI hooks; runs on edge/Workers | 25k | TypeScript | Vercel | | 16 | Zep | Agent memory platform built on a temporal knowledge graph; long-term context, fact extraction, cross-session recall | 4.7k | Python/TypeScript | getzep |