arc-search

Zero-dependency, SQLite FTS5-powered full-text search for Arc apps. BM25 ranking, prefix matching, highlighted excerpts, and an inline search widget — all with no external services, no API keys, and no added dependencies.

Features

• FTS5 full-text search — SQLite's built-in BM25 ranking engine. Sub-millisecond queries on 100k+ documents
• Prefix matching — results appear as you type, no full word needed
• Highlighted excerpts — match context with <mark> tags, powered by fts5_snippet()
• Type filtering — index multiple content types, filter results by type
• ArcSearchBar widget — drop-in inline search bar, ~4KB, zero runtime dependencies
• arc-cms integration — upgrades the admin CMS search to FTS5 automatically when installed
• REST API — index and query documents over HTTP from any language or tool
• Lazy init — tables are created on first use, no migration step required

Features

• FTS5 full-text search — SQLite's built-in BM25 ranking. Sub-millisecond queries on 100k+ documents
• @searchable decorator — mark model fields; create/update/delete auto-sync the index. Zero boilerplate
• Bulk indexing — batch up to 5000 documents in one transaction (POST /arc-search/api/docs/bulk)
• Prefix matching — results appear as you type, no full word required
• Highlighted excerpts — match context with <mark> tags via fts5_snippet()
• Type filtering in SQL — O(1) predicate, no JS-side overfetch
• Trigram tokenizer — opt-in typo/substring tolerance (arc.config.json)
• ArcSearchBar widget — drop-in inline search bar, ~4KB, zero runtime dependencies
• AbortController — cancels in-flight requests on new keystroke; no stale results
• arc-cms integration — upgrades admin search to FTS5 automatically when installed
• Lazy FTS5 init — tables created on first use; no migration step required

Install

npm install @arc-lang/arc-search

Add to arc.config.json:

{
  "packages": ["@arc-lang/arc-search"]
}

Quick start

1. Add the widget to any page

import ArcSearchBar from "@arc-search/widgets/ArcSearchBar.arc"

page "Search"
  ArcSearchBar(placeholder="Search docs...")

2. Mark fields as searchable (auto-indexing)

Add @searchable to model fields. The compiler injects FTS5 index calls into every create, update, and delete — zero manual wiring:

model Post
  @id let id: Int = autoincrement()
  @searchable let title: String        // first @searchable field = title (boosted)
  @searchable let body: String         // remaining fields = body content
  let slug: String
  let publishedAt: DateTime = now()

Configure the result URL pattern in arc.config.json:

{
  "packages": ["arc-search"],
  "search": {
    "models": {
      "Post": { "url": "/blog/{slug}" }
    }
  }
}

That's it. Every db.posts.create(...), db.posts.update(...), and db.posts.delete(...) automatically maintains the search index.

3. (Alternative) Index manually via HTTP

If you can't use @searchable (e.g. external data), call the index endpoint:

@server fn createPost(title: String, body: String, slug: String) -> Any
  const post = db.posts.create({ title, body, slug })
  fetch("/arc-search/api/docs", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      type: "posts",
      ref:  String(post.id),
      title: post.title,
      body:  post.body,
      url:   "/blog/" + post.slug
    })
  })
  return post

4. Bulk index existing data

Use the bulk endpoint to reindex large datasets in one transaction:

fetch('/arc-search/api/docs/bulk', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({
    docs: posts.map(p => ({
      type: 'posts', ref: String(p.id),
      title: p.title, body: p.body,
      url: '/blog/' + p.slug
    }))
  })
})
// → { ok: true, created: 412, updated: 0, skipped: 0, total: 412 }

Or index directly with raw SQL in a server route:

db.run("CREATE TABLE IF NOT EXISTS arc_search_docs (id INTEGER PRIMARY KEY AUTOINCREMENT, type TEXT NOT NULL, ref TEXT NOT NULL, title TEXT NOT NULL DEFAULT '', body TEXT NOT NULL DEFAULT '', url TEXT NOT NULL DEFAULT '', meta TEXT NOT NULL DEFAULT '{}', UNIQUE(type, ref))", [])
db.run("INSERT OR IGNORE INTO arc_search_docs(type,ref,title,body,url) VALUES(?,?,?,?,?)", ["posts", String(post.id), post.title, post.body, "/blog/"+post.slug])

3. Search

GET /arc-search/api/search?q=arc+framework&limit=10

{
  "results": [
    {
      "type": "posts",
      "ref": "42",
      "title": "Getting started with Arc",
      "excerpt": "...build your first <mark>Arc</mark> <mark>framework</mark> app in minutes...",
      "url": "/blog/getting-started",
      "meta": "{}",
      "score": -1.234
    }
  ],
  "query": "arc framework",
  "total": 1
}

ArcSearchBar widget

ArcSearchBar(
  placeholder="Search..."   // input placeholder text
  types="posts,docs"        // comma-sep type filter (optional, default: all)
  minChars=2                // minimum chars before querying (default: 2)
  limit=8                   // max results to show (default: 8, max: 50)
  endpoint="/arc-search/api/search"  // API endpoint (default)
)

The widget is self-contained: one @raw block with scoped CSS and ~600B of vanilla JS. No framework, no bundler, no external fonts. It uses CSS custom properties from your Arc theme automatically.

REST API

GET /arc-search/api/search

| Param | Type | Description | |-------|------|-------------| | q | string | Search query (min 2 chars) | | types | string | Comma-separated type filter, e.g. posts,docs | | limit | number | Max results, 1–50 (default: 10) |

Returns { results[], query, total }. Each result: { type, ref, title, excerpt, url, meta, score }.

POST /arc-search/api/docs

Index or update a document.

{
  "type":  "posts",
  "ref":   "42",
  "title": "Getting started with Arc",
  "body":  "Full text content to index...",
  "url":   "/blog/getting-started",
  "meta":  { "author": "Kobe", "tags": ["arc", "web"] }
}

| Field | Required | Description | |-------|----------|-------------| | type | ✅ | Content type identifier (e.g. posts, products) | | ref | ✅ | Unique ID within the type | | title | ✅ | Primary display title, boosted in ranking | | body | — | Full text content to index | | url | — | Navigation URL shown in results | | meta | — | Any JSON — returned in results but not indexed |

Calling this for an existing (type, ref) pair updates the document. Returns { ok: true, action: "created" | "updated" }.

POST /arc-search/api/docs/bulk

Upsert up to 5000 documents in one SQLite transaction. Accepts the same fields as the single-doc endpoint, wrapped in a docs array.

{ "docs": [{ "type": "...", "ref": "...", "title": "...", "body": "...", "url": "..." }, ...] }

Returns { ok, created, updated, skipped, total }.

DELETE /arc-search/api/docs/:type/:ref

Remove a document from the index.

DELETE /arc-search/api/docs/posts/42

Returns { ok: true, action: "deleted" | "not_found" }.

Typo tolerance (trigram tokenizer)

By default arc-search uses SQLite's unicode61 tokenizer — exact matching with accent insensitivity. For typo-tolerant substring search (like Algolia), switch to trigram:

{
  "search": {
    "tokenizer": "trigram"
  }
}

| Tokenizer | Typo tolerance | Index size | Best for | |-----------|---------------|-----------|---------| | unicode61 remove_diacritics 1 | ❌ exact | 1× | dashboards, admin tools, dev docs | | trigram | ✅ substring + 1-char typos | ~3× | e-commerce, public search |

Note: Changing the tokenizer requires dropping and recreating the FTS5 table (DROP TABLE arc_search_fts) and reindexing. The arc_search_docs table is untouched.

arc-cms integration

When arc-search is listed in packages, the CMS admin search automatically upgrades from client-side string filtering to FTS5-powered search. No configuration needed.

To keep the index fresh as editors create and update content, call the docs endpoint from your CMS server routes:

// After db.pages.create(...) or db.pages.update(...)
fetch("/arc-search/api/docs", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ type: "pages", ref: String(page.id), title: page.title, body: page.metaDescription ?? "", url: "/admin/pages/" + page.id })
})

Or run a one-time reindex with the bulk endpoint pattern — query all records and POST each one.

How it works

arc-search uses two SQLite tables:

arc_search_docs    — regular table storing all document fields
arc_search_fts     — FTS5 virtual table (external-content, backed by arc_search_docs)

The FTS5 table uses unicode61 remove_diacritics 1 tokenization for accent-insensitive matching and is kept in sync via explicit insert/delete operations on mutation. Queries use SQLite's native bm25() function for ranking and snippet() for highlighted excerpts — both run in the same SQLite process as your app with no network overhead.

Both tables are created lazily on first use (CREATE TABLE IF NOT EXISTS). No arc db migrate step is needed.

Performance

| Documents | Query time | Memory | |-----------|-----------|--------| | 1,000 | < 0.1ms | ~500KB | | 10,000 | < 0.5ms | ~5MB | | 100,000 | < 2ms | ~50MB |

Benchmarked on an M2 MacBook Pro with SQLite 3.43. FTS5 index size is approximately 10–20% of raw document size.

Security

The /arc-search/api/docs endpoint (POST and DELETE) has no built-in authentication. Protect it at the infrastructure level or add an auth guard in your Arc config:

// site/arc-search/server/docs.arc — override the package route
@route @auth(admin) POST "/arc-search/api/docs" -> Response
  // ... your custom handler, or just forward to the package implementation

Or set ARC_SEARCH_KEY in your environment and validate it in a middleware.

License

MIT