@uvi/rag-stage-backend

A Backstage backend plugin that powers the RagStage experience. It handles LLM calls, RAG indexing and retrieval, conversation persistence, file upload processing, SSE streaming, and permission enforcement.

Works in conjunction with @uvi/rag-stage.

Installation

yarn --cwd packages/backend add @uvi/rag-stage-backend

Register in packages/backend/src/index.ts:

backend.add(import('@uvi/rag-stage-backend'));

Backend-only configuration

These keys are read exclusively by the backend for credentials. The frontend uses the same structure for safe display metadata.

# app-config.yaml

ragChat:
  # ── Backend-only: permission toggle ──────────────────────────────────────
  permission:
    enabled: false   # default; set true to enforce ragChatChatPermission / ragChatAdminPermission

  providers:
    type: openai             # openai | anthropic | google | custom
    apiToken: ${OPENAI_API_TOKEN}   # ⚠️ backend only
    apiBaseUrl: https://api.openai.com/v1
    embedding:
      model: text-embedding-3-small
    chatModel:
      - id: gpt-4
        name: GPT-4
        temperature: 0.7
      - id: gpt-4-turbo
        name: GPT-4 Turbo
        temperature: 0.3

  # ── Backend-only: source targets ─────────────────────────────────────────
  # id, name, type, description are also read by the frontend for display.
  # target is read ONLY by the backend to fetch and index content.
  sources:
    - id: my-docs
      name: My Custom Docs
      type: custom
      description: Internal runbooks
      target: https://example.com/runbooks   # ⚠️ backend only — URL or absolute file path

Required environment variables

| Variable | Used for | |---|---| | OPENAI_API_TOKEN | OpenAI provider chat + embedding | | ANTHROPIC_API_TOKEN | Anthropic Claude LLM calls | | GOOGLE_API_TOKEN | Google Gemini LLM calls and/or embedding |

Set these in your shell or .env file before running yarn start.

Implemented Features

LLM Providers

• OpenAI — gpt-3.5-turbo, gpt-4, gpt-4o, any OpenAI-compatible endpoint
• Anthropic — claude-3-* via @anthropic-ai/sdk, supporting native system role
• Google Gemini — gemini-pro and others via @google/genai, supporting native systemInstruction
• Common LlmProvider interface with chat() and stream()
• Support for system, user, and assistant roles across all providers

RAG Pipeline

• Catalog — fetches API, Component, Group, Template, User entities; chunks kind/name/ref/description/tags/spec
• TechDocs — fetches rendered HTML from the TechDocs backend, strips chrome, chunks article text
• Custom — fetches a URL (strips HTML) or reads a file path; configured via sources[].target
• File upload — TXT/PDF/DOCX extracted, chunked, and indexed scoped to a conversation
• Chunker — 512-word sliding window with 64-word overlap, metadata attached per chunk
• Embedding — OpenAiEmbeddingProvider, GoogleEmbeddingProvider, AnthropicEmbeddingProvider
• Vector store — InMemoryVectorStore (cosine similarity) for development; interface-based for easy swap to pgvector
• Retrieval — lazy indexing on first query; top-5 chunks injected as system context before conversation history
• Citations — Derived from retrieved RAG chunks and returned in the done event

Prompt Engineering

• System Role Optimization: Behavioral instructions and RAG context are isolated in the system role to prevent LLM hallucination and history repetition.
• General Chat Handling: Professional handling of greetings and capability inquiries without irrelevant citations.

Conversation Persistence

• Knex-backed via Backstage's DatabaseService — SQLite for dev, PostgreSQL for production
• Auto-migration on startup
• Tables: rag_chat_conversations, rag_chat_messages, rag_chat_conversation_sources
• Token usage tracking (promptTokens, completionTokens, totalTokens) persisted per message
• All data scoped per user_ref

Streaming

• POST /chat streams tokens via SSE as they arrive from the LLM
• All three providers implement stream() returning AsyncIterable<LlmStreamEvent>
• Full response assembled and persisted to the database on completion
• Auth/permission errors emitted as { type: 'error' } SSE events

Permissions

• ragChatChatPermission — gates chat, upload, and conversation endpoints
• ragChatAdminPermission — gates GET /admin/config
• Registered via coreServices.permissionsRegistry
• Controlled by ragChat.permission.enabled (default false)

Files

| File | Purpose | |---|---| | src/plugin.ts | Plugin registration, reads config, wires all services | | src/router.ts | Express router — all API endpoints | | src/permissions.ts | Permission definitions (shared with frontend plugin) | | src/services/ConversationService.ts | Knex-backed conversation and message persistence | | src/services/llm/LlmProvider.ts | LlmProvider interface and role definitions | | src/services/llm/LlmService.ts | Builds providers from config, exposes ILlmService | | src/services/llm/OpenAiProvider.ts | OpenAI chat and streaming | | src/services/llm/AnthropicProvider.ts | Anthropic chat and streaming with system role support | | src/services/llm/GoogleProvider.ts | Google Gemini chat and streaming with systemInstruction support | | src/services/rag/RagService.ts | Orchestrates indexing and retrieval | | src/services/rag/EmbeddingProvider.ts | Embedding interface + implementations | | src/services/rag/VectorStore.ts | VectorStore interface + InMemoryVectorStore | | src/services/rag/Chunker.ts | Sliding-window text chunker | | src/services/rag/CatalogRagSource.ts | Catalog entity fetching and chunking | | src/services/rag/TechDocsRagSource.ts | TechDocs HTML fetching and chunking | | src/services/rag/CustomRagSource.ts | URL/file fetching and chunking | | src/services/rag/FileTextExtractor.ts | TXT/PDF/DOCX text extraction | | src/services/migrations/ | Knex migration files |

How the two plugins relate

Both plugins share a single ragChat: config block in app-config.yaml, but each reads a different subset of it:

| Config key | Read by | Purpose | |---|---|---| | ragChat.permission.enabled | Frontend + Backend | Toggles permission enforcement in both | | ragChat.providers.type | Backend only; sanitized via /config | Provider type used to build the LLM client | | ragChat.providers.apiToken | Backend only ⚠️ | Shared provider secret key — never sent to browser | | ragChat.providers.apiBaseUrl | Backend only; sanitized via /config | Optional shared API endpoint for the provider | | ragChat.providers.chatModel[] | Backend only; sanitized via /config | Chat models available for the selected provider | | ragChat.providers.chatModel[].temperature | Backend only; sanitized via /config | Chat completion temperature for that model | | ragChat.providers.embedding.model | Backend only; sanitized via /config | Embedding model used for RAG vector search | | ragChat.sources[].id/name/type/description | Frontend + Backend | Source identity and display | | ragChat.sources[].target | Backend only | URL or file path for custom sources |

Security: apiToken values are resolved from environment variables server-side by Backstage's config pipeline. They are never included in the browser bundle. Always use ${ENV_VAR} substitution — never hardcode tokens.

API Endpoints

All endpoints are mounted under /api/rag-stage/.

| Method | Path | Permission | Description | |--------|------|-----------|-------------| | POST | /chat | rag-stage.chat | Send a message; streams SSE response tokens | | POST | /upload | rag-stage.chat | Upload a file (TXT/PDF/DOCX) for RAG indexing | | GET | /conversations | rag-stage.chat | List conversations for the authenticated user | | POST | /conversations | rag-stage.chat | Create or update a conversation | | DELETE | /conversations/:id | rag-stage.chat | Delete a conversation | | POST | /credentials | rag-stage.chat | Save user-specific API tokens for a model | | GET | /config | rag-stage.chat | Return safe provider metadata for the frontend (no tokens) | | GET | /admin/config | rag-stage.admin | Return safe server-side config (no tokens) |

Permissions are only enforced when ragChat.permission.enabled: true.

SSE event format (POST /chat)

data: {"type":"token","token":"Hello"}
data: {"type":"token","token":" world"}
data: {"type":"done","conversationId":"conv-123","messageId":"msg-456","citations":[],"usage":{"totalTokens":15}}
data: {"type":"error","error":"No LLM provider configured for modelId 'gpt-4'"}

Development

cd plugins/rag-stage-backend
yarn start   # standalone mode
yarn test