@telebort/question-banks

Question Banks SDK for Telebort educational assessments. Provides Zod schemas, TypeScript types, public-stem quiz generation, v2.0 Knowledge Graph access, CAT (Computerized Adaptive Testing) orchestration, server-side grading, and rubric contracts.

v3.0.0 - Server contract for the public stem/private key architecture: public stems ship without answers, private key shards stay server-only, gradeResponse() grades selected answers, createServerCAT() wraps CAT for practice APIs, and ./knowledge and ./rubric are first-class subpath exports.

Installation

# SDK + CLI (recommended)
npm install @telebort/question-banks zod

# CLI only (no install required)
npx @telebort/question-banks <command>

Quick Start (3 Scenarios)

Scenario A: Validate Course Files (CLI)

Quickly validate JSON files against schemas:

# Validate all course files in a directory
npx @telebort/question-banks validate ./data/courses/ --verbose

# Show question bank statistics
npx @telebort/question-banks stats --data-dir ./data/courses/

# Run guessability analysis (anti-guessing QA)
npx @telebort/question-banks score ./data/courses/ai-1.json

Scenario B: Build a Quiz App (SDK)

Use the factory function to create a fully-featured SDK instance:

import { createQuestionBanksSDK } from '@telebort/question-banks'

const sdk = createQuestionBanksSDK({
  dataSource: {
    async loadCourses() {
      const res = await fetch('https://cdn.telebort.com/exit-tickets/v2.0/courses/CS1.json')
      return [await res.json()]
    }
  }
})

// Generate a quiz
const quiz = await sdk.quiz.generateQuiz({ questionCount: 5 })

// Check an answer with feedback
const result = await sdk.validate.checkAnswer('cs-1-l1-q1', 'B', 'formative')
// { isCorrect: true, feedback: {...}, misconceptionId: null }

Scenario C: Validate Data Structures (Schemas Only)

Import just the Zod schemas for lightweight validation:

import { QuestionSchema, CourseSchema } from '@telebort/question-banks/schemas'

const result = QuestionSchema.safeParse(myQuestion)
if (!result.success) {
  console.error('Validation failed:', result.error.issues)
}

// TypeScript type inference
import type { Question, Course } from '@telebort/question-banks'

Entry Points

| Import Path | Purpose | Use Case | |-------------|---------|----------| | @telebort/question-banks | Full SDK factory | Quiz apps, data access | | @telebort/question-banks/schemas | Zod schemas only | Lightweight validation | | @telebort/question-banks/validation | ValidationClient | Answer checking | | @telebort/question-banks/premium | PremiumClient (JWT) | Full API access | | @telebort/question-banks/irt-calibration | IRT calibration tools | Psychometric analysis | | @telebort/question-banks/cat | CAT engine | Adaptive testing | | @telebort/question-banks/knowledge | Knowledge Graph factories | Topic and taxonomy queries | | @telebort/question-banks/rubric | Rubric schemas and helpers | Native curriculum rubric validation | | @telebort/question-banks/server | Server-only grading and CAT facade | Practice APIs over private key shards |

v3 Stem/Key Architecture

v3 makes the three-tier model real at the API boundary:

• Public stems are safe to send to clients. They include the question prompt, options, and metadata needed to render an item, but no correctAnswer, no correctAnswerText, no option isCorrect, and no misconception feedback that reveals the key.
• Private key shards are server-only. Each entry is keyed by questionId and has { correctAnswer, correctAnswerText, irt }. The key entry is required for grading and for CAT ability updates because the IRT calibration lives with the private key.
• Server grading uses gradeResponse(keyShardEntry, response) from @telebort/question-banks/server. It returns { correct, correctAnswer, matched }; matched is the normalized selected token set that intersects the key. Only return correct or server-approved feedback to untrusted clients.
• Server CAT uses createServerCAT(config) from @telebort/question-banks/server. Its step({ stemItems, keyShard, theta, responses }) call joins stems to private IRT, replays graded or raw responses through the existing CAT engine/EAP estimator, evaluates stopping rules, and returns { nextItemId, theta, se, done, reason }.
• Rubrics use the @telebort/question-banks/rubric export. This is separate from answer-key grading and validates weighted or banded rubric payloads for native curriculum workflows.

Consumers pinned to ^2.6 must migrate to 3.x before using v3 stem/key data. Code that previously relied on client-visible correctAnswer, correctAnswerText, option.isCorrect, or client-side sdk.validate.checkAnswer() must move grading behind a server boundary and call gradeResponse() with the private key shard.

import { createServerCAT, gradeResponse } from '@telebort/question-banks/server'

const result = gradeResponse(keyShard[questionId], { selectedAnswer: 'B' })

const cat = createServerCAT({
  keyShard,
  stoppingRules: { maxItems: 20, minItems: 5, seThreshold: 0.5 },
})

const next = cat.step({
  stemItems,
  theta: session.theta,
  responses: [{ questionId, correct: result.correct }],
})

⚠️ Migration Notice

If you're using the deprecated CLI package:

# Uninstall old CLI package
npm uninstall @telebort/question-banks-cli

# Use the canonical package (CLI included)
npx @telebort/question-banks validate ./data/

What's New in v3.0.0

• Public stem/private key split: Public stems are answer-free; correct answers, answer-revealing feedback, and IRT keys remain server-only.
• Server-side grading and CAT: The ./server export provides private-key grading and server-facing adaptive testing helpers.
• Rubric contracts: The ./rubric export provides the ratified rubric schemas and validation helpers.

The Knowledge Graph module is now also a first-class ./knowledge subpath export.

Architecture

The SDK provides public-stem access to question bank data via CDN plus server-only grading/CAT contracts:

┌─────────────────────────────────────────────────────────────┐
│                    SDK Access Patterns                       │
├─────────────────────────────────────────────────────────────┤
│                                                              │
│  v1.x Compatibility (Course-based)                          │
│  └── sdk.data.getCourse('CS1')                              │
│                                                              │
│  v2.0 Knowledge Graph (Topic-based)                         │
│  ├── sdk.knowledge.getByTopic('algorithms/sorting')         │
│  └── sdk.taxonomy.getDomains()                              │
│                                                              │
│  Premium API (JWT-authenticated)                             │
│  └── PremiumClient with offline fallback                    │
│                                                              │
└─────────────────────────────────────────────────────────────┘

Public stem rendering can run client-side. Grading, correct-answer access, and CAT response updates must run server-side with private key shards.

Quick Start (CDN Mode)

Fetch questions from CDN for automatic updates:

import { createQuestionBanksSDK } from '@telebort/question-banks'

const CDN_BASE = 'https://cdn.telebort.com/exit-tickets/v2.0/courses'

const sdk = createQuestionBanksSDK({
  dataSource: {
    async loadCourses() {
      const codes = ['AI1', 'M1', 'W1', 'CS1']
      const courses = await Promise.all(
        codes.map(code => fetch(`${CDN_BASE}/${code}.json`).then(r => r.json()))
      )
      return courses
    }
  }
})

// Generate a quiz
const quiz = await sdk.quiz.generateQuiz({ questionCount: 5 })

// Check an answer with feedback
const result = await sdk.validate.checkAnswer('ai-1-l1-q1', 'B', 'formative')
// { isCorrect: true, feedback: {...}, misconceptionId: null }

v2.0 Knowledge Graph

Access questions by topic instead of course. Query across curricula with the 4-level taxonomy.

Full Guide: See KNOWLEDGE-GRAPH.md for complete API reference and examples.

Taxonomy Structure

Domain (L1)          Strand (L2)          Topic (L3)
    │                    │                    │
computer-science ──→ algorithms ──────→ sorting
                     data-structures ──→ arrays, trees, graphs

web-development ──→ javascript ───────→ arrow-functions, promises
                     html-css ─────────→ flexbox, grid

6 Domains: computer-science, artificial-intelligence, web-development, mathematics, computational-thinking, mobile-development

Basic Usage

import { createKnowledgeModule, createTaxonomyModule } from '@telebort/question-banks'

const CDN_BASE = 'https://cdn.telebort.com'

// Create modules
const knowledge = createKnowledgeModule({ cdnBase: CDN_BASE })
const taxonomy = createTaxonomyModule({ cdnBase: CDN_BASE })

// Get questions by topic path
const sorting = await knowledge.getByTopic('computer-science/algorithms/sorting')
console.log(`${sorting.length} sorting questions`)  // Returns TopicQuestionRef[]

// Browse taxonomy
const domains = await taxonomy.getDomains()
const strands = await taxonomy.getStrands('computer-science')

Advanced Search with Filters

// Find medium-difficulty sorting questions from CS1
const results = await knowledge.search({
  domain: 'computer-science',
  strand: 'algorithms',
  topic: 'sorting',
  difficulty: 'medium',
  courseCode: 'CS1',
  limit: 10,
  offset: 0
})

// Paginate through all web-dev questions
let page = await knowledge.search({ domain: 'web-development', limit: 50, offset: 0 })

Error Handling with Debug Logging

const knowledge = createKnowledgeModule({
  cdnBase: CDN_BASE,
  cacheEnabled: true,           // Enable caching
  debug: true,                  // Log to console
  onError: (error, context) => {
    console.error(`[${context}]`, error.message)
    // Send to monitoring: Sentry.captureException(error)
  }
})

v2.0 CDN Structure

/exit-tickets/v2.0/
├── courses/{CODE}.json                      # Course-based (v1.x compatible)
├── by-topic/{domain}/{strand}/{topic}.json  # Topic-based indexes
├── taxonomy/topics.json                     # 4-level topic tree
└── master/
    ├── knowledge-bank-index.json            # Shard metadata (50KB)
    └── shards/shard-{00001-00026}.json      # 10K questions each (~20MB)

Total: 1,449,830+ questions in 26 CDN shards for global delivery.

CDN Endpoints

Base URL: https://cdn.telebort.com

| Endpoint | Size | Content | |----------|------|---------| | /exit-tickets/v2.0/courses/{CODE}.json | ~400KB | Single course | | /exit-tickets/v2.0/taxonomy/topics.json | ~49KB | 4-level taxonomy | | /exit-tickets/v2.0/master/knowledge-bank-index.json | ~7KB | Shard metadata | | /exit-tickets/v2.0/master/shards/shard-{00001-00026}.json | ~20MB | 10K questions each |

Available Courses: AI1, AI2, AI25, AI3, M1, M2, M3, W1, W2, W3, W35, W4, F1, F2, JC, BBW, BBP, BBD, BBAI, CS1, CS2, CS3, CS4

Static Mode (Bundled JSON)

For offline-first apps, bundle the JSON with your build:

import { createQuestionBanksSDK } from '@telebort/question-banks'
import coursesData from './data/courses.json'

const sdk = createQuestionBanksSDK({
  dataSource: {
    async loadCourses() {
      return coursesData.courses
    }
  }
})

// All SDK features work offline
const courses = await sdk.data.getCourses()
const quiz = await sdk.quiz.generateQuiz({ questionCount: 5 })
const result = await sdk.validate.checkAnswer('ai-1-l1-q1', 'B', 'formative')

Premium Client (JWT-Authenticated)

For full API access with offline fallback (v1.3.0+):

import { PremiumClient } from '@telebort/question-banks/premium'

const client = new PremiumClient({
  token: 'your-jwt-token',

  // Offline support (v1.3.0)
  offlineFallback: true,   // Use cache when network fails
  cacheTTL: 300000,        // 5 minutes cache TTL

  // Rate limiting (built-in: 10 req/s, burst 20)
  timeout: 30000,          // Request timeout

  // Debug logging
  logger: (msg, level) => console.log(`[${level}] ${msg}`),
})

// Preload courses for offline use
await client.preloadForOffline(['cs-1', 'cs-2', 'ai-1'])

// Fetch questions (uses cache if offline)
const { questions, total, hasMore } = await client.getQuestions({
  courseId: 'cs-1',
  difficulty: 'medium',
  limit: 10,
})

// Grade an assessment
const result = await client.gradeAssessment({
  assessmentId: 'uuid-here',
  courseId: 'cs-1',
  submittedAt: new Date().toISOString(),
  responses: [
    { questionId: 'q1', selectedAnswer: 'A' },
    { questionId: 'q2', selectedAnswer: 'B' },
  ]
})

// Cache management
const stats = client.getCacheStats()
console.log(`Cached entries: ${stats.entries}`)
client.clearCache()

SDK API Reference

The SDK is organized into focused modules. Each module handles a specific domain:

| Module | Purpose | Key Methods | |--------|---------|-------------| | sdk.data | Data access | getCourses(), getCourse(), getQuestion() | | sdk.quiz | Quiz generation | generateQuiz(), getLessonQuiz() | | sdk.validate | Answer checking | checkAnswer() | | sdk.query | Filtering | filterQuestions(), getRandomQuestions() | | sdk.search | Full-text search | search(), searchByTags() | | sdk.analytics | Statistics | getStatistics(), getCourseStatistics() | | sdk.knowledge | v2.0 Knowledge Graph | getByTopic(), getByDomain(), search() | | sdk.taxonomy | Topic taxonomy | getDomains(), getTopics() |

Common Workflows

Build a quiz from a specific course:

const quiz = await sdk.quiz.generateQuiz({
  courseId: 'cs-1',
  questionCount: 10,
  difficulty: 'medium'
})

Check an answer with feedback:

const result = await sdk.validate.checkAnswer('cs-1-l1-q1', 'B', 'formative')
// Returns: { isCorrect, feedback, misconceptionId }

Search by topic (v2.0 Knowledge Graph):

const questions = await sdk.knowledge.getByTopic('computer-science/algorithms/sorting')

Get course statistics:

const stats = await sdk.analytics.getCourseStatistics('ai-1')
// Returns: { totalQuestions, byDifficulty, byType, ... }

SDK Interface (TypeScript)

interface QuestionBanksSDK {
  // v1.x Course-based access
  data: {
    getCourses(): Promise<Course[]>
    getCourse(courseId: string): Promise<Course | null>
    getLesson(courseId: string, lessonNumber: number): Promise<Lesson | null>
    getQuestion(questionId: string): Promise<Question | null>
    getAllQuestions(): Promise<Question[]>
  }

  query: {
    filterQuestions(filters: QuestionFilters): Promise<QueryResult<Question>>
    getRandomQuestions(count: number, filters?: QuestionFilters): Promise<Question[]>
  }

  quiz: {
    generateQuiz(criteria: QuizCriteria): Promise<{ questions: Question[]; metadata: object }>
    getLessonQuiz(courseId: string, lessonNumber: number): Promise<Question[]>
  }

  search: {
    search(options: SearchOptions): Promise<QueryResult<Question>>
    searchByTags(tags: string[], matchAll?: boolean): Promise<Question[]>
  }

  analytics: {
    getStatistics(): Promise<Statistics>
    getCourseStatistics(courseId: string): Promise<Statistics>
  }

  validate: {
    checkAnswer(questionId: string, answer: string, mode: 'formative' | 'summative'): Promise<AnswerResult>
  }

  // v2.0 Topic-based access (NEW)
  knowledge: {
    getByTopic(topicPath: string): Promise<TopicIndex>
    getByDomain(domainId: string): Promise<Question[]>
    search(filters: KnowledgeFilters): Promise<Question[]>
  }

  taxonomy: {
    getDomains(): Promise<string[]>
    getTopics(): Promise<TaxonomyTree>
  }
}

Zod Schemas (Free Validation)

Import Zod schemas for offline validation:

import { QuestionSchema, CourseSchema } from '@telebort/question-banks/schemas'
import { z } from 'zod'

// Validate a question
const result = QuestionSchema.safeParse(myQuestion)
if (!result.success) {
  console.error(result.error.issues)
}

// Type inference
type Question = z.infer<typeof QuestionSchema>

Available Schemas

// Question schemas
QuestionSchema          // Full question with v1.1 enhancements
OptionSchema            // Answer option with misconception tracking
FeedbackSchema          // Structured feedback (short/detailed/socraticHint)

// Course schemas
CourseSchema            // Course with lessons and questions
LessonSchema            // Lesson with questions

// Assessment schemas
AssessmentSubmissionSchema  // User assessment submission
GradedAssessmentSchema      // Graded result with misconception analysis

CLI Reference

The CLI provides validation, statistics, and QA tools. Use via npx (no install) or globally:

npx @telebort/question-banks <command> [options]

Commands

| Command | Description | |---------|-------------| | validate <path> | Validate JSON files against schemas | | stats | Show question bank statistics | | quiz | Generate sample quizzes | | score | Guessability analysis (anti-guessing QA) |

validate - Schema Validation

# Validate a single file
npx @telebort/question-banks validate ./data/courses/ai-1.json

# Validate all files in a directory
npx @telebort/question-banks validate ./data/courses/

# Verbose output with all errors
npx @telebort/question-banks validate ./data/courses/ --verbose

stats - Question Bank Statistics

# Show aggregate statistics across all courses
npx @telebort/question-banks stats --data-dir ./data/courses/

# Statistics for a specific course
npx @telebort/question-banks stats --course ai-1 --data-dir ./data/courses/

quiz - Sample Quiz Generation

# Generate 5 random questions
npx @telebort/question-banks quiz --data-dir ./data/courses/

# Generate 10 questions from a specific course
npx @telebort/question-banks quiz --course ai-1 --count 10 --data-dir ./data/courses/

# Filter by difficulty and show correct answers
npx @telebort/question-banks quiz --difficulty hard --show-answers --data-dir ./data/courses/

score - Guessability Analysis

Run psychometric QA analysis to detect testwiseness vulnerabilities:

# Analyze a single course file
npx @telebort/question-banks score ./data/courses/ai-1.json

# Analyze with detailed output
npx @telebort/question-banks score ./data/courses/ --verbose

Output includes:

• Guessability score (0-100, lower is better)
• Distractor quality metrics
• Option length bias detection
• Grammatical cue warnings

Question Schema (v1.1)

Questions include v1.1 enhancements for misconception tracking:

interface Question {
  questionId: string           // "ai-1-l1-q1"
  globalId: string             // "exit-ticket-0351"
  questionType: QuestionType   // vocabulary, code_understanding, etc.
  questionArchetype?: string   // vocabulary, trace, bebras, blockmodel

  prompt: string
  hasCodeBlock: boolean
  codeLanguage: string | null
  codeContent: string | null

  // v1.1: Misconception targeting
  misconceptionTargets?: string[]

  options: Option[]            // 4 options with misconception feedback
  correctAnswer: 'A' | 'B' | 'C' | 'D'

  metadata: {
    difficulty: 'easy' | 'medium' | 'hard' | 'challenge'
    bloomsTaxonomy: 'remember' | 'understand' | 'apply' | ...
    tags: string[]
  }
}

interface Option {
  key: 'A' | 'B' | 'C' | 'D'
  text: string
  isCorrect: boolean

  // v1.1: Misconception feedback
  misconceptionId?: string     // "INPUT_PROMPT_VS_OUTPUT"
  feedback?: {
    short: string              // "Not quite! That's the prompt."
    detailed: string           // Full explanation
    socraticHint?: string      // "Where does the program display its result?"
  }
}

TypeScript Support

All types are inferred from Zod schemas:

import type {
  Question,
  Course,
  AssessmentSubmission,
  GradedAssessment
} from '@telebort/question-banks'

License

MIT

Related Documentation

• ARCHITECTURE.md - Technical deep-dive (CAT, IRT, CDN)
• KNOWLEDGE-GRAPH.md - Topic-based query guide
• ../ECOSYSTEM-OVERVIEW.md - High-level ecosystem overview

Support

• Issues: GitHub Issues