Track B: Business Context LLM Reviewer

This is NOT generic code review. This system understands business context from documentation, ADRs, and requirements.

What Makes This Different

❌ Generic Code Review (what ChatGPT/Copilot do):

"This SQL query is vulnerable to injection"
"Consider adding error handling"
"Variable could be more descriptive"

✅ Business Context Review (what WE do):

"This query violates ADR-003 Section 6: Database Performance Standards
 [ADR-003:lines 78-85]. The ADR explicitly requires parameterized queries
 because our architecture uses connection pooling, and SQL injection here
 would compromise the entire pool.

 Ticket PB-123 states 'must support high-concurrency search', but this
 implementation will cause connection exhaustion under load.

 Recommended: Use $1, $2 parameterization as shown in ADR-003 example."

Key Components

1. ADR Analyzer

Reads and understands Architecture Decision Records:

• WHY decisions were made
• Context and rationale
• Consequences and trade-offs
• Finds relevant ADRs for code changes

2. Business Context Extractor

Extracts requirements from:

• Jira tickets (description, acceptance criteria)
• Code comments with business logic
• Performance targets and SLAs
• User stories and constraints

3. LLM Client with Citation Validator

• Requires EXACT quotes from source documents
• Validates all citations against original text
• Rejects hallucinated claims
• Supports streaming for real-time feedback

4. Review Orchestrator

Coordinates specialists:

• ADR Compliance Reviewer
• Business Logic Validator
• Architecture Reviewer
• Performance Context Reviewer

Example Usage

import { ReviewOrchestrator } from '@review-policy/llm-reviewer';
import { GitPRContextGatherer } from '@review-policy/policy-validator';

// Configure LLM
const orchestrator = new ReviewOrchestrator({
  model: 'claude-sonnet-4.5',
  apiKey: process.env.ANTHROPIC_API_KEY!
});

// Gather PR context
const gatherer = new GitPRContextGatherer({
  repoPath: './demo-enterprise-repo',
  branch: 'pr/security-violations',
  baseBranch: 'master'
});

const prContext = await gatherer.gatherContext();

// Run business-context review
const result = await orchestrator.review(prContext, {
  adrDirectory: './demo-enterprise-repo/docs/adr',
  policyPath: './demo-enterprise-repo/CODE-POLICY.md'
});

console.log(`Found ${result.findings.length} findings`);
console.log(`Referenced ${result.stats.adrsReferenced} ADRs`);
console.log(`${result.stats.citationsVerified}/${result.stats.citationsProvided} citations verified`);

for (const finding of result.findings) {
  console.log(`\n${finding.title}`);
  console.log(`Impact: ${finding.businessImpact}`);
  console.log(`Citations:`);
  for (const cite of finding.citations) {
    console.log(`  [${cite.source}] "${cite.quote}"`);
  }
}

Output Example

🤖 Starting Business Context Review

📋 Loading ADRs...
   Loaded 3 ADRs from ./demo-enterprise-repo/docs/adr

🔍 Finding relevant ADRs for code changes...
   Found 2 relevant ADRs

   - ADR-003: database performance (high relevance)
   - ADR-005: security standards (high relevance)

💼 Extracting business context...
   Requirements: 5
   Acceptance Criteria: 3
   Business Rules: 2

🔬 Running specialist reviews...

   🏛️  ADR Compliance Review...
      ✓ Reviewed against ADR-003
      ✓ Reviewed against ADR-005

   💡 Business Logic Validation...
      ✓ Validated 3 business logic aspects

   🏗️  Architecture Review...
      ✓ Identified 1 architectural concerns

📊 Synthesizing findings...

✅ Review complete!

FINDINGS:
----------

1. ADR Violation: ADR-003 (HIGH)
   Title: SQL Injection in User Search
   Impact: ADR-003 requires parameterized queries for connection pooling safety.
          This violation could compromise entire pool affecting all users.
   Citation: [ADR-003:lines 78-85] "All database queries MUST use parameterized
            statements to prevent SQL injection and protect connection pool integrity"
   Location: src/routes/search.ts:14
   Fix: Replace template literal with: db.query('SELECT * FROM users WHERE id = $1', [id])

2. Business Logic Mismatch (HIGH)
   Title: Missing Phone Number Search
   Impact: Ticket PB-101 requires email OR phone search, but only email is implemented
   Citation: [PB-101:acceptance-criteria] "Users can search by email OR phone number"
   Location: src/routes/search.ts:10-20
   Fix: Add phone_number parameter and search logic

STATS:
------
- 2 findings
- 2 ADRs referenced
- 3/3 citations verified (100%)

Anti-Hallucination Features

1. Mandatory Citations: Every claim must reference source document
2. Quote Verification: All quotes checked against original text
3. Confidence Scoring: Low-confidence responses flagged
4. Source Attribution: Clear trail from finding to documentation

Requirements

• Node.js 18+
• Anthropic API key
• ADRs in markdown format
• CODE-POLICY.md
• Jira integration (optional)

Testing

npm test

License

MIT