LanceDB Viewer

A simple web-based viewer for LanceDB databases. Browse tables, filter columns, apply SQL WHERE clauses, and explore your vector database data through an intuitive web interface.

✨ Features

• 🌐 Modern Web Interface: Clean, professional dark theme with responsive design
• 📊 Advanced Table Browsing: View all tables in your LanceDB database with metadata
• 🔍 Smart Column Filtering: Selectively display columns to focus on relevant data
• 🗃️ SQL WHERE Clauses: Apply powerful filtering using familiar SQL syntax
• 📄 Intelligent Pagination: Navigate through large datasets efficiently
• 🏗️ Schema Inspection: View detailed table schemas and field metadata
• 🚀 High Performance: Optimized queries using LanceDB's native capabilities
• 🔒 Enterprise Security: Input validation, rate limiting, and security headers
• 🛡️ SQL Injection Protection: Built-in safeguards against malicious queries
• 📱 Cross-Platform: Works on Linux, macOS, and Windows
• 🔧 Configurable: Extensive environment variable configuration options

🚀 Quick Start

Global Installation (Recommended)

npm install -g lanceviewer

Local Installation

npm install lanceviewer

Basic Usage

# Start the viewer (serves current directory)
lanceviewer

# Specify a database path
lanceviewer /path/to/your/database

# Use custom settings
lanceviewer --port 8080 --host 0.0.0.0 /path/to/database

Local Installation

npm install lanceviewer

Usage

Command Line

# Start the viewer with default settings (serves current directory)
lanceviewer

# Specify a database path
lanceviewer /path/to/database

# Use a custom port
lanceviewer --port 8080 /path/to/database

# Show help
lanceviewer --help

Programmatic Usage

const { createServer } = require('lanceviewer');

// Start the server
const server = createServer();
server.listen(3000, () => {
  console.log('LanceDB Viewer running at http://localhost:3000');
});

Web Interface

1. Open your browser and navigate to http://localhost:3000
2. Enter the path to your LanceDB database directory
3. Click "Open" to load the database
4. Select a table from the sidebar to view its data
5. Use column checkboxes to filter which columns to display
6. Enter SQL WHERE clauses to filter rows (e.g., id > 100, name LIKE 'test%')

SQL WHERE Clause Examples

The viewer supports SQL-like WHERE clauses for filtering data:

-- Numeric comparisons
id > 100
price BETWEEN 10.0 AND 50.0

-- String matching
name LIKE 'test%'
status = 'active'

-- Multiple conditions
id > 100 AND status = 'active'
created_at > '2024-01-01' OR priority = 'high'

-- Vector operations (if supported by LanceDB)
score > 0.8

Requirements

• Node.js >= 16.0.0
• A LanceDB database directory

Development

# Clone the repository
git clone https://github.com/littlellmjs/lanceviewer.git
cd lanceviewer

# Install dependencies
npm install

# Start development server
npm run dev

# Build for production
npm run build

📖 Documentation

Command Line Options

lanceviewer [options] [database-path]

Options:
  -h, --help              Show help information
  -v, --version           Show version number
  -p, --port <port>       Server port (default: 3000)
  --host <host>           Server host (default: localhost)
  --verbose               Enable verbose logging

Arguments:
  database-path           Path to LanceDB database directory (default: current directory)

Environment Variables

| Variable | Description | Default | |----------|-------------|---------| | PORT | Server port | 3000 | | HOST | Server host | localhost | | LOG_LEVEL | Logging level (error, warn, info, debug) | info | | CORS_ORIGIN | CORS allowed origin | * | | MAX_REQUEST_SIZE | Maximum request body size in bytes | 1048576 | | REQUEST_TIMEOUT | Request timeout in milliseconds | 30000 | | MAX_CONCURRENT_REQUESTS | Rate limiting threshold | 10 | | ENABLE_HTTPS | Enable HTTPS mode | false | | SSL_CERT_PATH | Path to SSL certificate | - | | SSL_KEY_PATH | Path to SSL private key | - |

SQL WHERE Clause Examples

LanceDB Viewer supports SQL-like WHERE clauses for advanced filtering:

-- Basic comparisons
id > 100
name = 'John Doe'
price BETWEEN 10.0 AND 50.0

-- String operations
name LIKE 'John%'
email LIKE '%@example.com'

-- Logical operators
active = true AND age > 21
status IN ('active', 'pending')

-- Complex conditions
(created_at > '2024-01-01' AND category = 'electronics') OR priority = 'high'

Security Note: WHERE clauses are validated to prevent SQL injection attacks. Dangerous patterns like DROP, DELETE, UPDATE, etc. are blocked.

🔌 API Reference

REST Endpoints

| Method | Endpoint | Description | Status | |--------|----------|-------------|--------| | GET | / | Main web interface | ✅ | | GET | /api/health | Health check endpoint | ✅ | | POST | /api/open | Open database at specified path | ✅ | | GET | /api/current-path | Get current database path | ✅ | | GET | /api/tables | List all tables in current database | ✅ | | GET | /api/table/:name | Get table data with optional query parameters | ✅ |

Query Parameters for /api/table/:name

| Parameter | Type | Description | Default | Limits | |-----------|------|-------------|---------|--------| | limit | number | Number of rows to return | 100 | 1-1000 | | offset | number | Number of rows to skip | 0 | ≥ 0 | | columns | string | Comma-separated column names | All columns | Valid column names only | | where | string | SQL WHERE clause | No filter | Max 10KB, safe SQL only |

Example API Usage

# Health check
curl http://localhost:3000/api/health

# List all tables
curl http://localhost:3000/api/tables

# Get first 50 rows with specific columns
curl "http://localhost:3000/api/table/my_table?limit=50&columns=id,name,vector"

# Filter rows with WHERE clause
curl "http://localhost:3000/api/table/my_table?where=id%20%3E%20100%20AND%20active%20%3D%20true"

# Open a different database
curl -X POST -H "Content-Type: application/json" \
  -d '{"dbPath":"/path/to/database"}' \
  http://localhost:3000/api/open

# Get paginated results
curl "http://localhost:3000/api/table/products?limit=25&offset=50&columns=name,price,category"

Response Format

{
  "data": [
    {
      "id": 1,
      "name": "Example Record",
      "vector": [0.1, 0.2, 0.3]
    }
  ],
  "count": 150,
  "limit": 100,
  "offset": 0,
  "schema": {
    "fields": [
      {
        "name": "id",
        "type": "Int64",
        "nullable": false
      }
    ]
  }
}

🛠️ Development

Prerequisites

• Node.js >= 16.0.0
• npm >= 7.0.0

Setup

# Clone the repository
git clone https://github.com/littlellmjs/lanceviewer.git
cd lanceviewer

# Install dependencies
npm install

# Run tests
npm test

# Start development server
npm run dev

# Run linting
npm run lint

Testing

# Run all tests
npm test

# Run integration tests only
npm run test:integration

# Run with coverage (if istanbul is installed)
npm run test:coverage

Building

# Lint code
npm run lint

# Fix linting issues automatically
npm run lint:fix

# Run security audit
npm run security

🔒 Security

LanceDB Viewer implements multiple security measures:

Input Validation

• Path traversal protection
• SQL injection prevention
• Request size limits
• Input sanitization

Rate Limiting

• Configurable concurrent request limits
• Client-based rate limiting
• Automatic request throttling

Security Headers

• Content Security Policy (CSP)
• XSS protection
• Frame options
• Content type validation

Data Protection

• No sensitive data logging
• Secure error messages
• Environment variable validation

🚀 Deployment

Docker

FROM node:18-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --only=production
COPY . .
EXPOSE 3000
CMD ["npm", "start"]

Systemd Service

[Unit]
Description=LanceDB Viewer
After=network.target

[Service]
Type=simple
User=lancedb
Environment=NODE_ENV=production
Environment=PORT=3000
ExecStart=/usr/bin/node /path/to/lanceviewer/server.js
Restart=always

[Install]
WantedBy=multi-user.target

Production Checklist

• [ ] Set NODE_ENV=production
• [ ] Configure proper logging
• [ ] Set up monitoring
• [ ] Configure rate limiting
• [ ] Enable HTTPS in production
• [ ] Set appropriate file permissions
• [ ] Configure firewall rules
• [ ] Set up log rotation

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Workflow

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and add tests
4. Run the test suite: npm test
5. Ensure code style: npm run lint
6. Commit your changes: git commit -m 'Add amazing feature'
7. Push to the branch: git push origin feature/amazing-feature
8. Open a Pull Request

Code Style

This project uses ESLint for code quality. Please run npm run lint before submitting PRs.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• LanceDB - The powerful vector database
• Apache Arrow - High-performance columnar data processing
• Node.js - The JavaScript runtime

📞 Support

• 📧 Email: [email protected]
• 🐛 Issues: GitHub Issues
• 📖 Documentation: GitHub Wiki

Made with ❤️ for the data science community

If you find this project helpful, please give it a ⭐️ on GitHub!

🌟 Star on GitHub • 📖 Documentation • 🐛 Report Issues