TinyKit - TypeSafe Functional Query Client for Tinybird

A TypeScript library that provides a functional, composable query API for building typesafe SQL queries, pipes, and data ingestion for Tinybird.

Features

• Functional Query Building: Compose queries using a chainable, functional API
• Full Type Safety: End-to-end type safety from schema definition to query results
• Schema Definition: Define your data sources with typed schemas and column definitions
• Pipe Creation: Build reusable query pipes with parameter validation
• Data Ingestion: Multiple ingestion strategies with validation and error handling
• SQL Generation: Automatically generate Tinybird-compatible SQL and .datasource files
• Parameter Validation: Runtime validation of query parameters using Zod
• CLI Tools: Command-line tools for code generation and datasource management

Prerequisites

Tinybird CLI for deployment

TinyKit's client and file generator run without a local Tinybird installation. Install Tinybird's CLI only when you want TinyKit to proxy deployment or workspace commands.

Install Tinybird CLI

# macOS/Linux
curl -sSL https://install.tinybird.co | sh

# Or via npm
npm install -g @tinybirdco/cli

# Verify installation
tb --version

Authentication Setup

# Login to your Tinybird workspace
tb auth

# Or set token directly
export TINYBIRD_TOKEN=your_token_here
tb auth --token $TINYBIRD_TOKEN

Local Development Workflow

1. Initialize Tinybird project (if not already done):

tb init

2. Deploy datasources generated by TinyKit:

tb push datasources/*.datasource

3. Deploy pipes created with TinyKit:

tb push pipes/*.pipe

4. Test locally with Tinybird's local development server:

tb server start
# Your TinyKit queries will now work against local Tinybird instance

Documentation

• Usage Guide - A guide to the main workflows for using TinyKit.
• API Reference - A detailed reference for the TinyKit API.

Installation

bun add @vyr-e/tinykit

Quick Start

For a detailed guide on how to get started, see the Usage Guide.

1. Define Your Schema and DataSource

import { defineSchema, defineDataSource, string, int64 } from '@vyr-e/tinykit';

const eventsSchema = defineSchema({
  id: string('id'),
  userId: string('userId'),
  event: string('event'),
  timestamp: int64('timestamp'),
  properties: string('properties'),
});

const eventsDataSource = defineDataSource({
  name: 'events__v1',
  schema: eventsSchema,
  engine: 'MergeTree',
  sortingKey: ['timestamp', 'userId'],
});

2. Build Pipes with Parameters

TinyKit offers two approaches for building pipes: the type-safe query builder and raw SQL for complex cases.

Query Builder Approach

import { definePipe, defineParameters, stringParam, int64Param, query, count, param } from '@vyr-e/tinykit';

const getEventsByUser = definePipe({
  name: 'get_events_by_user__v1',
  schema: eventsSchema,
  parameters: defineParameters({
    userId: stringParam('userId', { required: true }),
    limit: int64Param('limit', { default: 100 }),
  }),
}).endpoint((q, params) =>
  query(eventsSchema)
    .select('id', 'userId', 'event', 'timestamp')
    .from('events__v1')
    .where(`userId = ${param('userId', 'String', true)}`)
    .orderBy('timestamp DESC')
    .limit(params.limit)
);

Raw SQL Approach

For complex queries where the query builder becomes limiting, use .raw:

const complexAnalyticsPipe = definePipe({
  name: 'complex_analytics__v1',
  schema: eventsSchema,
  parameters: defineParameters({
    tenantId: stringParam('tenantId', { required: true }),
    startDate: stringParam('startDate', { required: true }),
    cohortSize: int64Param('cohortSize', { default: 1000 }),
  }),
}).raw(`
  WITH user_cohorts AS (
    SELECT 
      userId,
      toStartOfWeek(fromUnixTimestamp64Milli(timestamp)) as cohort_week,
      COUNT(*) as event_count
    FROM events__v1 
    WHERE tenantId = {{ String(tenantId, required=True) }}
      AND timestamp >= toUnixTimestamp64Milli('{{ String(startDate, required=True) }}')
    GROUP BY userId, cohort_week
    HAVING event_count >= {{ Int64(cohortSize, 1000) }}
  )
  SELECT 
    cohort_week,
    COUNT(DISTINCT userId) as active_users,
    AVG(event_count) as avg_events_per_user
  FROM user_cohorts
  GROUP BY cohort_week
  ORDER BY cohort_week
`);

Raw SQL literals carry a conservatively inferred output row. Direct source columns use the declared schema, common ClickHouse aggregates and casts use their known scalar types, and expressions TinyKit cannot prove become unknown.

3. Setup Client and Execute Queries

import { Tinybird } from '@vyr-e/tinykit';
import { z } from 'zod';

const tb = new Tinybird({
  token: process.env.TINYBIRD_TOKEN,
  datasources: { events: eventsDataSource },
  pipes: { getEventsByUser, complexAnalyticsPipe },
});

const getUserEvents = tb.pipe({
  pipe: 'get_events_by_user__v1',
  data: z.object({
    id: z.string(),
    userId: z.string(),
    event: z.string(),
    timestamp: z.number(),
  }),
});

const result = await getUserEvents({ userId: 'user-123' });
console.log(result.data); // Array of user events

// Registered raw pipes can use their inferred output without repeating a schema.
const getComplexAnalytics = tb.pipe({
  pipe: 'complex_analytics__v1',
});

const analytics = await getComplexAnalytics({
  tenantId: 'tenant-123',
  startDate: '2025-01-01',
});

analytics.data[0]?.active_users; // number

SQL inference is static only. Pass data: z.object(...) when runtime response validation is required; the explicit Zod schema overrides the inferred row.

4. Data Ingestion

import { defineIngest } from '@vyr-e/tinykit';

const eventsIngest = defineIngest({
  datasource: 'events__v1',
  schema: eventsSchema,
});

const ingest = tb.ingest(eventsIngest);
await ingest([
  {
    id: 'evt-1',
    userId: 'user-456',
    event: 'page_view',
    timestamp: Date.now(),
    properties: JSON.stringify({ page: '/home' }),
  },
]);

Core APIs

Schema Column Types

TinyKit supports all major ClickHouse/Tinybird column types:

import { string, int32, int64, float64, boolean, dateTime, date, uuid, array, map, tuple, nested, lowCardinality, nullable, json, ipv4, ipv6 } from '@vyr-e/tinykit';

const schema = defineSchema({
  id: string('id'),
  count: int64('count'),
  revenue: float64('revenue'),
  active: boolean('active'),
  createdAt: dateTime('createdAt'),
  tags: array('tags', z.string(), { innerType: 'String' }),
  metadata: json('metadata'),
  status: lowCardinality('status', z.string(), { innerType: 'String' }),
  optionalField: nullable('optionalField', z.string(), { innerType: 'String' }),
});

Query Functions

Build complex queries with type-safe functions:

import { count, sum, avg, min, max, timeGranularity, fromUnixTimestamp64Milli, conditional, rowNumber, lag, firstValue } from '@vyr-e/tinykit';

const analyticsQuery = query(schema)
  .selectRaw(`
    ${timeGranularity(fromUnixTimestamp64Milli('timestamp'), '1h')} as hour,
    ${count()} as event_count,
    ${sum('revenue')} as total_revenue,
    ${avg('revenue')} as avg_revenue
  `)
  .from('events__v1')
  .groupBy('hour')
  .orderBy('hour');

Parameter Types

Define typed parameters for your pipes:

import { stringParam, int64Param, float64Param, dateTimeParam, booleanParam, enumParam } from '@vyr-e/tinykit';

const params = defineParameters({
  userId: stringParam('userId', { required: true }),
  limit: int64Param('limit', { default: 100 }),
  minRevenue: float64Param('minRevenue', { default: 0.0 }),
  active: booleanParam('active', { default: true }),
  period: enumParam('period', ['1h', '1d', '1w'] as const, { default: '1d' }),
});

TinyKit CLI Commands

TinyKit provides a comprehensive CLI for project management and Tinybird integration.

Project Initialization

# Initialize a new TinyKit project with interactive setup
tinykit init --dir ./my-project

Code Generation

# Generate Tinybird files from TypeScript definitions
tinykit generate

# Watch for changes and regenerate automatically
tinykit generate --watch

# Dry run to see what would be generated
tinykit generate --dry-run

# Generate from one or more explicit TypeScript entry files
tinykit generate --file src/schema.ts --file src/pipes.ts

Deployment Commands

# Deploy all resources to Tinybird
tinykit deploy

# Deploy specific resources
tinykit push datasources/events.datasource
tinykit push pipes/analytics.pipe

# Validate a deployment without applying it
tinykit deploy --check

# Explicitly allow destructive schema changes
tinykit deploy --allow-destructive-operations

# Pull resources from Tinybird
tinykit pull

Local Development

# Start local Tinybird development server
tinykit local:start

# Stop local development server
tinykit local:stop

# Check local server status
tinykit local:status

# Start development mode with auto-reload
tinykit dev

Data Source Management

# List all datasources
tinykit datasource ls --format json

# Inspect specific datasource
tinykit datasource get events__v1

# Analyze data file for schema generation
tinykit datasource analyze --file data.csv
tinykit datasource analyze --url https://example.com/data.json

# Generate datasource from file
tinykit datasource generate --file data.csv --name events

Dependency Analysis

# Show resource dependencies
tinykit dependencies --format json

# Show dependencies for specific pipe
tinykit dependencies --pipe analytics__v1

# Show resources with no dependencies
tinykit dependencies --no-deps

Examples and Project Structure

Generated Project Structure

When you initialize a TinyKit project, you get a well-organized structure:

my-tinybird-project/
├── src/                    # TypeScript source files
│   ├── events.ts          # Event tracking schema & pipes
│   ├── analytics.ts       # Analytics pipes
│   └── ingestion.ts       # Data ingestion patterns
├── tinybird/              # Generated Tinybird files
│   ├── datasources/       # .datasource files
│   └── pipes/             # .pipe files
├── package.json
├── tsconfig.json
└── README.md

Working Examples

See the complete working example in the repository:

examples/events.ts - Complete event tracking system with:

• Schema definition with JSON path mapping
• Data source configuration with proper indexing
• Event activity aggregation pipes (both query builder and raw SQL approaches)
• Streaming ingestion setup
• Client configuration and usage patterns

To run the example:

# Clone the repository
git clone https://github.com/Vyr-e/tinykit.git
cd tinykit

# Install dependencies
bun install

# Make sure Tinybird is running locally
tinykit local start

# Generate and deploy the example
bun run examples/events.ts

The example demonstrates:

• Type-safe schema definitions
• Parameter validation and templating
• Time-based aggregations with proper indexing
• Both functional query building and raw SQL escape hatches
• Real-time data ingestion patterns
• Client setup and typed query execution

Development

Development

Prerequisites

• Node.js 20+ or Bun
• Tinybird CLI installed and authenticated
• Active Tinybird workspace

Setup

1. Initialize a new TinyKit project:

tinykit init my-analytics-project
cd my-analytics-project

2. Install dependencies:

bun install  # or npm/yarn/pnpm

3. Start development:

# Generate Tinybird files and watch for changes
bun run generate:watch

# In another terminal, start local Tinybird server
tinykit local start

# Deploy generated files
tinykit deploy

Development Workflow

1. Define schemas and pipes in TypeScript files under src/
2. Generate Tinybird files with tinykit generate or --watch mode
3. Deploy to local/remote using tinykit deploy or tinykit local start
4. Test your pipes using the Tinybird API or web interface
5. Iterate - TinyKit watches for changes and regenerates automatically

Testing

Tests require a running Tinybird instance:

# Start local Tinybird server
tinykit local start

# Run tests in another terminal
bun test

# Or run tests against remote workspace
TINYBIRD_TOKEN=your_token bun test

Building

bun run build

Configuration

TinyKit looks for configuration in multiple places:

• Command line arguments (--token, --config)
• Environment variables (TINYBIRD_TOKEN)
• Config files (.tinykitrc, .tinybird, ~/.tinykitrc)

Example .tinykitrc:

{
  "token": "your_tinybird_token",
  "outputDir": "./tinybird"
}

Local Development Tips

1. Use watch mode (tinykit generate --watch) for faster iteration
2. Start with local server (tinykit local start) before deploying to cloud
3. Use the raw SQL escape hatch for complex queries the builder can't express
4. Version your schemas and pipes to enable safe migrations
5. Validate generated SQL with tinybird validate before deploying
6. Keep config files in version control for team consistency

Advanced Usage

Custom Output Directory

tinykit generate --dir ./custom-tinybird-output

Environment-Specific Deployments

# Use different tokens for different environments
TINYBIRD_TOKEN=$STAGING_TOKEN tinykit deploy
TINYBIRD_TOKEN=$PROD_TOKEN tinykit deploy --allow-destructive-operations

Batch Operations

# Deploy only datasources
tinykit push tinybird/datasources/*.datasource

# Deploy only pipes
tinykit push tinybird/pipes/*.pipe