@robotops/rosql

v0.5.5

Published

a month ago

The query language for ROS2 telemetry data

0High
0Medium
0Low

ros2 robotics telemetry query-language opentelemetry

ROSQL (pronounced "RAW-skul") is Robot Ops Structured Query Language — a SQL-like language and driver purpose-built for ROS2 telemetry data stored via OpenTelemetry. It lets robotics engineers query traces, logs, and metrics using familiar SQL-like syntax with first-class support for ROS2 concepts — nodes, actions, topics, and message causality.

Robot observability is hard. ROS2 systems generate a firehose of traces, logs, and sensor data across dozens of nodes, but general-purpose query languages have no awareness of topics, action graphs, or message causality. ROSQL closes that gap: write queries in the language of your robot, not your database.

Built in Rust and available as a library, CLI, gRPC server, and WASM package, ROSQL is created and used by Robot Ops, Inc. to power the Robot Ops observability platform.

Docs, cookbook, and live demo → rosql.org

ROSQL is a trademark of Robot Ops, Inc.

Architecture

  ROS2 System
       │
       │  OTel attributes (ros.node, ros.action.*, ros.topic, ParentSpanId)
       ▼
  Robot Ops Agent (robotops.com) or OTel Collector (community)
       │
       │  OTLP gRPC
       ▼
  Datastore (e.g. PostgreSQL + TimescaleDB, ClickHouse, or any SQL-compatible DB)
       │
       │  OTel standard schema
       ▼
  rosql (parse + compile + execute)
       │
       ▼
  Query results

Usage modes

  ┌─────────────────────────────────────────────────────┐
  │ Mode 1: Parse + Execute (library)                   │
  │   ROSQL text → parser → AST → ROSQLBackend → DB    │
  ├─────────────────────────────────────────────────────┤
  │ Mode 2: CLI + gRPC Server                           │
  │   rosql parse / compile / query / serve             │
  ├─────────────────────────────────────────────────────┤
  │ Mode 3: WASM (frontend editor)                      │
  │   parse() / validate() / get_completions()          │
  └─────────────────────────────────────────────────────┘

Quick start

What ROSQL looks like

Express in one sentence what would take a page of SQL: find every navigation failure that happened while the battery was critically low.

SELECT trace_id, span_name, service_name, duration, status_code, span_attributes
FROM traces
WHERE status = 'ERROR' AND action_name = '/navigate_to_pose'
DURING(
  FROM topics WHERE topic_name = '/battery_state'
  AND fields['percentage'] < 15
)
SINCE 6 hours ago

{
  "columns": ["trace_id", "span_name", "service_name", "duration", "status_code", "span_attributes"],
  "rows": [
    ["a3f1c9d2e8b04f7a", "navigate_to_pose", "bt_navigator", 1423187000, "ERROR",
      {"ros.node": "/bt_navigator", "ros.action.name": "/navigate_to_pose", "ros.action.status": "aborted"}],
    ["c7d2f1a3b9e54c2b", "navigate_to_pose", "bt_navigator",  873456000, "ERROR",
      {"ros.node": "/bt_navigator", "ros.action.name": "/navigate_to_pose", "ros.action.status": "aborted"}]
  ],
  "metadata": { "rows_returned": 2, "elapsed_ms": 14 }
}

duration is stored as nanoseconds (BIGINT). 1423187000 ns ≈ 1.42 s.

Or trace the full message causality chain from a single span — something SQL has no primitive for. TRACE 'id' walks parent_span_id → span_id recursively and returns all columns from otel_traces:

TRACE 'a3f1c9d2e8b04f7a'

{
  "columns": ["timestamp", "trace_id", "span_id", "parent_span_id", "span_name", "span_kind", "service_name", "duration", "status_code", "span_attributes", "resource_attributes"],
  "rows": [
    ["2025-03-25T14:32:11.012Z", "a3f1c9d2e8b04f7a", "f0e1d2c3b4a59687", "",                 "goal_received",     "INTERNAL", "bt_navigator",       0,         "OK",    {"ros.node": "/bt_navigator"},                                       {"host.name": "robot_03"}],
    ["2025-03-25T14:32:11.013Z", "a3f1c9d2e8b04f7a", "1a2b3c4d5e6f7089", "f0e1d2c3b4a59687", "compute_path",      "INTERNAL", "planner_server",    38000000,  "OK",    {"ros.node": "/planner_server",    "ros.topic": "/plan"},              {"host.name": "robot_03"}],
    ["2025-03-25T14:32:11.051Z", "a3f1c9d2e8b04f7a", "9876543210abcdef", "1a2b3c4d5e6f7089", "follow_path",       "INTERNAL", "controller_server", 412000000, "ERROR", {"ros.node": "/controller_server", "ros.topic": "/cmd_vel"},            {"host.name": "robot_03"}],
    ["2025-03-25T14:32:11.463Z", "a3f1c9d2e8b04f7a", "fedcba9876543210", "9876543210abcdef", "obstacle_detected", "INTERNAL", "local_costmap",       6000000, "OK",    {"ros.node": "/local_costmap",     "ros.topic": "/costmap_update"},    {"host.name": "robot_03"}]
  ],
  "metadata": { "rows_returned": 4, "elapsed_ms": 3 }
}

As a CLI

# Install (Linux x86_64 / arm64, macOS Intel & Apple Silicon)
curl -fsSL https://rosql.org/install.sh | sh
# Or on macOS via Homebrew (supports brew upgrade)
brew install robotopsinc/tap/rosql

# Or build from source (all platforms)
cargo install rosql --features server,duckdb

# Try the public demo dataset instantly — no setup, no credentials
rosql query "FROM traces WHERE status = 'ERROR' LIMIT 5" \
  --backend parquet --url s3://robotops-production-rosql-demo/data

# Query your own local Parquet telemetry files
rosql query "FROM traces WHERE status = 'ERROR' SINCE 1 hour ago" \
  --backend parquet --url ./telemetry/robotops_demo_agent/20260403-141530/

# Query Parquet files on a private S3 bucket
rosql query "FROM traces WHERE status = 'ERROR' SINCE 1 hour ago" \
  --backend parquet --url s3://my-bucket/robot-01/robotops_demo_agent/20260403-141530/

# Query a PostgreSQL database
rosql query "FROM traces WHERE status = 'ERROR' SINCE 1 hour ago" \
  --backend postgres --url postgresql://user:pass@localhost:5432/telemetry

# Compile to SQL (inspect what ROSQL generates — no DB or --url needed)
rosql compile "FROM traces WHERE duration > 500 ms" --backend parquet

# Trace a message causality chain
rosql query "TRACE 'a3f1c9d2e8b04f7a'" \
  --backend postgres --url postgresql://user:pass@localhost:5432/telemetry

As a library

cargo add rosql

use rosql::{parse, drivers::{SqlBackend, ExecOptions}};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let backend = SqlBackend::new("postgresql://user:pass@localhost/telemetry").await?;
    let query = parse("
        SELECT trace_id, span_name, service_name, duration, status_code
        FROM traces WHERE status = 'ERROR' AND action_name = '/navigate_to_pose'
        DURING(
          FROM topics WHERE topic_name = '/battery_state' AND fields['percentage'] < 15
        )
        SINCE 6 hours ago
    ")?;
    let result = backend.execute(&query, &ExecOptions::default()).await?;
    println!("{}", serde_json::to_string_pretty(&result)?);
    Ok(())
}

Driver support

| Backend | Feature flag | Status | |---------|-------------|--------| | PostgreSQL / TimescaleDB | postgres | | | MySQL / MariaDB | mysql | | | Parquet (local / S3) powered by DuckDB | duckdb | | | AWS Athena | athena | | | Google BigQuery | bigquery | |

Feature flags

| Feature | What it enables | Dependencies | |---------|----------------|--------------| | (default) | Parser, AST, unit system, SQL compiler, proto types | logos, serde, prost | | postgres | PostgreSQL / TimescaleDB driver | sqlx, tokio | | mysql | MySQL / MariaDB driver | sqlx, tokio | | duckdb | Parquet file backend (local + S3) powered by DuckDB — use with --backend parquet | duckdb (bundled), tokio | | server | rosql CLI binary + gRPC server | tonic, tokio, clap | | wasm | WASM exports for frontend editors | wasm-bindgen |

CLI

# Parse → JSON AST
rosql parse "FROM traces WHERE duration > 500 ms SINCE 1 hour ago"

# Compile → SQL (shows what SQL ROSQL generates — no DB or --url needed)
rosql compile "FROM traces WHERE duration > 500 ms" --backend parquet
rosql compile "FROM traces WHERE duration > 500 ms" --backend postgres

# Execute → query results as JSON (public demo dataset — no credentials needed)
rosql query "FROM traces WHERE status = 'ERROR' LIMIT 5" \
  --backend parquet --url s3://robotops-production-rosql-demo/data

# Execute → query results as JSON (Parquet backend — local or S3)
rosql query "FROM traces WHERE status = 'ERROR' SINCE 1 hour ago" \
  --backend parquet --url ./telemetry/robotops_demo_agent/20260403-141530/

rosql query "FROM traces WHERE status = 'ERROR' SINCE 1 hour ago" \
  --backend parquet --url s3://my-bucket/robot-01/robotops_demo_agent/20260403-141530/

# Execute → query results as JSON (PostgreSQL)
rosql query "FROM traces WHERE status = 'ERROR'" \
  --backend postgres --url postgresql://user:pass@localhost:5432/db

# Validate syntax
rosql validate "SELECT * FROM logs"

# Autocomplete suggestions at cursor position
rosql completions "FROM " 5

# Start gRPC server on Unix socket
rosql serve --socket /tmp/rosql.sock

# Schema profiles (match your OTel Collector exporter):
#   --schema otel-postgres    (lowercase columns, default)
#   --schema otel-clickhouse  (PascalCase columns)

Parquet backend: expected directory layout

The --backend parquet --url <path> option expects the following directory structure (matching the demo-agent output format):

<url>/
  traces/            *.parquet  →  otel_traces view
  logs/              *.parquet  →  otel_logs view
  metrics/           *.parquet  →  otel_metrics view
  topic_messages/    *.parquet  →  topic_messages view
  tf_states/         *.parquet  →  tf_states view        (optional — needed for FROM tf)
  mcap_metadata/     *.parquet  →  mcap_metadata view
  robot_joint_map/   *.parquet  →  robot_joint_map view  (optional — needed for SHOW JOINTS / JOINT DEVIATION)
  ros2_events/       *.parquet  →  ros2_events view      (optional — needed for SHOW DEPLOYMENTS)

Files are discovered recursively via **/*.parquet globs. Missing subdirectories are silently skipped — queries against absent tables return a DataSourceUnavailable error.

S3 credentials for the Parquet backend

When --url s3://... is used, ROSQL loads the DuckDB httpfs extension and reads credentials from standard AWS environment variables:

| Variable | Description | |---|---| | AWS_ACCESS_KEY_ID | Static access key | | AWS_SECRET_ACCESS_KEY | Static secret key | | AWS_REGION / AWS_DEFAULT_REGION | AWS region (e.g. us-east-1) | | AWS_PROFILE | Named credentials profile | | AWS_ENDPOINT_URL | Override endpoint for S3-compatible storage (MinIO, Cloudflare R2 etc.) |

Examples

-- Find slow navigation actions
SELECT span_name, duration
FROM traces
WHERE action_name = '/navigate_to_pose' AND duration > 500 ms
SINCE 1 hour ago
ORDER BY duration DESC
LIMIT 20

-- Cross-signal correlation: errors during low battery
FROM traces WHERE status = 'ERROR'
DURING(
  FROM topics WHERE topic_name = '/battery_state'
  AND fields['percentage'] < 20
)
SINCE yesterday

-- TF2 transform broadcasts: find robots with the arm raised above 1 m
FROM tf
WHERE parent_frame = 'base_link' AND child_frame = 'tool0'
AND translation_z > 1.0
SINCE 1 hour ago
FACET robot_id

-- Time-bucketed error rate dashboard (TIMESERIES)
SELECT COUNT(*) AS errors FROM traces WHERE status = 'ERROR'
TIMESERIES 5 min
SINCE 1 hour ago
FACET robot_id

-- Enrich failed navigation traces with log context (ENRICH WITH)
SELECT * FROM traces
WHERE status = 'ERROR' AND action_name = '/navigate_to_pose'
SINCE 1 hour ago
ENRICH WITH logs LIMIT 20

-- System topology: active topics and nodes
SHOW TOPICS FOR ROBOT 'robot_42' SINCE 30 minutes ago
SHOW NODES FOR ROBOT 'robot_42' SINCE 30 minutes ago
SHOW NODE GRAPH FOR ROBOT 'robot_42' SINCE 30 minutes ago

-- Trace span tree walk
TRACE 'abc123def456'

-- Pipeline syntax
FROM traces
| WHERE duration > 500 ms
| TIMESERIES 1 min
| FACET robot_id

See examples/ for a full walkthrough with Docker Compose, PostgreSQL fixture data, and runnable queries.

Cookbook: Investigating a failed navigation with enriched logs

When a navigation action fails, the recommended workflow is:

-- 1. Find the failing trace
SELECT trace_id, span_name, duration
FROM traces
WHERE status = 'ERROR' AND action_name = '/navigate_to_pose'
SINCE 30 min ago
ORDER BY duration DESC
LIMIT 5

-- 2. Walk the full causality chain for the worst offender
TRACE 'your-trace-id-here'

-- 3. Correlate with log output for that same window
SELECT * FROM traces
WHERE status = 'ERROR' AND action_name = '/navigate_to_pose'
SINCE 30 min ago
ENRICH WITH logs LIMIT 50

-- 4. Check which topics were active during the failure
SHOW TOPICS FOR ROBOT 'robot_42' SINCE 30 min ago

-- 5. See the node graph to spot missing pub/sub edges
SHOW NODE GRAPH FOR ROBOT 'robot_42' SINCE 30 min ago

WASM API

The @robotops/rosql npm package exposes parsing and validation for browser editors:

import init, { parse, validate, get_completions } from '@robotops/rosql';

await init();

const result = parse('FROM traces WHERE duration > 500 ms SINCE 1 hour ago');
console.log(result);

const errors = validate('INSERT INTO logs');
console.log(errors); // { valid: false, errors: [...] }

const completions = get_completions('FROM ', 5);
console.log(completions); // [{ label: 'logs', ... }, { label: 'traces', ... }, ...]

Schema

ROSQL expects telemetry data in the OpenTelemetry schema conventions for ROS2. This includes standard OTel tables (otel_traces, otel_logs, otel_metrics) with ROS2-specific span attributes (ros.node, ros.action.*, ros.topic).

Performance

See BENCHMARKS.md for performance data (coming soon — #35).

Installation

One-liner (Linux x86_64 / arm64, macOS Intel & Apple Silicon)

curl -fsSL https://rosql.org/install.sh | sh

Homebrew (macOS)

brew install robotopsinc/tap/rosql

Supports brew upgrade rosql and brew uninstall rosql.

Pre-built binaries include the Parquet backend (--backend parquet) and are available for:

| Platform | Architecture | Notes | |----------|-------------|-------| | Linux | x86_64 | — | | Linux | aarch64 | — | | macOS | arm64 (Apple Silicon) | — | | macOS | x86_64 (Intel) | — | | Windows | any | Build from source (see below) |

Build from source

cargo install rosql --features server,duckdb   # Parquet + CLI (recommended)
cargo install rosql --features server,postgres  # PostgreSQL + CLI
cargo install rosql --features server           # CLI only (compile/parse/validate)

Or clone and build locally:

git clone https://github.com/RobotOpsInc/rosql
cd rosql

# Library only (default)
cargo build --release

# CLI with Parquet backend (--backend parquet)
cargo build --release --features server,duckdb --bin rosql

# CLI with PostgreSQL
cargo build --release --features server,postgres --bin rosql

Local development

git clone https://github.com/RobotOpsInc/rosql
cd rosql
just build       # build default features
just test        # run tests
just build-wasm  # build WASM package
just check       # full CI: build + test + clippy + fmt + buf-lint

Prerequisites: Rust (stable, 1.80+), protoc, buf (optional). See CONTRIBUTING.md for details.

Contributing

ROSQL is in early development and contributions are welcome.

See CONTRIBUTING.md for development workflow, build variants, and release process
File bugs and feature requests in the issue tracker
Questions? Email [email protected]

Generative AI contributions policy

You may use generative AI tools to help prepare contributions, but you are responsible for all submitted work. Before opening a pull request, review AI-assisted changes for correctness, security, licensing, and consistency with this project's style and documentation. This is in alignment with the OSRF Generative AI Contributions Policy (May 2025).

Do not submit code, docs, data, or other content unless you have the right to contribute it under the Apache 2.0 license. If a meaningful part of your contribution was generated or substantially assisted by AI, disclose that in the pull request description.

Robot Ops platform

For fleet-scale telemetry with managed ingestion, storage, and dashboards — including lifecycle anchors, fleet-wide anomaly detection, and ClickHouse performance — see the Robot Ops platform.

License

Apache 2.0 — see LICENSE.