@spark-connect-js/core
v0.3.0
Published
Pure TypeScript core: logical DataFrame API and Spark Connect plan builder — no runtime dependencies
Maintainers
Readme
@spark-connect-js/core
DataFrame API and logical plan builder for Spark Connect, in pure TypeScript with zero runtime dependencies.
[!NOTE] This project is in early development. APIs may change between releases.
npm install @spark-connect-js/coreimport { SparkSession, col, sum, desc, type Transport } from "@spark-connect-js/core";
const spark = new SparkSession(transport);
const df = spark
.table("events")
.filter(col("ts").gt(lit("2025-01-01")))
.groupBy("category")
.agg(sum("amount").alias("total"))
.sort(desc("total"));Provides DataFrame, Column, SparkSession, Catalog, WindowSpec, DataFrameWriter, GroupedData, and 248 built-in functions. Plans are serialized to Spark Connect protobuf but no I/O happens here. Bring your own Transport implementation, or use @spark-connect-js/node which wires everything together.
Documentation
See the spark-connect-js repository.
