df-script
v2.3.0
Published
A zero-dependency, high-performance, expression-based DataFrame library for TypeScript/JavaScript.
Maintainers
Keywords
Readme
🚀 df-script: High-Performance TypeScript DataFrame Library
df-script is a blazing-fast, zero-dependency, expression-based DataFrame and data manipulation library for TypeScript and JavaScript.
Heavily inspired by modern columnar engines like Polars and Pandas, df-script brings declarative, high-performance data transformations directly to JavaScript runtimes (Node.js, Browser, Bun, Deno, and Edge Workers). Easily execute complex multi-column groupings, relational joins, windowed rolling statistics, string parsing, temporal conversions, nested structs, and hundreds more with a fluent, expressive API and zero external dependencies.
📦 Installation
npm install df-scriptOr with Yarn, PNPM, or Bun:
yarn add df-script
pnpm add df-script
bun add df-scriptImport in TypeScript or JavaScript:
// ES Module / TypeScript
import { $df, DataFrame, ColumnExpr, DataType } from "df-script";// CommonJS / Node.js
const { $df, DataFrame, ColumnExpr, DataType } = require("df-script");🚀 Quick Start
1. Basic DataFrame & Expression Pipeline
import { $df } from "df-script";
// Create a DataFrame with structured data and automatic schema inference
const df = $df.data([
{ id: 1, name: "Alice", joinDate: "2026-01-15", sales: 1200.50, tags: ["sales", "east"] },
{ id: 2, name: "Bob", joinDate: "2026-02-20", sales: 850.00, tags: ["support", "west"] },
{ id: 3, name: "Charlie", joinDate: "2026-03-05", sales: 2300.00, tags: ["sales", "north"] },
{ id: 4, name: "David", joinDate: "2026-03-12", sales: null, tags: ["marketing"] },
]);
// Select columns, transform strings, format dates, and compute expressions
const processedDf = df.select(
$df.col("id"),
$df.col("name").str.upper().alias("NAME_UPPER"),
$df.col("joinDate").str.toDatetime().dt.year().alias("joinYear"),
$df.col("sales").add(500).alias("salesAdjusted"),
$df.col("tags").arr.lengths().alias("tagCount")
);
console.log(processedDf.toDicts());
/* Output:
[
{ id: 1, NAME_UPPER: 'ALICE', joinYear: 2026, salesAdjusted: 1700.5, tagCount: 2 },
{ id: 2, NAME_UPPER: 'BOB', joinYear: 2026, salesAdjusted: 1350, tagCount: 2 },
{ id: 3, NAME_UPPER: 'CHARLIE', joinYear: 2026, salesAdjusted: 2800, tagCount: 2 },
{ id: 4, NAME_UPPER: 'DAVID', joinYear: 2026, salesAdjusted: null, tagCount: 1 }
]
*/2. GroupBy & Multi-Metric Aggregations
const transactions = $df.data([
{ region: "East", category: "Hardware", revenue: 450, quantity: 3, timestamp: "2026-02-01" },
{ region: "East", category: "Hardware", revenue: 550, quantity: 5, timestamp: "2026-02-10" },
{ region: "West", category: "Software", revenue: 1200, quantity: 1, timestamp: "2026-01-20" },
]);
const salesSummary = transactions
.groupBy(["region", "category"])
.agg([
$df.col("revenue").sum().alias("total_revenue"),
$df.col("revenue").mean().round(2).alias("avg_revenue"),
$df.col("quantity").sum().alias("units_sold"),
$df.col("timestamp").str.toDatetime().dt.year().alias("year")
])
.sort("total_revenue", { descending: true });
console.log(salesSummary.toDicts());
/* Output:
[
{ region: 'East', category: 'Hardware', total_revenue: 1000, avg_revenue: 500, units_sold: 8, year: 2026 },
{ region: 'West', category: 'Software', total_revenue: 1200, avg_revenue: 1200, units_sold: 1, year: 2026 }
]
*/3. Relational Joins & Conditional Columns
const orders = $df.data([
{ orderId: 101, customerId: 1, amount: 250 },
{ orderId: 102, customerId: 2, amount: 1400 },
]);
const customers = $df.data([
{ id: 1, firstName: "Alice", lastName: "Smith" },
{ id: 2, firstName: "Bob", lastName: "Jones" },
]);
// Join tables with complete null safety and compute new columns
const enrichedOrders = orders.join(customers, {
leftOn: "customerId",
rightOn: "id",
how: "left"
}).withColumns([
$df.col("firstName").str.concat(" ").str.concat($df.col("lastName")).alias("fullName"),
$df.when($df.col("amount").gt(1000)).then("VIP").otherwise("Standard").alias("tier")
]);
console.log(enrichedOrders.toDicts());
/* Output:
[
{ orderId: 101, customerId: 1, amount: 250, id: 1, firstName: 'Alice', lastName: 'Smith', fullName: 'Alice Smith', tier: 'Standard' },
{ orderId: 102, customerId: 2, amount: 1400, id: 2, firstName: 'Bob', lastName: 'Jones', fullName: 'Bob Jones', tier: 'VIP' }
]
*/4. Rolling Moving Windows & Cumulative Calculations
const sensorData = $df.data([
{ timestamp: "2026-03-01T00:00:00Z", temperature: 20.1, powerUsage: 12 },
{ timestamp: "2026-03-01T01:00:00Z", temperature: 20.9, powerUsage: 14 },
{ timestamp: "2026-03-01T02:00:00Z", temperature: 21.5, powerUsage: 13 },
]);
// Compute moving averages and cumulative metrics without manual loop state
const telemetryWithTrends = sensorData
.sort("timestamp")
.withColumns([
$df.col("temperature").rollingMean(2).alias("temp_rolling_avg"),
$df.col("powerUsage").cumSum().alias("cumulative_kwh"),
$df.col("temperature").diff().alias("temp_delta")
]);
console.log(telemetryWithTrends.toDicts());
/* Output:
[
{ timestamp: '2026-03-01T00:00:00Z', temperature: 20.1, powerUsage: 12, temp_rolling_avg: 20.1, cumulative_kwh: 12, temp_delta: null },
{ timestamp: '2026-03-01T01:00:00Z', temperature: 20.9, powerUsage: 14, temp_rolling_avg: 20.5, cumulative_kwh: 26, temp_delta: 0.8 },
{ timestamp: '2026-03-01T02:00:00Z', temperature: 21.5, powerUsage: 13, temp_rolling_avg: 21.2, cumulative_kwh: 39, temp_delta: 0.6 }
]
*/✨ Key Features
- 📦 Zero Dependencies — 0 external runtime dependencies; completely standalone (<45 KB gzipped).
- ⚡ Columnar Execution — Flat memory layouts and contiguous loops that eliminate intermediate array allocations.
- 🔗 Expression-Based API — Compose complex transformations, aggregations, and conditions using fluent
$dfexpressions. - 📂 Strict Namespaces:
.str— Unicode string manipulations, regex extractions, and JSON path lookups..dt— Timezone conversions, business days, microsecond-precision datetimes, and durations..arr— Array/list column operations and element-wise.arr.eval()mapping..struct— Nested object handling and.struct.unnest()column flattening.
- 🛡️ Defensive & Type-Safe — Automatic type coercion, Kleene three-valued logic for null safety, and strict schema validation.
📌 Compatibility & Runtime Specifications
| Attribute | Specification |
| :--- | :--- |
| Supported Languages | TypeScript (.ts, .tsx), JavaScript (.js, .jsx), ESM (.mjs), CommonJS (.cjs) |
| Target Standard | Standard ES2020+ compatible |
| Target Platforms | Node.js (14+), Bun, Deno, Modern Web Browsers (Chrome 80+, Safari 13.1+, Firefox 74+, Edge 80+), Cloudflare Workers, Fastly Compute, AWS Lambda |
| UI Frameworks | React, Next.js, Vue, Nuxt, Svelte, SolidJS, Astro, Vite, Webpack, esbuild |
| Operational Scope | In-memory columnar DataFrame & vectorized expression engine with zero intermediate allocations |
| I/O & Formats | High-throughput RFC-compliant CSV parsing & streaming, JSON / JSONLines, and row/column records |
| Runtime Footprint | Pure TypeScript/JavaScript (0 external dependencies, 0 native C++/Rust bindings, 0 WebAssembly overhead) |
| Bundle Size | ~44 KB gzipped (~127 KB minified) for the complete engine; ~21 KB gzipped for standalone utils |
| Distribution | Dual ESM (dist/index.mjs) & CommonJS (dist/index.js) with comprehensive .d.ts declaration maps |
| Package Managers | npm, pnpm, bun, yarn |
💡 Why df-script? (Polars & Pandas for TypeScript)
For the vast majority of web applications, transforming data in a meaningful way still imposes a heavy architectural tax: spinning up a dedicated backend service (commonly Python with Pandas or Polars).
Suddenly, a straightforward transformation requirement introduces container pipelines, deployment overhead, network latency, serialization bottlenecks, UI loading spinners, and ongoing cloud hosting bills, all just to run operations modern client CPUs can crunch in single-digit milliseconds.
When developers attempt to eliminate the backend and transform data directly in JavaScript or TypeScript, they hit another roadblock: hand-rolling bespoke data processing helpers. What begins as a seemingly innocent 10-line utility for grouping, joining, or rolling metrics quickly balloons into hundreds of lines of fragile, bug-ridden boilerplate. Real-world client data immediately exposes an overwhelming mountain of edge cases:
- Composite Keys: Hashing multi-column keys without collisions, distinguishing string
"null"from literalnullandundefined, maintaining deterministic order, and handling sparse indices. - Relational Joins: Preserving outer join semantics without silent truncation, resolving column name collisions with automatic suffixing, and aligning time-series with as-of nearest matches.
- Windowing & Rolling Metrics: Managing sliding window boundaries, min-sample thresholds, cumulative accumulators, and lead/lag offsets without unbounded memory growth.
- Numeric Gotchas: Eliminating silent precision loss from floating-point drift,
NaNcontagion,-0vs+0, integer overflows, andBigIntcross-type operations. - Date & String Edge Cases: Handling timezone offsets across client environments, ISO/RFC parsing variations, regex escaping, and Unicode surrogate pairs.
Attempting to solve these edge cases using naive chains of .map(), .filter(), .flatMap(), and .reduce() compounds the issue: each chained operation performs another full pass over the dataset and floods the V8 garbage collector with millions of short-lived intermediate objects, triggering noticeable UI freezes, frame drops, and memory leaks.
Groupings, relational joins, and rolling windows are only a fraction of the data transformations modern applications need. Hand-rolling solutions for all of them is an unsustainable engineering sinkhole.
df-script changes this equation entirely. It introduces a zero-dependency, column-oriented DataFrame engine constructed specifically to bring native transformation power directly to the browser and edge:
- 🎯 Target Sweet Spot (1,000 to 500,000+ rows): Purpose-built for operational client datasets, real-time dashboards, local CSV exploration, and IoT telemetry streams directly in-memory or inside a Web Worker.
- 🚀 Eliminate the Python Backend Tax: Execute complex multi-column groupings, rolling statistics, relational joins, time-series matching, and pivots locally and synchronously right where the user interacts with the data. Zero API round-trips, zero server downtime, and zero backend maintenance.
- 🛠️ Never Hand-Roll
groupBy,join, or Windowing Again: Complex grouping logic, Kleene three-valued null safety, and edge cases are already solved and verified across thousands of unit tests. - 🪶 Lightweight Footprint (<45 KB gzipped): Zero external dependencies (~127 KB minified) with no WebAssembly startup latency, no asynchronous compilation steps, and no native C++ bindings.
- ⚡ Compiled Columnar Performance: Compiles declarative
$df.col(...)expression trees down to contiguous, element-wise loops with cached lengths and minimal memory allocations, keeping your UI responsive at 60 FPS. - 🛡️ TypeScript-First & Universal: Clean, Polars-inspired domain namespaces (
.str,.dt,.arr,.struct) with automatic schema deduction, full compile-time type safety, and standard ES2020+ portability across browsers, Web Workers, Node.js, Bun, and Deno.
📖 Core Concepts
The $df Entry Point
df-script uses the $df namespace to instantiate DataFrames, reference columns, construct expressions, and specify data types.
$df.data(dataRowsOrCols, schema?): Instantiates a newDataFrame.$df.readJson(content, options?): Reads JSON/NDJSON content into a newDataFrame.$df.readCsv(content, options?): Reads CSV content into a newDataFramewith automatic schema inference.$df.col(selector): Creates a column reference expression by column name ("a"), multiple names (["a", "b"]), RegExp pattern (/^user_/), or DataType selector ($df.Float64,$df.Numeric).$df.all(): Selects all columns in the DataFrame.$df.coalesce(...exprs): Returns the first non-null value among columns or expressions.$df.concat(items, options?): Concatenates multiple DataFrames along vertical, horizontal, or diagonal axes.$df.duration(optionsOrString): Constructs aDurationexpression from component options ({ days: 1, hours: 12 }) or compound duration strings ("1d 12h 30m").$df.element(): References the current array element within an.arr.eval(...)expression.$df.exclude(columns): Matches all columns except the specified ones.$df.horizontal(...exprs): Combines multiple columns/expressions horizontally per row for row-wise evaluation (.sum(),.arr,.eval()).$df.implode(column): Aggregates a column's rows or grouped values into a list.$df.lit(val): Explicitly wraps a raw value into a literal expression.$df.seqRange(value, options?): Generates a sequence range of values.$df.struct(fields): Constructs a nested struct object expression from named expressions or sibling columns.$df.when(predicate).then(value)...otherwise(value): Constructs a conditionalCASE WHENexpression chain.$df.Float64,$df.Int32,$df.Utf8, etc.: Direct access to data types and constructors for schema definitions and type-based column selection.
🛠️ DataFrame API Reference
All methods and getters on DataFrame in alphabetical order:
| Method / Property | Description |
| :--- | :--- |
| cast(schema, options?) | Casts DataFrame columns to target data types using a schema mapping. |
| clone() | Deep-copies the DataFrame, duplicating all underlying columnar memory buffers and schema. |
| columns | Getter returning an array of all column names in the DataFrame. |
| concat(items, options?) | Concatenates multiple DataFrames along "vertical", "horizontal", or "diagonal" axes. |
| drop(...names) | Drops one or more specified columns from the DataFrame. |
| dropNulls(subset?) | Drops rows containing null or undefined values across all or specified columns. |
| dtypes | Getter returning a dictionary of column names to their registered DataType instances. |
| equals(other, options?) | Deep equality comparison with tolerance support for float columns. |
| explode(columns) | Unnests list-like columns into multiple rows, replicating other columns per list element. |
| fill(target, options?) | Replaces targeted values ("null", "nan", "all", or expressions) across columns with strategies or values. |
| fillNan(options?) | Replaces NaN values across numeric columns with scalars, expressions, or statistical strategies. |
| fillNull(ruleOrOptions) | Fills null values with strategies ("forward", "backward", "mean", "min", "max", or literal values). |
| filter(...predicates) | Filters rows where all predicate expressions evaluate to true. |
| groupBy(keys) | Groups data by one or more columns, returning a GroupedData object for aggregations. |
| groupByDynamic(indexCol, options) | Dynamic time-series/numeric window grouping over tumbling, sliding, or rolling temporal intervals. |
| head(n?) | Returns the first n rows of the DataFrame (defaults to 5). |
| height | Getter returning the total number of rows in the DataFrame. |
| insertColumn(index, name, values) | Inserts a new column at a specific zero-based index. |
| item(row?, col?) | Returns an individual scalar cell value at the specified row and column coordinate. |
| iterColumns() | Generator iterating over { name, values } pairs for each column. |
| iterRows() | Generator iterating over row record objects { [column]: value }. |
| join(other, onOrOptions, how?, suffixes?) | Relational joins ("inner", "left", "right", "outer", "semi", "anti", "cross"). |
| joinAsof(other, options) | Inexact time-series / nearest-neighbor joins on sorted key columns. |
| joinWhere(other, predicate, options?) | Arbitrary non-equi joins evaluated over candidate cartesian pairs. |
| limit(n, options?) | Returns a subset of rows with optional offset and direction ("start" or "end"). |
| partitionBy(by, options?) | Splits DataFrame into sub-DataFrames (as array or dictionary) partitioned by column keys or expressions. |
| pivot(index, columns, values) | Reshapes tabular data from long to wide format based on unique column keys. |
| rename(mapping) | Renames columns using a { oldName: newName } dictionary. |
| reverse() | Reverses the row ordering of the DataFrame. |
| schema | Getter returning the current column-to-DataType schema specification. |
| select(...exprs) | Projects and evaluates column expressions, selectors, regex patterns, or struct unnesting. |
| shape | Getter returning the [height, width] dimensions tuple. |
| slice(start, end?) | Slices a continuous range of rows by start and end indices. |
| sort(options) | Multi-column sorting with descending, nullsLast, and custom comparator functions. |
| tail(n?) | Returns the last n rows of the DataFrame (defaults to 5). |
| toArray() | Serializes the DataFrame into a two-dimensional row array (any[][]). |
| toDict() | Serializes the DataFrame into a column-oriented dictionary { [col]: any[] }. |
| toDicts() | Serializes the DataFrame into an array of row objects RowRecord[]. |
| transpose(options?) | Swaps rows and columns into a transposed DataFrame. |
| unique(columns?) | Returns distinct rows, optionally deduplicating based on a subset of columns. |
| unpivot(idVars, valueVars, varName?, valName?) | Melts wide columns into long format name-value pairs. |
| unstack(step, options?) | Unstacks multi-level grouped columns. |
| width | Getter returning the total number of columns in the DataFrame. |
| withColumns(...exprs) | Adds or updates columns in-place using expression evaluations. |
| withRowIndex(name?, offset?) | Inserts a zero-based or offset row index column into the DataFrame. |
| writeCsv(file?, options?) | Serializes the DataFrame into RFC-4180 CSV string or writes to disk/stream. |
| writeJson(file?, options?) | Serializes the DataFrame into JSON or NDJSON string or writes to disk/stream. |
🧮 Column Expressions API Reference
All column expressions inherit from ColumnExpr / ExprBase and support fluent chaining.
1. Core & Math Expressions (Alphabetical)
| Method | Description |
| :--- | :--- |
| abs | Computes element-wise absolute value. |
| add | Element-wise addition with value or expression. |
| all | Returns true if all elements evaluate to true (Kleene logic). |
| allNull | Returns true if all elements in the expression are null or undefined. |
| and | Logical AND with Kleene three-valued null logic. |
| any | Returns true if at least one element evaluates to true. |
| anyNull | Returns true if any element is null or undefined. |
| arccos | Element-wise inverse cosine in radians. |
| arccosh | Element-wise inverse hyperbolic cosine. |
| arcsin | Element-wise inverse sine in radians. |
| arcsinh | Element-wise inverse hyperbolic sine. |
| arctan | Element-wise inverse tangent in radians. |
| arctan2 | Element-wise four-quadrant inverse tangent with second argument. |
| arctanh | Element-wise inverse hyperbolic tangent. |
| argMax | Returns the index of the first maximum value. |
| argMin | Returns the index of the first minimum value. |
| avg | Alias for mean(); computes the arithmetic mean. |
| between | Evaluates if elements are within an inclusive or exclusive numeric/date range. |
| bitwiseAnd | Bitwise AND (&) operation on integers. |
| bitwiseOr | Bitwise OR (|) operation on integers. |
| bitwiseXor | Bitwise XOR (^) operation on integers. |
| cbrt | Computes element-wise cube root. |
| ceil | Rounds numbers upward to the nearest integer. |
| clip | Clamps values between a lower and upper boundary. |
| copysign | Copies the sign of the magnitude argument to the target numbers. |
| corr | Computes Pearson correlation coefficient with another column or expression. |
| cos | Element-wise cosine in radians. |
| cosh | Element-wise hyperbolic cosine. |
| cot | Element-wise cotangent in radians. |
| count | Counts non-null elements (or all elements with { includeNulls: true }). |
| cov | Computes covariance with another column or expression. |
| cumCount | Cumulative count of elements along the column. |
| cumMax | Cumulative maximum value along the column. |
| cumMin | Cumulative minimum value along the column. |
| cumProd | Cumulative product of values along the column. |
| cumSum | Cumulative running sum along the column. |
| degrees | Converts angles from radians to degrees. |
| denseRank | Computes dense rank window order without rank gaps. |
| div | Element-wise division. |
| dot | Computes dot product vector multiplication with another expression. |
| entropy | Computes Shannon entropy (supports optional base and normalization). |
| eq | Strict equality (==) with Kleene null propagation. |
| eqMissing | Null-safe equality treating null and undefined as equivalent. |
| ewmKurt | Computes exponentially weighted moving kurtosis. |
| ewmMean | Computes exponentially weighted moving average/mean (supports alpha, span, com, halfLife, and by). |
| ewmSkew | Computes exponentially weighted moving skewness. |
| ewmStd | Computes exponentially weighted moving standard deviation. |
| ewmSum | Computes exponentially weighted moving sum. |
| ewmVar | Computes exponentially weighted moving variance. |
| exp | Computes natural exponential (e^x). |
| expm1 | Computes e^x - 1 with high precision for values near zero. |
| fill | Replaces targeted values ("null", "nan", "all", or expressions) with values or propagation strategies. |
| fillNan | Replaces NaN values with scalars, expressions, or statistical strategies. |
| fillNull | Fills null values with scalar values or forward/backward fill strategies. |
| filter | Filters expression elements by a boolean predicate mask. |
| first | Returns the first element of the expression or group. |
| floor | Rounds numbers downward to the nearest integer. |
| floordiv | Integer/floor division. |
| ge | Greater than or equal to (>=) comparison. |
| gt | Greater than (>) comparison. |
| hasNulls | Returns true if column contains at least one null or undefined value. |
| hypot | Computes hypotenuse sqrt(a^2 + b^2). |
| implode | Aggregates elements into a single list/array column. |
| isClose | Evaluates if numeric values are approximately equal within tolerances. |
| isDuplicated | Returns boolean mask where true marks duplicated values. |
| isFinite | Checks if number is finite (neither NaN, Infinity, nor -Infinity). |
| isIn | Checks if value is present in a set or array of values. |
| isInfinite | Checks if number is Infinity or -Infinity. |
| isNDistinct | Matches the N-th distinct value in order (supports negative index). |
| isNan | Checks if numeric value is NaN. |
| isNotNan | Checks if numeric value is not NaN. |
| isNotNull | Checks if value is neither null nor undefined. |
| isNull | Checks if value is null or undefined. |
| isUnique | Returns boolean mask where true marks unique (non-duplicated) values. |
| kurtosis | Computes fourth standardized moment (peakedness/kurtosis). |
| lag | Shifts values backward by offset within partition window. |
| last | Returns the last element of the expression or group. |
| le | Less than or equal to (<=) comparison. |
| lead | Shifts values forward by offset within partition window. |
| log | Computes logarithm with optional base (defaults to natural log). |
| log1p | Computes natural log of (1 + x) for values near zero. |
| lt | Less than (<) comparison. |
| max | Returns the maximum non-null value. |
| maxBy | Returns the value of this expression where another expression achieves maximum. |
| mean | Computes the arithmetic average of non-null numbers. |
| median | Computes the 50th percentile median value. |
| min | Returns the minimum non-null value. |
| minBy | Returns the value of this expression where another expression achieves minimum. |
| mod | Modulo division remainder. |
| mode | Computes the most frequently occurring value(s). |
| mul | Element-wise multiplication. |
| nUnique | Returns the count of distinct non-null values. |
| nanMax | Returns maximum treating NaN values according to IEEE 754 rules. |
| nanMin | Returns minimum treating NaN values according to IEEE 754 rules. |
| ne | Strict inequality (!=) with Kleene null propagation. |
| neMissing | Null-safe inequality. |
| negate | Negates numeric values (-x). |
| not | Logical NOT operator. |
| notIn | Checks if value is not present in a set or array. |
| nullCount | Returns the total count of null and undefined values. |
| or | Logical OR with Kleene three-valued null logic. |
| over | Evaluates the expression over an analytical partition window (grouping keys). |
| pow | Computes exponentiation (x^y). |
| product | Computes product of all non-null values. |
| quantile | Computes quantile value at probability p (0 <= p <= 1). |
| radians | Converts angles from degrees to radians. |
| rand | Generates uniform pseudo-random values between 0 and 1. |
| rank | Computes competition rank order (1, 2, 2, 4). |
| reverse | Reverses the element order of the column. |
| rolling | Evaluates custom function or expression over a sliding window. |
| rollingMax | Computes rolling maximum over a sliding window. |
| rollingMean | Computes rolling moving average over a sliding window. |
| rollingMedian | Computes rolling median over a sliding window. |
| rollingMin | Computes rolling minimum over a sliding window. |
| rollingQuantile | Computes rolling quantile over a sliding window. |
| rollingRank | Computes rolling rank over a sliding window. |
| rollingStd | Computes rolling standard deviation over a sliding window. |
| rollingSum | Computes rolling sum over a sliding window. |
| round | Rounds numbers to specified decimal places. |
| roundSigFigs | Rounds numbers to a specified number of significant figures. |
| rowNumber | Computes sequential 1-based row index within partition window. |
| shift | Shifts column values by a signed offset (filling empty cells with null). |
| sign | Extracts sign of number (-1, 0, 1). |
| sin | Element-wise sine in radians. |
| sinh | Element-wise hyperbolic sine. |
| skew | Computes third standardized moment (skewness asymmetry). |
| spearmanCorr | Computes Spearman rank correlation coefficient. |
| sqrt | Computes square root. |
| std | Computes sample standard deviation. |
| sub | Element-wise subtraction. |
| sum | Computes sum of all non-null numeric values. |
| tan | Element-wise tangent in radians. |
| tanh | Element-wise hyperbolic tangent. |
| trunc | Truncates fractional parts toward zero. |
| variance | Computes sample variance. |
| wAvg | Computes weighted average using weight expression. |
| xor | Logical exclusive OR (XOR). |
📂 Namespaces API Reference
Specific domain transforms are grouped under dedicated, clean namespaces with full IDE autocomplete:
🔤 String Namespace (.str)
| Method | Description |
| :--- | :--- |
| .str.concat | Concatenates strings across columns or literal arguments. |
| .str.contains | Checks if string contains substring or matches RegExp pattern. |
| .str.containsAny | Checks if string contains any substring from a list. |
| .str.countMatches | Counts non-overlapping occurrences of pattern or substring. |
| .str.decode | Decodes binary bytes/strings using specified encoding (hex, base64, etc.). |
| .str.decodeUriComponent | Decodes URI-encoded characters. |
| .str.encode | Encodes strings to binary formats (hex, base64, utf8). |
| .str.encodeUriComponent | Encodes string components for safe URI transmission. |
| .str.endsWith | Checks if string ends with specified suffix. |
| .str.escapeRegex | Escapes regex metacharacters in string for literal matching. |
| .str.explode | Splits string by delimiter and explodes into multiple rows. |
| .str.extract | Extracts first matching regex capture group (supports group index or name). |
| .str.extractAll | Extracts all regex match occurrences as an array of strings. |
| .str.extractGroups | Extracts all named and numbered capture groups as a struct record. |
| .str.extractMany | Extracts matches from multiple regex patterns with leftmost tie-breaking. |
| .str.find | Finds zero-based character index of first match (or null if not found). |
| .str.findMany | Finds character indices of multiple patterns simultaneously. |
| .str.head | Extracts the first n characters of the string. |
| .str.join | Joins string values across rows or groups using a delimiter. |
| .str.jsonDecode | Parses JSON string into typed struct/object or list column. |
| .str.jsonPathMatch | Extracts values matching a JSONPath query expression. |
| .str.len | Returns string character count. |
| .str.lenBytes | Returns UTF-8 encoded byte count of the string. |
| .str.lenChars | Returns Unicode code point count (surrogate-pair aware). |
| .str.lower | Converts ASCII/Unicode characters to lowercase. |
| .str.lpad | Pads string on the left up to specified width. |
| .str.normalize | Applies Unicode normalization form (NFC, NFD, NFKC, NFKD). |
| .str.padEnd | Alias for rpad(); pads string on the right. |
| .str.padStart | Alias for lpad(); pads string on the left. |
| .str.replace | Replaces first pattern match with replacement string or function. |
| .str.replaceAll | Replaces all pattern matches across the string. |
| .str.replaceMany | Replaces multiple patterns in a single pass with conflict resolution. |
| .str.reverse | Reverses characters in the string. |
| .str.rpad | Pads string on the right up to specified width. |
| .str.slice | Extracts a substring slice by offset and length. |
| .str.split | Splits string by delimiter into a list/array of substrings. |
| .str.startsWith | Checks if string starts with specified prefix. |
| .str.stripChars | Strips specified leading and trailing characters (defaults to whitespace). |
| .str.stripCharsEnd | Strips specified trailing characters from the end of the string. |
| .str.stripCharsStart | Strips specified leading characters from the start of the string. |
| .str.stripPrefix | Removes prefix from start of string if present. |
| .str.stripSuffix | Removes suffix from end of string if present. |
| .str.strptime | Parses string to Datetime or Date using format specifiers. |
| .str.tail | Extracts the last n characters of the string. |
| .str.toCamelCase | Converts string to camelCase. |
| .str.toDate | Parses ISO 8601 date string to Date. |
| .str.toDatetime | Parses ISO 8601 datetime string to Datetime. |
| .str.toDecimal | Parses numeric string to fixed-point Decimal(precision, scale). |
| .str.toInteger | Parses numeric string to integer (Int8..Int64, UInt8..UInt64). |
| .str.toKebabCase | Converts string to kebab-case. |
| .str.toLowerCase | Alias for lower(). |
| .str.toPascalCase | Converts string to PascalCase. |
| .str.toSnakeCase | Converts string to snake_case. |
| .str.toTime | Parses time string (HH:MM:SS) to Time. |
| .str.toTitleCase | Converts string to Title Case. |
| .str.toUpperCase | Alias for upper(). |
| .str.upper | Converts ASCII/Unicode characters to uppercase. |
| .str.zfill | Pads string on the left with zeros up to specified width. |
📅 Temporal Namespace (.dt)
| Method | Description |
| :--- | :--- |
| .dt.castTimeUnit | Casts duration/timestamp to target time unit precision. |
| .dt.century | Extracts 1-based Gregorian century. |
| .dt.convertTimeZone | Converts datetime to target IANA timezone. |
| .dt.date | Extracts Date component (zeroing time component). |
| .dt.day | Extracts 1-based day of month (1-31). |
| .dt.daysInMonth | Returns number of days in the current month (28-31). |
| .dt.epoch | Returns elapsed units since Unix Epoch (1970-01-01T00:00:00Z). |
| .dt.hour | Extracts hour of day (0-23). |
| .dt.isBusinessDay | Returns true if date falls on Monday through Friday. |
| .dt.isLeapYear | Returns true if calendar year is a leap year. |
| .dt.isoYear | Extracts ISO 8601 week-numbering year. |
| .dt.microsecond | Extracts microsecond component (0-999). |
| .dt.millennium | Extracts 1-based Gregorian millennium. |
| .dt.millisecond | Extracts millisecond component (0-999). |
| .dt.minute | Extracts minute of hour (0-59). |
| .dt.month | Extracts 1-based month of year (1-12). |
| .dt.monthEnd | Snaps date to the final calendar day of the current month. |
| .dt.monthStart | Snaps date to the first calendar day of the current month. |
| .dt.nanosecond | Extracts nanosecond component (0-999). |
| .dt.offsetDay | Adds or subtracts signed integer calendar days. |
| .dt.ordinalDay | Extracts 1-based day of year (1-366). |
| .dt.quarter | Extracts calendar quarter (1-4). |
| .dt.replace | Replaces specific datetime components (year, month, day, etc.). |
| .dt.second | Extracts second of minute (0-59). |
| .dt.strftime | Formats datetime as a string using strftime-style format specifiers. |
| .dt.time | Extracts Time component (microseconds since midnight). |
| .dt.timestamp | Returns Unix timestamp in specified time unit. |
| .dt.totalDays | Computes total fractional duration in days. |
| .dt.totalHours | Computes total fractional duration in hours. |
| .dt.totalMicroseconds | Computes total duration in integer/fractional microseconds. |
| .dt.totalMilliseconds | Computes total fractional duration in milliseconds. |
| .dt.totalMinutes | Computes total fractional duration in minutes. |
| .dt.totalNanoseconds | Computes total duration in integer/fractional nanoseconds. |
| .dt.totalSeconds | Computes total fractional duration in seconds. |
| .dt.utcOffset | Returns timezone UTC offset in minutes or milliseconds. |
| .dt.week | Extracts calendar week number. |
| .dt.weekday | Extracts day of week (1 = Monday, 7 = Sunday). |
| .dt.year | Extracts calendar year. |
📊 Array/List Namespace (.arr)
| Method | Description |
| :--- | :--- |
| .arr.agg | Aggregates elements of each list row using an aggregation expression. |
| .arr.all | Returns true if all elements in the list row are truthy (Kleene logic). |
| .arr.any | Returns true if at least one element in the list row is truthy. |
| .arr.argMax | Returns the index of the first maximum element in each list row. |
| .arr.argMin | Returns the index of the first minimum element in each list row. |
| .arr.contains | Checks if list row contains the specified literal item. |
| .arr.containsAll | Checks if list row contains all items in the candidate set. |
| .arr.containsAny | Checks if list row contains any item in the candidate set. |
| .arr.countMatches | Counts occurrences of the target item in each list row. |
| .arr.eval | Evaluates an expression element-wise over each list item using $df.element(). |
| .arr.explode | Explodes/unnests list rows into individual scalar rows. |
| .arr.filter | Filters list row elements using an element-wise predicate expression. |
| .arr.first | Extracts the first element of each list row (supports nullOnOob). |
| .arr.gather | Gathers elements at specified indices (supports negative indexing). |
| .arr.gatherEvery | Gathers elements at periodic interval steps with optional offset. |
| .arr.get | Extracts element at zero-based index (supports negative indexing). |
| .arr.join | Joins array elements into a string separated by delimiter. |
| .arr.last | Extracts the last element of each list row (supports nullOnOob). |
| .arr.len | Returns the element count of each list row. |
| .arr.lengths | Alias for len(); returns length of each list row. |
| .arr.max | Computes maximum element in each list row. |
| .arr.mean | Computes arithmetic mean of numeric elements in each list row. |
| .arr.median | Computes median value of numeric elements in each list row. |
| .arr.min | Computes minimum element in each list row. |
| .arr.mode | Computes most frequent element(s) in each list row. |
| .arr.nUnique | Returns count of distinct elements in each list row. |
| .arr.reverse | Reverses element ordering in each list row. |
| .arr.shift | Shifts list elements by offset, filling vacated slots with null. |
| .arr.slice | Slices a sub-array from offset with optional length. |
| .arr.sort | Sorts elements of each list row with optional descending flag. |
| .arr.splice | Deletes and/or inserts elements at an index in each list row. |
| .arr.std | Computes sample standard deviation of elements in each list row. |
| .arr.sum | Computes sum of numeric elements in each list row. |
| .arr.toStruct | Converts array elements into a struct object with indexed/custom field names. |
| .arr.unique | Deduplicates elements within each list row. |
| .arr.variance | Computes sample variance of elements in each list row. |
🗃️ Struct/Object Namespace (.struct)
| Method | Description |
| :--- | :--- |
| .struct.field | Extracts a specific nested field from the struct column. |
| .struct.renameFields | Renames nested struct fields using a dictionary mapping. |
| .struct.unnest | Unnests/flattens all struct fields into individual top-level columns in select(). |
| .struct.withFields | Adds or overrides fields within the struct column. |
🛡️ Typing and Schema Registry
import { $df } from "df-script";
const schema = {
id: $df.DataType.Int32,
price: $df.DataType.Decimal(10, 2),
active: $df.DataType.Boolean,
createdAt: $df.DataType.Datetime
};
const df = $df.data(rawData, schema);Supported Data Types
- Integers:
Int8,Int16,Int32,Int64,UInt8,UInt16,UInt32,UInt64 - Floats & Decimals:
Float32,Float64,Decimal(precision?, scale?) - General:
Boolean,Utf8(Strings),Binary,Null,Object - Temporal:
Date,Datetime,Time,Duration - Nested Structures:
List(Arrays),Struct(Objects)
🧑💻 Contributing & Development
# Run test suite
npm test
# Build production bundles
npm run build📄 License
df-script is open-source software licensed under the MIT License.
