@lossless.org/client

One TypeScript client for lossless.org NoSQLDB, SQLDB and ObjectStorage, with MongoDB, MariaDB, ClickHouse and S3 protocol adapters. LosslessOrgClient owns named connections and provides nosqldb(), sqldb() and objectstorage() interfaces. Server engines remain separate packages; this client never starts a database or creates a bucket implicitly.

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

Install and connect

pnpm add @lossless.org/client

Document access works immediately: mongodb is a dependency because the NoSQLDB/MongoDB family is the package baseline. The relational and object drivers are optional peer dependencies, so install the ones your families use:

| Family | Additional install | | --- | --- | | nosqldb connections, @lossless.org/client/nosqldb, @lossless.org/client/testsupport | none | | sqldb connections with backend sqldb or mariadb | pnpm add mariadb | | sqldb connections with backend clickhouse | pnpm add @clickhouse/client | | objectstorage connections with backend s3 | pnpm add @aws-sdk/client-s3 |

The aggregate entry point imports a family only when the configuration names it, so an absent driver stays absent until it is used; connect() then fails with driver_missing, naming the package, the entry point and the install command. Importing @lossless.org/client/sqldb directly needs both relational drivers, because that entry point exports SqlConnection and ClickHouseConnection as values. optionalDrivers exposes the same table to tooling.

import { LosslessOrgClient } from '@lossless.org/client';

const client = new LosslessOrgClient({
  nosqldb: {
    app: { backend: 'mongodb', url: process.env.MONGODB_URL!, database: 'app' },
  },
  sqldb: {
    primary: { backend: 'mariadb', host: process.env.SQL_HOST!, database: 'app',
      user: process.env.SQL_USER!, password: process.env.SQL_PASSWORD! },
    analytics: { backend: 'clickhouse', url: process.env.CLICKHOUSE_URL!, database: 'app',
      username: process.env.CLICKHOUSE_USER!, password: process.env.CLICKHOUSE_PASSWORD! },
  },
  objectstorage: {
    assets: { backend: 's3', endpoint: process.env.S3_ENDPOINT!, region: 'us-east-1',
      credentials: { accessKeyId: process.env.S3_ACCESS_KEY!, secretAccessKey: process.env.S3_SECRET_KEY! },
      readinessBucket: 'existing-assets' },
  },
});

await client.connect({ timeoutMs: 30_000 });
try {
  const primary = client.sqldb('primary'); // SqlConnection
  const analytics = client.sqldb('analytics'); // ClickHouseConnection
  const rows = await primary.query<{ id: bigint }>({
    sql: 'SELECT id FROM accounts WHERE email = ?', values: ['[email protected]'],
  }, { maxRows: 100 });
  for await (const row of analytics.stream<{ temperature: number }>({
    sql: 'SELECT temperature FROM readings WHERE device = {device:String}',
    values: { device: 'ssd-1' },
  })) {
    // Process each row before requesting more; filtering happens on the server.
  }
  const readiness = await client.ready();
} finally {
  await client.close();
}

Select nosqldb, sqldb or objectstorage as the backend for the corresponding lossless.org server. An engine profile declares capabilities; it does not translate unsupported SQL or add backend features. SQLDB requires engine version 0.2.3 or later for pooled connection reset and parameterless prepared statements. It has a smaller SQL/type subset than MariaDB, and does not currently support TLS, savepoints or ALTER TABLE. ClickHouse uses its HTTP protocol and SQL dialect. There are no cross-backend transactions or automatic replication.

Literal connection names and backend discriminators determine the return types. Each client owns the connections it constructs. connect() probes databases without changing their schemas. S3 construction is local; ready() checks an explicitly configured existing bucket and returns readiness_bucket_required if none was supplied. close() is idempotent, stops admission, cancels owned SQL operations, closes pools and joins final metrics-writer flushes. Create a new aggregate client after failure or close.

Interfaces

| Import | API | | --- | --- | | @lossless.org/client | LosslessOrgClient, configuration, capabilities, readiness, LosslessClientError | | @lossless.org/client/nosqldb | Migrated SmartData models, decorators, collections, cursors, sessions, exact persistence and administration; NoSqlConnection | | @lossless.org/client/sqldb | SqlConnection, SqlTransaction, SqlTable, ClickHouseConnection, SmartClickHouseDb, tables, query builders, TimeDataTable, MetricWriter | | @lossless.org/client/objectstorage | Migrated SmartBucket, buckets, directories, files, metadata, watchers, exact operations; ObjectStorageConnection | | @lossless.org/client/testsupport | Explicit disposable-database testing helpers |

Family imports preserve the established SmartData and SmartBucket names and constructors. Their implementations live here; the client does not depend on the old packages. The aggregate loads only configured families. Direct family entry points do not initialize unrelated connections.

| @lossless.org/client/nosqldb addition | API | | --- | --- | | Monotonic counters and timestamps | $max / $min in ISmartdataAtomicUpdate, for declared numeric and date fields | | Race-free registration on a second unique key | $setOnInsert may seed a declared @unI() identity when upsert: true | | Bounded plural upsert | Model.atomicUpsertMany(operations, opts?) — 1..1000 per-document filter/update pairs in one unordered round trip | | Content-addressed primary keys | model option identityAsDocumentId on defineCollectionModel() and @Collection() | | Identities on a migrated unique index | @unI({ indexName }) names the index that backs the identity instead of <field>_1 |

docs/source/smartdata/readme.md documents each one with its refusals.

Relational SQL

SqlConnection uses the official MariaDB connector for parameters, protocol, TLS and decoding. This client owns cancellable admission and TCP sockets from authentication onward. A transaction holds one physical session; releasing a session resets its state before reuse.

• execute({ sql, values }, options) returns affectedRows, optional exact insertId: bigint, and completion: 'acknowledged'. Use it for statements that return an update result.
• query<Row>(statement, options) materializes at most 10,000 rows and 16 MiB by default. Set maxRows and maxBytes explicitly to change those limits. Overflow rejects with result_limit and closes the stream.
• stream<Row>(statement, options) iterates rows with backpressure. Early return, timeout, cancellation and connection shutdown close its owned stream/socket.
• insert(table, iterable, options) accepts iterable or async-iterable rows with the same columns, batching at 500 rows/4 MiB by default. Batches commit independently. A later failure reports partial_write with acknowledged rows, or ambiguous_write if the current batch outcome is unknown. Use a transaction with explicit statements for atomic multi-statement writes.
• transaction(async tx => ..., options) joins an outstanding final transaction operation before commit. Await each operation; concurrent operations on one transaction are rejected. A thrown callback rolls back when the session is still connected. An interrupted commit has an unknown outcome.
• table<Row>(name) handles an existing table with bound equality/null selectors, typed query, stream, insert, update and delete. Empty mutation selectors and undefined selector values are rejected. Schema creation is explicit SQL.

Values use placeholders; identifiers use a separate quoting function. Raw SQL text is trusted application code. A result generic is a caller-declared shape, not static validation of arbitrary SQL.

MariaDB returns BIGINT as bigint, DECIMAL as a string, binary columns as Buffer, SQL null as null, and dates/times as strings. Fractional timestamp strings retain server precision. JavaScript Date inputs bind in UTC with millisecond precision; MariaDB sessions use UTC. Exact fractional values beyond milliseconds should be supplied as strings. Unsafe integer numbers, non-finite numbers, invalid dates and unsupported parameter objects are rejected; use bigint or decimal strings for exact large values. SQLDB's supported scalar types follow its engine contract.

ClickHouse and metrics

ClickHouseConnection uses the official streaming HTTP connector. stream() consumes JSONEachRow batches without buffering the complete response. query() applies the same materialization limits as relational SQL. insert() streams an iterable with a default 4 MiB per-row limit and backpressure. It does not make a large insert atomic; a failed insert may have stored part or all of the input. JavaScript JSON row values must be serializable; represent 64-bit input integers as decimal strings. Int64/UInt64 and decimal results are returned as strings; decimal formatting follows the server and may omit trailing zeros. Binary data needs an explicit application encoding. Date/time strings follow the column's ClickHouse type and timezone.

execute() waits for server response completion and requests synchronous mutations, returning a query ID and acknowledged completion. It does not invent relational transactions for ClickHouse. Named parameters use ClickHouse syntax, such as {device:String}. Identifiers are quoted separately, including literal dots and backslashes.

connection.metrics exposes the migrated table/query features. Canonical metrics.createTable() defaults autoSchemaEvolution to false; preparation and schema changes are explicit. Standalone SmartClickHouseDb retains its existing opt-in startup/schema behavior for migrating applications.

const table = await client.sqldb('analytics').metrics.createTable<{ id: number; temperature: number }>({
  tableName: 'readings', orderBy: 'id',
  columns: [{ name: 'id', type: 'UInt32' }, { name: 'temperature', type: 'Float64' }],
});
const writer = table.createInsertStream({ batchSize: 500, maxBatchBytes: 4 * 1024 * 1024 });
await writer.write({ id: 1, temperature: 38.5 });
await writer.close(); // Joins the final server acknowledgement; rejects on failure.

Await every writer write() for backpressure. flush() and close() propagate errors, including background flush failures. Client close flushes owned writers before closing the connection. Timestamp-only polling watches cannot guarantee delivery of equal-timestamp or late rows and explicitly report unsupported_capability.

SQLDB advanced object metrics and disk-resident analytical scans remain unavailable. The client does not download objects to discover nested paths or aggregate them. Automatic deep-path indexing, correlated array predicates, expiry-driven path retirement and terabyte-scale storage require the corresponding qualified engine capability.

Budgets, errors and capabilities

New SQL methods accept signal and a total timeoutMs (30 seconds by default, including pool admission). Read cancellation closes owned transport resources. User-provided iterators receive return() on cancellation; their own pending I/O must also cooperate with the caller's signal. JavaScript cannot forcibly interrupt an arbitrary promise or transaction callback. Expiry revokes the transaction handle, so a callback resuming later cannot dispatch a write.

LosslessClientError provides a safe outer message, a code, optional backendCode, retryability, outcome and acknowledged-row evidence. cause preserves the original error and can contain statement values; do not log it indiscriminately. The taxonomy includes invalid arguments, unsupported capability, a missing optional driver, authentication, conflict, timeout, cancellation, result limits, backend failure, partial writes and unknown write outcomes. There are no automatic write retries. A transient hint never establishes that retrying an ambiguous write is safe.

Migrated document and object APIs retain their existing typed errors and exact-operation evidence. Their established per-operation timeout/ownership contracts continue to apply. capabilities distinguishes available, unsupported and unknown; MongoDB transaction/change-stream availability still depends on deployment topology. NoSQLDB change streams are explicitly unsupported. S3 exact-operation capability tests remain explicit and require owned disposable resources.

Migrating existing applications

Replace imports and the corresponding manifest dependency:

| Previous package | New import | | --- | --- | | @push.rocks/smartdata | @lossless.org/client/nosqldb | | @push.rocks/smartdata/testsupport | @lossless.org/client/testsupport | | @push.rocks/smartbucket | @lossless.org/client/objectstorage | | @push.rocks/smartclickhouse | @lossless.org/client/sqldb |

SmartData persisted identities, decorator symbols, BSON/null/undefined behavior and exact persistence contracts are retained. Existing normal application access stays through those public model APIs; direct driver access belongs to this foundation or explicit versioned migrations.

ObjectStorage retains companion <key>.metadata objects and .trash/<encoded-original-key> layouts. Existing buffer/replay helpers, list-array methods and watcher state materialize data and are outside the new streaming-memory guarantee. Metadata locks are advisory, not atomic distributed locks; directory move remains unsupported. The getStorageClient() migration escape hatch is retained. No readiness call creates or deletes a bucket.

SmartClickHouse migration changes: replace RxJS next/complete insertion with awaited writer write/close; join database close during application shutdown. Materialized SQL reads now have explicit limits. Query-builder toSQL() contains placeholders; use toStatement() or pass its parameters with the SQL. Watch APIs reject the unsupported continuation guarantee. Insert errors always reach the caller. Existing stored dotted-column names remain unchanged. Original API documentation and licenses are preserved in the repository under docs/source.

nosqldb lineage

@lossless.org/client/nosqldb is a fork of @push.rocks/smartdata 11.14.2 (commit 9868817a63de72caef787bd8e68bac3deb9d6131), taken on 2026-09-10; this repository continues that package's git history, so every upstream fix up to and including 11.14.2 is present here. nosqldbLineage, exported from the family entry point, carries the same baseline for runtime and tooling checks, and docs/source/smartdata keeps the upstream readme, hints and changelog.

The fork departs from that baseline in three places: NoSqlConnection adds the client backend, capability and readiness contract; the family entry point exports it together with nosqldbLineage; and watch() refuses to open a change stream when the connected backend reports change streams as unsupported. The package version file was dropped, because versioning belongs to @lossless.org/client. Document-model capabilities added after the fork — $max/$min, identity seeding through $setOnInsert, atomicUpsertMany() and identityAsDocumentId — are this package's own additions and have no upstream counterpart.

Fixes land here. @push.rocks/smartdata is no longer a release channel for this implementation, so a later upstream version is not a source to merge from and not an upgrade path for consumers of this package.

Verification

pnpm build builds production declarations. pnpm test checks aggregate lifecycle and writers. pnpm run test:nosqldb, test:mongodb, test:objectstorage and test:sqldb exercise isolated backends. pnpm run test:package packs the package and resolves every entry point; pnpm run test:install installs the packed tarball into a scratch project and proves that a document-only consumer receives no relational or object driver, and that the aggregate client refuses a missing one by name. Wrappers own and remove their disposable servers, containers and data directories. Never supply a production database or bucket to these destructive suites.

Qualification includes NoSQLDB 8.0.2, MongoDB 8.0.26 replica sets, ObjectStorage 10.1.0, SQLDB 0.2.3, MariaDB 11.8 and ClickHouse 25.8. pnpm run test:minio uses a digest-pinned MinIO image: sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e. That MinIO version enforces the tested conditional uploads but does not enforce conditional deletion; its exact-purge capability is correctly unavailable. ObjectStorage passes both live probes. AWS S3 has not been qualified against a live account in this migration; no universal S3 exact-operation guarantee is inferred from the SDK.

License and Legal Information

This repository contains open-source code licensed under the MIT License. A copy of the license can be found in the repository license file.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH or third parties, and are not included within the scope of the MIT license granted herein.

Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines or the guidelines of the respective third-party owners, and any usage must be approved in writing. Third-party trademarks used herein are the property of their respective owners and used only in a descriptive manner, e.g. for an implementation of an API or similar.

Company Information

Task Venture Capital GmbH Registered at District Court Bremen HRB 35230 HB, Germany

For any legal inquiries or further information, please contact us via email at [email protected].

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.