LumiDB SDK for JavaScript/Typescript

The official LumiDB SDK for the browser and node.js.

[!CAUTION] The LumiDB API is currently in active development and should be considered unstable. While we strive to maintain functionality, breaking changes may occur between package versions. The API does not yet provide backward compatibility guarantees, and endpoints, request/response formats, and behavior may change significantly between versions.

Installation

npm install --save @lumidb/lumidb

API

An instance of LumiDB provides the following functionality:

Authentication

• setAuthentication Configure the authentication method used for queries.

Table Management

• listTables Retrieve a list of all accessible tables in the database.

Data Operations

• query Query for point data.
• getMetadata List the metadata for all multiple tables with optional filtering.
• getTableMetadata Get complete metadata for a single table, including table-level information and metadata for all original files with optional filtering.
• getDownloadUrls Get download URLs for original files.
• get3DTileset Get a 3D Tiles tileset description.
• get3DTile Get a single 3D tile from a tileset.

Exports

• createExport Start a data export operation.
• getExportStatus Poll for export progress.
• downloadExportUrl Download a finished export.

Usage example

import { LumiDB } from "@lumidb/lumidb";

const lumidb = new LumiDB({ baseUrl: "<LUMIDB_URL>", auth: { apikey: "<LUMIDB_APIKEY>" } });

// Get a list of accessible database tables
const tables = await lumidb.listTables();

// Query points from a table
const response = await lumidb.query({
    tableName: tables[0].tableName,
    queryBoundary: {
        Polygon: [
            // polygon boundary
            [
                [
                    [2777342, 8438905],
                    [2777342, 8439315],
                    [2777573, 8439315],
                    [2777573, 8438905],
                    [2777342, 8438905],
                ],
            ],
            // optional z range
            [0, 100],
        ],
    },
    queryProj: "EPSG:3857",
    maxPoints: 5_000_000,
    maxDensity: 10.0,
    sourceFileFilter: null, // see below
    classFilter: null,
    outputType: "threejs", // "laz" or "e57" for exporting to other software
});

console.log(`LumiDB responded with ${response.pointCount} points and ${response.bytes} bytes`);

// Load the points into three.js:

import * as THREE from "three";
import { PointCloudMaterial } from "@lumidb/lumidb";

const scene = new THREE.Scene();

const material = new PointCloudMaterial({ pointSize: 3.0 });
const points = new THREE.Points(response.pointGeometry, material);

scene.add(points);

For a more complete example, check out a more complete example of integrating LumiDB into a three.js -based viewer.

Authentication

Option 1: Server-Created Tokens (Recommended)

Enhanced security with fine-grained access control. Token-based auth gives you full control of what each client is allowed to access.

You must implement a custom authentication endpoint on your server and configure the LumiDB client to use it for fetching tokens.

// Example callback function:
async function getLumiDbTokenExample(): string {
    // Call a custom endpoint on your server. You can re-use existing logic for authenticated API calls.
    const response = await fetch("/path/to/your/endpoint", {
        headers: {
            /* your auth headers */
        },
        body: JSON.stringify({
            /* extra info when using POST */
        }),
    });

    // The server should:
    // 1. Run it's own auth logic
    // 2. If request is allowed, call the lumidb /api/create_token. See HTTP API documentation further down.
    // 3. Pass the lumidb token back to with the reponse

    // Parse the token from your server response
    const body = await response.json();
    const token = body.lumidb_token;

    return token;
}

// Register the callback
lumidb.setAuthentication({ getTokenCallback: getLumiDbTokenExample });

// The client ill now automatically refresh tokens as needed before making API calls.

Option 2: Direct API key

Simplest approach, but the client is provided full access to all resources.

lumidb.setAuthentication({ apikey: "<LUMIDB_APIKEY>"});

⚠️ Security Considerations:

• Grants full scope access to your tables
• Not recommended for client-side applications
• Appropriate for protected server environments only

Filtering

Query filtering based on file metadata is a central concept in the API. You can get the metadata for a table using getMetadata or getTableMetadata, which returns a TableMetadata object containing both table-level and file-level metadata. Each file entry contains the metadata given by the user at the time of import, in addition to metadata augmented by LumiDB automatically, such as file names, sizes, point counts and so on. The file metadata is stored in a map where the keys correspond to each files source file ID.

These IDs can then be passed in an array as the sourceFileFilter whenever calling LumiDB.query() to filter the point data on the server to contain only points coming from files corresponding to the given IDs.

The same mechanism will work on any metadata, such as scanner type, author name, vehicle identifier etc.

Example: By date

As a practical example, let's say you want to filter the points based on a scan time/date range. For each file, at import you would provide a metadata field, such as scanDate with a value corresponding to the time at which the file was scanned.

Then in your application, you'd call getTableMetadata() and choose the valid source file IDs that are within your specified time/date range of interest. Then, you'd pass this list of IDs in the sourceFileFilter when calling query(), and the resulting set of points will only contain points scanned within the specified time span.

LumiDB HTTP API

POST /api/create_token

Create a LumiDB token that can be used for authentication. The token will have access to the specified tables.

Request

Headers

| Header | Value | | ------------- | ---------------------- | | Content-Type | application/json | | Authorization | apikey <LUMIDB_APIKEY> |

Body Parameters

| Parameter | Type | Required | Description | | --------- | -------- | -------- | --------------------------------------------------- | | tables | string[] | Yes | List of tables that the token gives read access to. |

Example Request

{
    "tables": ["tokyo", "new_york"]
}

Response

{
    "lumidb_token": "<LUMIDB_TOKEN>"
}

POST /api/assets/upload

Create a presigned upload URL for uploading an asset to LumiDB. This endpoint generates a unique asset ID and returns a presigned URL that can be used to upload the asset file directly.

[!NOTE] The API only supports only uploading files up to 5GB.

Request

Headers

| Header | Value | | ------------- | ---------------------- | | Content-Type | application/json | | Authorization | apikey <LUMIDB_APIKEY> |

Body Parameters

| Parameter | Type | Required | Description | | ---------- | ------------------- | -------- | ---------------------------------------------- | | name | string | Yes | Name of the asset being uploaded. | | metadata | object | No | Optional metadata associated with the asset. |

Example Request

{
    "name": "my-pointcloud-data.laz",
    "metadata": {
        "source": "drone_survey",
        "date": "2024-01-15",
        "location": "helsinki"
    }
}

Response

{
    "upload_url": "<PRESIGNED UPLOAD URL>",
    "asset_id": "01961997-4e23-7962-aeac-f0ddc706b402",
    "expiration_seconds": 3600
}

GET /api/assets

List all assets available in the LumiDB instance. Returns metadata for all uploaded assets.

Request

Headers

| Header | Value | | ------------- | ---------------------- | | Authorization | apikey <LUMIDB_APIKEY> |

Response

{
    "assets": [
        {
            "asset_id": "01961997-4e23-7962-aeac-f0ddc706b402",
            "name": "my-pointcloud-data.laz",
            "started_at": "2024-01-15T10:30:00Z",
            "metadata": {
                "source": "drone_survey",
                "date": "2024-01-15",
                "location": "helsinki"
            }
        }
    ]
}

POST /api/tables/import

Import a set of files into a LumiDB table.

[!NOTE] Calling import on an already existing table will create a new version of the table, overwriting the previous version.

Request

Headers

| Header | Value | | ------------- | ---------------------- | | Content-Type | application/json | | Authorization | apikey <LUMIDB_APIKEY> |

Body parameters

{
	"table_name": string,
	"table_proj": string,
	
	// If true, the import will fail if any of the input files have incompatible projections.
	// Defaults to false.
	"strict_mode": false,
	
	// Optional valid bounds for the table. Any assets outside these bounds will be rejected during import.
	// This can be useful to skip inputs where the projection information is incorrect.
	// Depending on `strict_mode` setting, the import will either fail or assets outside the bounds will be ignored.
	// Defaults to null
	"valid_bounds": {
	    // [xmin, ymin, xmax, ymax]
	    "extent": [24, 61, 26, 63],
		"proj": "EPSG:4326"
	},
	
	"inputs": {
	    // NOTE: The `src` and `id` fields are mutually exclusive. You must provide either a source URL or an asset ID.

		// Reference to an asset ID if the file is already uploaded to LumiDB.
		"asset_id": "<ASSET_ID>",

		// Location for the input file (e.g. a presigned S3 url).
		// LumiDB will automatically fetch the file from this URL.
		"src": string,

		// File projection. EPSG code or PROJ.4 string.
		"proj": string,

		// rest of the per-file fields are treated as user-defined metadata
		// common ones could be `date`, `license`, `producer`, etc..
	}[]
}

Example Request

{
    "table_name": "helsinki",
    "table_proj": "EPSG:3879",
    "inputs": [
        {
            "src": "https://ptp.hel.fi/DataHandlers/Lidar_kaikki/Default.ashx?q=673496b&y=2021",
            "proj": "EPSG:3879",
            "date": "2017",
            "license": "Helsingin kaupunki"
        }
    ]
}

Response

{
    "status": "queued",
    "table_version": "01961997-4e23-7962-aeac-f0ddc706b402"
}

GET /api/tables/import_status/:version_id

Get the status of an import job. Can be polled to track the progress of an ongoing import. The version_id parameter is the ID returned in the response of the /api/tables/import endpoint.

Request

Headers

| Header | Value | | ------------- | ---------------------- | | Authorization | apikey <LUMIDB_APIKEY> |

Response

{
  status: "queued" | "processing" | "failed" | "ready",
  progress?: number
  error_msg?: string
}

Example

{
    "status": "processing",
    "progress": 72.25
}

Changelog

v0.7.2

• Support non-ASCII characters in table names.
• Support glTF format for 3D Tiles.
• Add patchTableUserData() for updating user-defined metadata.
• get3DTileset: Support creating a single 3D tileset from multiple datasets.
• createExport: Add outputOrigin option to modify the georeferencing of exported data.

v0.7.1

• Fix incorrect JWT caching behaviour when authenticating with server-created tokens.

v0.7.0

• getMetadata: New function for fetching metadata for multiple tables with optional spatial filtering.
• getTableMetadata: Added support for spatial filtering.
• Restructured table metadata format:
  • Added more system-created metadata fields
  • Separated user-defined metadata into its own dedicated object
  • Table boundaries now returned as GeoJSON MultiPolygon

v0.6.0

• Improve TypeScript configuration for better compatibility across tools and bundlers.
• New Export API
• getTableMetadata: return table boundaries as GeoJSON MultiPolygons
• getTableMetadata: return asset id's for all files in a table

v0.5.4

• Update license to Apache-2.0
• Added validation to all LumiDB methods.

v0.5.3

• get3DTileset: Add tileId for each 3D tile.
• get3DTileset: Require outputProj to be defined. Use null with Cesium.
• Remove unnecessary console logging.

v0.5.2

• Add support for 3D Tiles
• listTables, getTableMetadata: Return table bounding box.

v0.5.1

• Fix example code in the README

v0.5.0

• Support for token-based authentication
• Projections can be defined as PROJ.4 strings in addition to EPSG codes.
• Add LumiDBError and LumiDBHttpError

v0.4.0

• Change HTTP query parameters to be compatible with updated HTTP API.
• query: Add option to limit Z range when query area is a polygon.
• query: Return the bounding box with query responses.
• query: Add processing time to the JSON header.
• listMetadata: return sensor_poses (for E57 files)

v0.3.0

• Add JSON header to point data response to enable better backward compatibility for future updates.
• Return intensity range and classification list with point response.

v0.2.3

• Allow newer version of three.js to be used as peer dependency
• Return query result offset as an array instead of THREE.Vector3

v0.2.2

• Make three.js peer dependency version limit less strict (from ^0.169.0 to ^0.152.0)

v0.2.1

• Add missing auth header to getDownloadUrl

v0.2.0

• First public release