pinot-client
v0.3.0
Published
Node.js client for Apache Pinot
Downloads
1,701
Maintainers
Readme
Node.js client library for Apache Pinot :wine_glass:
JavaScript client library for connecting to and querying Apache Pinot :wine_glass:, a realtime distributed OLAP datastore.
GitHub Repo, TypeDoc Reference, npm Package
Features
- Implements a controller-based broker selector that periodically updates the table-to-broker mapping via the controller API.
- Provides TypeScript definitions of broker responses (Pinot query results).
- Has 100% test coverage.
Quick start
Start a development Pinot cluster with baseballStats
demo data:
docker run \
-p 9000:9000 -p 8000:8000 \
apachepinot/pinot:0.9.3 QuickStart \
-type batch
Installation
npm install pinot-client
or
yarn add pinot-client
Creating a connection to a Pinot cluster
With a simple broker selector (that chooses randomly between the provided brokers upon each query):
import { ConnectionFactory } from "pinot-client";
const connection = ConnectionFactory.fromHostList(["localhost:8000"]);
With a controller-based broker selector (that maintains a periodically updated table-to-broker mapping obtained via controller API):
import { ConnectionFactory } from "pinot-client";
const connection = await ConnectionFactory.fromController("localhost:9000");
Querying Pinot
const r = await connection.execute(
"baseballStats", // table name
"select league, sum(hits) as hits from baseballStats group by league order by hits desc" // SQL query
);
console.log(`Scanned ${r.numDocsScanned} docs in ${r.timeUsedMs}ms`);
console.log("Results:");
console.log(r.resultTable.dataSchema.columnNames.join("\t"));
r.resultTable.rows.forEach((row) => {
console.log(row.join("\t"));
});
Output:
Scanned 97889 docs in 8ms
league hits
NL 1890198
AL 1650039
AA 88730
NA 24549
FL 21105
PL 10523
UA 7457
Usage
fromHostList()
and fromController()
options
ConnectionFactory.fromHostList()
may optionally take as a second parameter an object with the following keys:
logger
: a logger instance conforming to the standard Log4j interface w/ .child() method (i.e. pino, winston or log4js)brokerReqHeaders
: additional HTTP headers (object key: value) to include in broker query API requestscustomHttpClient
: a custom HTTP client implementation
on top of that, ConnectionFactory.fromController()
options may include two additional keys:
controllerReqHeaders
: additional HTTP headers (object key: value) to include in controller API requestsbrokerUpdateFreqMs
: wait time in milliseconds between table-to-broker mapping refreshes
Example usage:
const options = {
brokerReqHeaders: {
Authorization: "Basic asdf123",
},
controllerReqHeaders: {
Authorization: "Basic xyz123",
},
brokerUpdateFreqMs: 500,
};
const connection = await ConnectionFactory.fromController("localhost:9000", options);
Using a custom logger
// let's use pino (not to be confused with pinot) as an example logger
import * as pino from "pino";
const pinoInstance = pino({ level: "debug" });
const childLogger = pinoInstance.child({ lib: "pinot-client" });
const options = {
logger: childLogger,
};
const connection = await ConnectionFactory.fromController("localhost:9000", options);
Using a custom HTTP client
By default pinot-client
uses undici
for performing HTTP requests against Pinot Borkers and Controllers. A custom HttpClient
interface implementation (containing POST and GET methods) can be provided instead via the customHttpClient
options key:
const myClient: HttpClient = {
get: async function <T>(url: string, options: { headers: Record<string, string> }) {
const { statusCode, parsedBody } = await otherHTTPClientLib<T>(url, ...);
// data is of type T
return { status: statusCode, data: parsedBody };
},
post: async function <T>(url: string, data: object, options: { headers: Record<string, string> }) {
const { statusCode, parsedBody } = await otherHTTPClientLib<T>(url, ...);
// data is of type T
return { status: statusCode, data: parsedBody };
},
};
const options = {
customHttpClient: myClient,
}
const c = await ConnectionFactory.fromController("http://localhost:9000", options);