hologres-search-sdk
v0.4.1
Published
Hologres Search SDK for Node.js - Vector search and full-text search
Maintainers
Readme
Hologres Search SDK
A TypeScript SDK for Hologres data retrieval operations, supporting vector search and full-text indexing.
✨ Attributes
- 🔍 Vector search: Retrieval feature based on semantic similarity
- 📝 Full-text index: Traditional keyword-based retrieval
- 💾 CRUD operations: Full create, read, update, delete capabilities (upsert, update, delete, truncate, overwrite)
- 🛡️ Type safety: Full TypeScript support with comprehensive type definitions
- 🧩 Modular design: Clear layered architecture for easy extension and maintenance
📦 Install
Install from npm
npm install hologres-search-sdk🚀 Get started
Basic usage
import { connect } from 'hologres-search-sdk';
// Connect to database
const client = await connect({
host: 'your-host',
port: 80,
database: 'your-database',
accessKeyId: 'your-access-key-id',
accessKeySecret: 'your-access-key-secret',
schema: 'public'
});
// Open table
const table = await client.openTable('table_name');
// Insert data
const data = [
[1, 'Hello world', [0.1, 0.2, 0.3]],
[2, 'TypeScript SDK', [0.4, 0.5, 0.6]],
[3, 'Vector search', [0.7, 0.8, 0.9]]
];
await table.insertMulti(data, ['id', 'content', 'vector']);
// Set vector index
await table.setVectorIndex(
'vector',
'Cosine',
'rabitq',
64, // maxDegree
400 // efConstruction
);
// Vector search
const queryVector = [0.1, 0.2, 0.3];
// Limit results
const results = await table.searchVector(queryVector, 'vector', undefined, 'Cosine')
.limit(10)
.fetchall();
// Set minimum distance
const filteredResults = await table.searchVector(queryVector, 'vector', undefined, 'Cosine')
.minDistance(0.5)
.fetchall();
// Set maximum distance
const maxDistResults = await table.searchVector(queryVector, 'vector', undefined, 'Cosine')
.maxDistance(0.9)
.fetchall();
// Close connection
await client.disconnect();Use try/finally for cleanup
import { connect } from 'hologres-search-sdk';
const client = await connect({
host: 'your-host',
port: 80,
database: 'your-database',
accessKeyId: 'your-access-key-id',
accessKeySecret: 'your-access-key-secret'
});
try {
// Execute database operations
const table = await client.openTable('table_name');
const results = await table.searchVector([0.1, 0.2, 0.3], 'vector_column', undefined, 'Cosine')
.fetchall();
} finally {
// Connection will close
await client.disconnect();
}Use await using (ES2022+)
import { connect } from 'hologres-search-sdk';
// Connection will close automatically when out of scope
await using client = await connect({
host: 'your-host',
port: 80,
database: 'your-database',
accessKeyId: 'your-access-key-id',
accessKeySecret: 'your-access-key-secret'
});
const table = await client.openTable('table_name');
const results = await table.searchVector([0.1, 0.2, 0.3], 'vector_column', undefined, 'Cosine')
.fetchall();📚 Detailed document
Core concepts
1. Client
The client is the primary API for interacting with the database:
import { connect } from 'hologres-search-sdk';
// Create and connect client
const client = await connect({
host: 'localhost',
port: 80,
database: 'test_db',
accessKeyId: 'your_key',
accessKeySecret: 'your_secret'
});
// Execute SQL
const result = await client.execute('SELECT COUNT(*) FROM users', true);
// Table operations
const table = await client.openTable('table_name');2. Table Operations
Table is the basic unit of data storage and search:
// Open existing table
const table = await client.openTable('table_name');
// Check if table exists
const exists = await client.checkTableExist('table_name');
// Delete
await client.dropTable('table_name');3. Data insert
Supports single and batch data inserts:
// Insert single record
await table.insertOne(
[1, 'title', 'Content', [0.1, 0.2, 0.3]],
['id', 'title', 'content', 'vector']
);
// Bulk insert
const data = [
[1, 'document1', 'Content1', [0.1, 0.2, 0.3]],
[2, 'document2', 'Content2', [0.4, 0.5, 0.6]],
[3, 'document3', 'Content3', [0.7, 0.8, 0.9]]
];
await table.insertMulti(data, ['id', 'title', 'content', 'vector']);4. CRUD operations
Supports full create, read, update, and delete operations:
// Upsert (insert or update)
// Single upsert
await table.upsertOne(
'id', // indexColumn
[1, 'updated content', [0.1, 0.2, 0.3]], // values
['id', 'content', 'vector'], // columnNames
true // update: Perform update on conflict
);
// Batch upsert
await table.upsertMulti(
'id', // indexColumn
[
[1, 'content1', [0.1, 0.2, 0.3]],
[2, 'content2', [0.4, 0.5, 0.6]]
],
['id', 'content', 'vector'], // columnNames
true, // update
['content', 'vector'] // updateColumns: Columns to update on conflict
);
// Update
await table.update(
['content', 'updated_at'], // columns
['new content', '2026-01-22'], // values
undefined, // tableAlias
undefined, // fromTable
undefined, // fromAlias
'id = 1' // condition
);
// Update with table alias and FROM clause
await table.update(
['status'], // columns
['active'], // values
't1', // tableAlias
'other_table', // fromTable
't2', // fromAlias
't1.id = t2.ref_id AND t2.flag = true' // condition
);
// Delete
await table.delete('id > 100'); // Delete records matching the condition
// Truncate
await table.truncate(); // Remove all data from the table while keeping the schema
// Overwrite - using values
await table.overwrite([
[1, 'new data1', [0.1, 0.2, 0.3]],
[2, 'new data2', [0.4, 0.5, 0.6]]
]);
// Overwrite - using query results
const sourceTable = await client.openTable('source_table');
await table.overwrite(undefined, sourceTable.select('*').where('active = true'));
// Drop table
await table.drop();5. Vector index
Create efficient retrieval indexes for vector columns:
// Set single vector index
await table.setVectorIndex(
'vector', // column
'Cosine', // distanceMethod: "Euclidean", "InnerProduct", "Cosine"
'rabitq', // baseQuantizationType: "sq8", "sq8_uniform", "fp16", "fp32", "rabitq"
64, // maxDegree
400, // efConstruction
true, // useReorder
'fp32', // preciseQuantizationType
'block_memory_io', // preciseIoType
4096, // maxTotalSizeToMergeMb: Maximum File Size of Data when merging on disk, in MB
16 // buildThreadCount: Number of threads used during index Build procedure
);
// Set multiple vector indexes
await table.setVectorIndexes({
content_vector: {
distanceMethod: 'Cosine',
baseQuantizationType: 'rabitq',
maxDegree: 64,
efConstruction: 400,
useReorder: true,
preciseQuantizationType: 'fp32',
maxTotalSizeToMergeMb: 4096,
buildThreadCount: 16
},
title_vector: {
distanceMethod: 'Euclidean',
baseQuantizationType: 'rabitq',
maxDegree: 32,
efConstruction: 200,
useReorder: true,
preciseQuantizationType: 'fp32',
maxTotalSizeToMergeMb: 4096,
buildThreadCount: 16
}
});
// Delete all vector indexes
await table.deleteVectorIndexes();6. Vector search
Execute semantic similarity retrieval:
// Basic vector search
const queryVector = [0.1, 0.2, 0.3];
const results = await table.searchVector(
queryVector, // vector
'vector', // column
undefined, // outputName
'Cosine' // distanceMethod
).fetchall();
// Retrieval with outputs alias
const resultsWithAlias = await table.searchVector(
queryVector, // vector
'vector', // column
'similarity_score', // outputName
'Cosine' // distanceMethod
).fetchall();7. Data query
Supports primary key-based term query:
// Query single record by primary key
const result = await table.getByKey(
'id', // keyColumn
1, // keyValue
['id', 'content', 'vector'] // returnColumns: Optional. If not specified, returns all columns
).fetchone();
// Batch query by primary key list
const results = await table.getMultiByKeys(
'id', // keyColumn
[1, 2, 3], // keyValues
['id', 'content'] // returnColumns: Optional. If not specified, returns all columns
).fetchall();8. Vector index management
Query and manage vector index information:
// Retrieve vector index information
const indexInfo = await table.getVectorIndexInfo();
if (indexInfo) {
console.log('Current vector index configuration:', indexInfo);
} else {
console.log('Vector index configuration not found');
}
// Index information sample return format
// {
// "vector_column": {
// "algorithm": "HGraph",
// "distance_method": "Cosine",
// "builder_params": {
// "max_degree": 64,
// "ef_construction": 400,
// "base_quantization_type": "rabitq",
// "use_reorder": true,
// "precise_quantization_type": "fp32",
// "precise_io_type": "block_memory_io",
// "max_total_size_to_merge_mb": 4096,
// "build_thread_count": 16
// }
// }
// }9. Full-text index
Create a full-text index for text columns:
// Create a full-text index
await table.createTextIndex(
'ft_idx_content', // indexName
'content', // column
'jieba' // tokenizer: "jieba", "ik", "icu", "whitespace", "standard", "simple", "keyword", "ngram", "pinyin"
);
// Set full-text index (modify the tokenizer of an existing index)
await table.setTextIndex(
'ft_idx_content', // indexName
'ik' // tokenizer
);
// Delete a full-text index
await table.dropTextIndex('ft_idx_content');10. Full-text retrieval
Execute a full-text retrieval query:
// Basic full-text retrieval
const results = await table.searchText(
'content', // column
'machine learning', // expression
0, // minThreshold
true, // returnScore
true // returnAllColumns
).fetchall();
// Full-text retrieval with scores returned
const resultsWithScore = await table.searchText(
'content', // column
'deep learning', // expression
0, // minThreshold
true, // returnScore
false, // returnAllColumns
'relevance_score' // returnScoreName
).select(['id', 'title', 'content']).fetchall();
// Use different retrieval patterns
// Keyword pattern (default)
const matchResults = await table.searchText(
'content', // column
'python programming', // expression
0, // minThreshold
true, // returnScore
false, // returnAllColumns
'text_search', // returnScoreName
'match', // mode
'AND' // operator: Require containing all keywords
).fetchall();
// Phrase pattern
const phraseResults = await table.searchText(
'content', // column
'machine learning', // expression
0, true, false, 'text_search',
'phrase' // mode: Exact phrase match
).fetchall();
// Natural language pattern
const nlResults = await table.searchText(
'content', // column
'+python -java', // expression: Must contain python, cannot contain java
0, true, false, 'text_search',
'natural_language' // mode
).fetchall();
// Term retrieval
const termResults = await table.searchText(
'content', // column
'python', // expression
0, true, false, 'text_search',
'term' // mode: Do not perform tokenization. Directly perform an exact match in the index
).fetchall();11. Advanced query build
Use the query builder to perform complex queries:
import { Filter, AndFilter, OrFilter, NotFilter } from 'hologres-search-sdk';
// Combine full-text search and filter conditions
const results = await table.searchText(
'content',
'artificial intelligence',
0, true, false, 'score'
)
.where("publish_date > '2023-01-01'")
.orderBy('score', 'desc')
.limit(10)
.fetchall();
// Use filter expressions
const filteredResults = await table.select(['id', 'title', 'content'])
.where(
AndFilter(
Filter("category = 'technology'"),
Filter('views > 1000')
)
)
.orderBy('views', 'desc')
.fetchall();
// Use OR filter expressions (method chaining)
const orResults = await table.select(['id', 'title', 'content'])
.where(
Filter("category = 'technology'").or(Filter('views > 1000'))
)
.orderBy('views', 'desc')
.fetchall();
// Use NOT filter
const notDeleted = await table.select(['id', 'title', 'content'])
.where(Filter("deleted = true").not())
.fetchall();
// Use TextSearchFilter for text search in WHERE clause
const textFilterResults = await table.select(['id', 'content'])
.where(TextSearchFilter('content', 'machine learning', 0.5, {
mode: 'match',
operator: 'AND',
tokenizer: 'jieba'
}))
.limit(10)
.fetchall();
// Use the tokenization feature
const tokenizeResults = await table.select(['id', 'content'])
.selectTokenize(
'content', // column
undefined, // text
'tokens', // outputName
'jieba' // tokenizer
)
.limit(5)
.fetchall();12. Table join query
Support multi-table join queries:
// Inner join
const table1 = await client.openTable('articles', 'a');
const table2 = await client.openTable('authors', 'b');
const joinResults = await table1.select(['a.id', 'a.title', 'b.name'])
.innerJoin('authors', 'a.author_id = b.id', 'b')
.where("a.publish_date > '2023-01-01'")
.fetchall();
// Left join
const leftJoinResults = await table1.select(['a.id', 'a.title', 'b.name'])
.leftJoin('authors', 'a.author_id = b.id', 'b')
.fetchall();
// Right join
const rightJoinResults = await table1.select(['a.id', 'a.title', 'b.name'])
.rightJoin('authors', 'a.author_id = b.id', 'b')
.fetchall();
// Full outer join
const fullJoinResults = await table1.select(['a.id', 'a.title', 'b.name'])
.fullJoin('authors', 'a.author_id = b.id', 'b')
.fetchall();
// Cross join
const crossJoinResults = await table1.select(['a.id', 'b.category'])
.crossJoin('categories', 'c')
.fetchall();13. GUC Configuration
Manage Global User Configuration (GUC) settings:
// Set a GUC parameter
await client.setGuc('hg_experimental_enable_vector_disk_mode', 'on');
// Set a GUC parameter at database level
await client.setGuc('hg_experimental_enable_vector_disk_mode', 'on', true);
// Set GUC to "on"
await client.setGucOn('hg_experimental_enable_vector_disk_mode');
// Set GUC to "off"
await client.setGucOff('hg_experimental_enable_vector_disk_mode');
// Show current GUC value
const value = await client.showGuc('hg_experimental_enable_vector_disk_mode');
console.log('GUC value:', value);14. Table Metadata
Query table schema and column information:
// Get all column names
const columns = await table.getColumnNames();
console.log('Columns:', columns);
// Get primary key columns
const primaryKeys = await table.getPrimaryKeys();
console.log('Primary keys:', primaryKeys);
// Get full table schema
const schema = await table.getTableSchema();
console.log('Table schema:', schema);
// Vacuum the table
await table.vacuum();15. Query Builder
Build custom queries using the query builder:
// Build a query from client
const query = client.buildQuery('my_table', 't');
const results = await query
.select(['t.id', 't.name'])
.where("t.status = 'active'")
.orderBy('t.created_at', 'desc')
.limit(10)
.fetchall();
// Build a query from table
const tableQuery = table.buildQuery();
const count = await tableQuery
.select(['COUNT(*) as total'])
.where('created_at > NOW() - INTERVAL \'7 days\'')
.fetchone();
// Fetch multiple results with limit
const batchResults = await table.select('*')
.fetchmany(50); // Fetch up to 50 results
// Explain query execution plan
const plan = await table.select('*')
.where('id > 100')
.explain();
// Explain analyze with actual execution
const analyzePlan = await table.select('*')
.where('id > 100')
.explainAnalyze();Configuration options
Connection configuration
import type { ConnectionConfig } from 'hologres-search-sdk';
const config: ConnectionConfig = {
host: 'your-host.com',
port: 80,
database: 'production_db',
accessKeyId: 'user...',
accessKeySecret: 'secret...',
schema: 'analytics', // The default is "public"
autocommit: false // Whether to enable autocommit mode (default: false)
};Vector index configuration
distanceMethod: Distance computation method
"Euclidean": Euclidean distance"InnerProduct": Inner product distance"Cosine": Cosine distance
baseQuantizationType: Base quantization type
"sq8","sq8_uniform","fp16","fp32","rabitq"
maxDegree: The quantity of nearest neighbors that each vertex attempts to connect to during the graph build procedure (default: 64)
efConstruction: Retrieval depth control during the graph build procedure (default: 400)
useReorder: Specifies whether to use the HGraph high-precision index (default: false)
preciseQuantizationType: Precise quantization type (default: "fp32")
preciseIOType: Precise IO type (default: "block_memory_io")
maxTotalSizeToMergeMb: The maximum file size of data during disk merge, in MB (default: 4096)
buildThreadCount: The number of threads used during the index build procedure (default: 16)
Full-text search configuration
- tokenizer: Tokenizer type
- mode: Full-text search mode
match: Keyword match. Default.phrase: Phrase searchnatural_language: Natural language searchterm: Term search
- operator: Keyword search operator (Applicable only to match mode. Default: "OR")
- Token filter:
lowercase: Converts uppercase letters in tokens to lowercase.stop: Removes stop word tokens.stemmer: Converts tokens to their corresponding stems based on the syntax rules of the corresponding language.length: Removes tokens that exceed the specified length.removepunct: Removes tokens that contain only punctuation characters.pinyin: Pinyin token filter
🔧 API Reference
Main classes
Client: Database client. It manages connections and table operations.HoloTable: Table operation API. It supports data insertion, vector search, and full-text search.QueryBuilder: Query builder. It supports chain invocation to build complex queries.ConnectionConfig: Connection configuration interface.VectorIndexConfig: Vector index configuration interface.
Filter classes
Filter: Basic filter expression factory function.FilterExpression: Filter expression class with.and(),.or(),.not()methods for combining conditions.AndFilter: AND logic filter factory function (combines multiple conditions with AND).OrFilter: OR logic filter factory function (combines multiple conditions with OR).NotFilter: NOT logic filter factory function (negates a condition).TextSearchFilter: Full-text search filter factory function for use in WHERE clauses.LogicalOperator: Enum for AND/OR operators.TextSearchFilterOptions: Options interface for TextSearchFilter (mode, operator, tokenizer, etc.).
Main functions
Connection and Table Management:
connect(): Creates and connects a database client.openTable(): Opens an existing table.checkTableExist(): Checks whether a table exists.dropTable(): Deletes a table.
Data operation:
insertOne(): Inserts a single record.insertMulti(): Bulk inserts records.upsertOne(): Inserts or updates a single record (ON CONFLICT).upsertMulti(): Bulk inserts or updates records.update(): Updates records that match the given condition.delete(): Deletes records that match the given condition.truncate(): Removes all data from the table.overwrite(): Overwrites all data in the table.drop(): Drops the table.getByKey(): Queries a single record based on the primary key.getMultiByKeys(): Batch queries based on a primary key list.
Vector search:
setVectorIndex(): Sets a single vector index.setVectorIndexes(): Sets multiple vector indexes.deleteVectorIndexes(): Deletes all vector indexes.getVectorIndexInfo(): Retrieves vector index information.searchVector(): Executes vector search.
Full-text search:
createTextIndex(): Creates a full-text index.setTextIndex(): Modifies a full-text index.dropTextIndex(): Deletes a full-text index.resetTextIndex(): Resets a full-text index.getIndexProperties(): Retrieves index properties.showTokenizeEffect(): Shows tokenization effect for text.searchText(): Executes full-text search.
Query building:
select(): Selects the columns to returnwhere(): Adds filter conditionsandWhere(): Adds AND filter conditionsorWhere(): Adds OR filter conditionsorderBy(): Sorts the resultsgroupBy(): Groups the resultslimit(): Limits the quantity of resultsoffset(): Skips the specified quantity of resultsjoin(): Table joininnerJoin(): Inner joinleftJoin(): Left joinrightJoin(): Right joinfullJoin(): Full outer joincrossJoin(): Cross joinselectTokenize(): Displays tokenization effectsselectTextSearch(): Performs full-text search in SELECTwhereTextSearch(): Performs full-text search filtering in WHEREandWhereTextSearch(): Adds AND full-text search filtering in WHEREorWhereTextSearch(): Adds OR full-text search filtering in WHEREbuildQuery(): Creates a QueryBuilder instanceminDistance(): Sets minimum distance threshold for vector searchmaxDistance(): Sets maximum distance threshold for vector searchfetchmany(): Fetches specified number of resultsexplain(): Returns query execution planexplainAnalyze(): Returns query execution plan with actual execution statistics
GUC Configuration:
setGuc(): Sets a GUC parametersetGucOn(): Sets a GUC parameter to "on"setGucOff(): Sets a GUC parameter to "off"showGuc(): Shows current GUC value
Table Metadata:
getColumnNames(): Gets all column namesgetPrimaryKeys(): Gets primary key columnsgetTableSchema(): Gets full table schemavacuum(): Vacuums the table
Exception classes
HoloSearchError: Base exception classConnectionError: Connection-related errorsQueryError: Query execution errorsSqlError: SQL generation errorsTableError: Table operation errorsErrorDetails: Error details interface
SQL Utility Functions
buildTextSearchSql(): Builds SQL for text searchbuildTokenizeSql(): Builds SQL for tokenizationbuildAnalyzerParamsSql(): Builds SQL for analyzer parametersquoteIdentifier(): Quotes SQL identifiersquoteLiteral(): Quotes SQL literals
Types
DistanceType: Distance calculation type ('Euclidean','InnerProduct','Cosine')BaseQuantizationType: Base quantization type ('sq8','sq8_uniform','fp16','fp32','rabitq')PreciseQuantizationType: Precise quantization type ('sq8','sq8_uniform','fp16','fp32')PreciseIOType: Precise IO type ('block_memory_io','reader_io')TokenizerType: Tokenizer type ('jieba','ik','icu','whitespace','standard','keyword','simple','ngram','pinyin')TextFilterType: Text filter type ('lowercase','stop','stemmer','length','removepunct','pinyin')TextSearchModeType: Text search mode ('match','phrase','natural_language','term')TextSearchOperatorType: Text search operator ('AND','OR')PinyinFilterParamType: Pinyin filter parametersVectorSearchFunction: Mapping of distance types to SQL functionsFilterParams: Filter parameters interfaceTokenizerParams: Tokenizer parameters interfaceTextSearchOptions: Text search options interfaceFilterParamValue: Filter parameter value type
🛠️ Development
Requirements
- Node.js 18+
- TypeScript 5+
Setup
npm installCommands
# Lint code
npm run lint
# Format code
npm run format
# Build the SDK
npm run buildRunning Tests
Unit Tests
Unit tests run without a database connection and test individual components:
# Run unit tests
npm test
# Run unit tests in watch mode
npm run test:watch
# Run unit tests with coverage report
npm run test:coverageIntegration Tests
Integration tests require a real Hologres database connection. Configure the following environment variables:
# Required environment variables
export HOLO_HOST="your-hologres-host"
export HOLO_DATABASE="your-database"
export HOLO_ACCESS_KEY_ID="your-access-key-id"
export HOLO_ACCESS_KEY_SECRET="your-access-key-secret"
# Optional environment variables
export HOLO_PORT="80" # Default: 80
export HOLO_SCHEMA="public" # Default: publicAlternatively, create a .env file in the project root:
HOLO_HOST=your-hologres-host
HOLO_DATABASE=your-database
HOLO_ACCESS_KEY_ID=your-access-key-id
HOLO_ACCESS_KEY_SECRET="your-access-key-secret" # Use quotes if password contains special characters like #
HOLO_PORT=80
HOLO_SCHEMA=publicRun integration tests:
# Run integration tests
npm run test:integration
# Run integration tests in watch mode
npm run test:integration:watchNote: Integration tests are automatically skipped if the required environment variables are not configured.
📦 Publishing
Prerequisites
Before publishing, ensure you have:
- Node.js 18+ installed
- npm account with access to the package registry
- All tests passing
Release Checklist
Run integration tests (optional, requires database connection)
npm run test:integrationUpdate version
# For patch releases (bug fixes) npm version patch # For minor releases (new features, backward compatible) npm version minor # For major releases (breaking changes) npm version majorNote: The
npm versioncommand automatically runs lint, unit tests, and build via lifecycle hooks (preversionandversionscripts in package.json).Verify npm login
npm whoamiIf not logged in, run:
npm loginPublish to npm
# Publish npm publish --dry-run # Test publish first npm publishNote: The
prepublishOnlyscript in package.json automatically runsnpm run buildbefore publishing.
Post-Publishing
After publishing, verify the package is available:
npm view hologres-search-sdk versions --jsonCreate a git tag for the release:
git push origin --tagsUpdate the CHANGELOG.md with the new version details.
📄 License
This project uses the Apache-2.0 License - View the LICENSE file for details.
Hologres Search SDK - Makes Hologres vector and full-text retrieval simple and efficient 🚀
