Typed ES

Automatically add output types to your Elasticsearch queries.

Tested with Elasticsearch ^8 and Elasticsearch ^9

Bucket Aggregations

| Aggregation | Status | Documentation | |-------------|--------|---------------| | Adjacency Matrix | ✅ | docs | | Auto Date Histogram | ✅ | docs | | Categorize Text | ✅ | docs | | Children | ✅ | docs | | Composite | ✅ | docs | | Date Histogram | ✅ | docs | | Date Range | ✅ | docs | | Diversified Sampler | ✅ | docs | | Filter | ✅ | docs | | Filters | ✅ | docs | | Frequent Item Sets | ✅ | docs | | Geohash Grid | ✅ | docs | | Geohex Grid | ✅ | docs | | Geotile Grid | ✅ | docs | | Global | ✅ | docs | | Histogram | ✅ | docs | | IP Prefix | ✅ | docs | | IP Range | ✅ | docs | | Missing | ✅ | docs | | Multi Terms | ✅ | docs | | Parent | ✅ | docs | | Nested | ✅ | docs | | Random Sampler | ✅ | docs | | Range | ✅ | docs | | Rare Terms | ✅ | docs | | Reverse Nested | ✅ | docs | | Sampler | ✅ | docs | | Significant Terms | ✅ | docs | | Significant Text | ✅ | docs | | Terms | ✅ | docs | | Time Series | ❌ | docs | | Variable Width Histogram | ✅ | docs |

Metrics Aggregations

| Aggregation | Status | Documentation | |-------------|--------|---------------| | Avg | ✅ | docs | | Boxplot | ✅ | docs | | Cardinality | ✅ | docs | | Cartesian Bounds | ✅ | docs | | Cartesian Centroid | ✅ | docs | | Extended Stats | ✅ | docs | | Geo Bounds | ✅ | docs | | Geo Centroid | ✅ | docs | | Geo Line | ✅ | docs | | Matrix Stats | ✅ | docs | | Max | ✅ | docs | | Median Absolute Deviation | ✅ | docs | | Min | ✅ | docs | | Percentile Ranks | ✅ | docs | | Percentiles | ✅ | docs | | Rate | ✅ | docs | | Scripted Metric | ✅ | docs | | Stats | ✅ | docs | | String Stats | ✅ | docs | | Sum | ✅ | docs | | T-Test | ✅ | docs | | Top Hits | ✅ | docs | | Top Metrics | ✅ | docs | | Value Count | ✅ | docs | | Weighted Avg | ✅ | docs |

Pipeline Aggregations

| Aggregation | Status | Documentation | |-------------|--------|---------------| | Average Bucket | ✅ | docs | | Bucket Script | ❌ | docs | | Bucket Count K-S Test | ❌ | docs | | Bucket Correlation | ❌ | docs | | Bucket Selector | ❌ | docs | | Bucket Sort | ✅ | docs | | Change Point | ❌ | docs | | Cumulative Cardinality | ❌ | docs | | Cumulative Sum | ❌ | docs | | Derivative | ❌ | docs | | Extended Stats Bucket | ✅ | docs | | Inference | ❌ | docs | | Max Bucket | ✅ | docs | | Min Bucket | ✅ | docs | | Moving Function | ❌ | docs | | Moving Percentiles | ✅ | docs | | Normalize | ❌ | docs | | Percentiles Bucket | ✅ | docs | | Serial Differencing | ❌ | docs | | Stats Bucket | ✅ | docs | | Sum Bucket | ✅ | docs |

Features

• Automatic type based on options: Automatically infers output types from query options (e.g., returning total count).
• Automatic output type based on requested fields and aggregations: Derives precise types from specified _source, fields, docvalue_fields and aggregations configurations.
• Understand wildcards: The library correctly detects and infers output types even when using wildcards in _source.
  For example, given an index with fields { created_at: string; title: string },
  specifying _source: ["*_at"] will correctly return { created_at: string } in the output type.
• Supports search, msearch and asyncSearch: You can still use the native types if something goes wrong (see What if the library is missing a feature that you need?).

Example Usage

type MyIndex = {
   "my-index": {
      id: number;
      name: string;
      created_at: string;
   }
};

// Having to use `as unknown` is less than ideal, but as we're overriding types, typescript isn't very happy
const client = new Client({/* config */}) as unknown as TypedClient<Indexes>;

// Query with _source (wildcard), fields, aggregation, and options
const query = typedEs(client, {
	index: "my-index",
	_source: ["id", "na*"],
	fields: [
		{
			field: "created_at",
			format: "yyyy-MM-dd",
		},
	],
	track_total_hits: true,
	rest_total_hits_as_int: true, // Ensures total value is returned as a number
	aggs: {
		name_counts: { terms: { field: "name" } },
	},
});

const result = await client.search(query);
const total = result.hits.total; // number
const firstHit = result.hits.hits[0]; // { _source: { id: number; name: string}, fields: { created_at: string[] } }
const aggregationBuckets = result.aggregations.name_counts.buckets; // Array<{ key: string | number; doc_count: number; }>

Why This Library?

To highlight the benefits, here's a comparison with/without the library:

Without providing any types

const result = await client.search(query);
const total = result.hits.total; // number | estypes.SearchTotalHits | undefined
const firstHit = result.hits.hits[0]._source; // unknown
const aggregationBuckets = result.aggregations!.name_counts.buckets; // any, ts error: Object is possibly 'undefined'.

With manual type definitions

const result = await client.search<
  { id: number; created_at: string; },
  {
    name_counts: {
      buckets: Array<{ key: string; doc_count: number }>;
    };
  }
>(query);

const total = result.hits.total; // number | estypes.SearchTotalHits | undefined
const firstHit = result.hits.hits[0]; // { _source: { id: number; created_at: string; } | undefined, fields: Record<string, unknown> }
const aggregationBuckets = result.aggregations!.name_counts.buckets; // Array<{ key: string; doc_count: number; }>

With @vahor/typed-es

// Automatic type inference - no manual definitions needed
const result = await client.search(query);
const total = result.hits.total; // number
const firstHit = result.hits.hits[0]._source; // { id: number; created_at: string }
const aggregationBuckets = result.aggregations.name_counts.buckets; // Array<{ key: string | number; doc_count: number }> 

Install

bun add @vahor/typed-es

Note: you can install it in dev-dependencies if you don't plan to use the typedEs function.

Usage

Step 1: Define your index types

type CustomIndexes = {
    "first-index": {
        score: number;
        entity_id: string;
        date: string;
    },
    "second-index": {
        "some-field": string;
    }
}

ex:

{
    "mappings": {
        "properties": {
            "location": {
                "type": "point"
            },
            "date": {
                "type": "date"
            }
        }
    }
}

would give:

type CustomIndexes = {
	"first-index": {
		location: string;
		date: string;
	};
};

Step 2: Create a client

import { Client } from "@elastic/elasticsearch";
import { TypedClient } from "@vahor/typed-es";

const client = new Client({
    ... // elasticsearch client config
}) as unknown as TypedClient<CustomIndexes>;

Step 3: Use the typedEs function

import { typedEs } from "@vahor/typed-es";

const query = typedEs(client, {
    index: "first-index",
    _source: ["score", "entity_id", "*ate"],
});

const queryWithAggs = typedEs(client, {
    index: "first-index",
    _source: ["score", "entity_id", "*ate"],
    aggs: {
        some_agg: {
            terms: {
                field: "entity_id",
            },
        },
    },
});

typedEs is a simple wrapper that adds type safety to index, autocompletes on _source. Check its definition in typed-es.ts, you can reuse the same definition to add default values to your queries.

Note: when _source is missing, the output will contain every fields.

Step 4: Enjoy an easy type-safe output

// Use the elasticsearch client as usual
const output = await client.search(query);

// And without having to add .search<Sources, Aggs>(query) everywhere, you now have access to the correct types
const hits = output.hits.hits;
for (const hit of hits) {
    // Here hit is typed as { _source: { score: number; entity_id: string, date: string } }
    const score = hit._source.score; // typed as number
    const entity_id = hit._source.entity_id; // typed as string
    const invalid = hit._source.invalid; // error: Property 'invalid' does not exist on type '{ score: number; entity_id: string; }'
}


const outputWithAggs = await client.search(queryWithAggs);
const aggs = outputWithAggs.aggregations;
const someAgg = aggs.some_agg;
const someAggTerms = someAgg.buckets;
for (const bucket of someAggTerms) {
    // Here bucket is typed as { key: string | number; doc_count: number }
    const key = bucket.key; // typed as string | number
    const doc_count = bucket.doc_count; // typed as number
}

With this you also get type-errors when you try to access a field that doesn't exist in the index. Or an invalid index. And with that, also autocompletion for these fields.

const invalidIndex = typedEs(client, {
    index: "invalid-index", // Here we get a: Type '"invalid-index"' is not assignable to type '"first-index" | "second-index"'. 
    _source: ["score", "entity_id"],
});

See more examples in the test files.

Usage with asyncSearch

The asyncSearch API has some complexity for us. The get method does not include the original query type information by default. To work around that we've added a new type definition.

const query = typedEs(...);

const result = await client.asyncSearch.get<typeof query>({ id: "abc" });
const data = result.response; // Same type as if you used client.search(query);

// If you don't have a query variable, you can pass the query type explicitly.
const result = await client.asyncSearch.get<{ query: ...}>({ id: "abc" });

Usage with msearch

Run multiple searches in a single request with msearch. The top-level index is used as the default. You can override the index per-search using the header that precedes each body.

// Assuming `client` is a TypedClient<YourIndexes>
const result = await client.msearch({
	index: "first-index",
	searches: [
		// 1) Uses top-level `index`: "first-index"
		{},
		{ _source: ["id", "name"] },

		// 2) Override index for this search
		{ index: "second-index" },
		{ _source: ["title"] },
	],
});

const first = result.responses[0];
const doc1 = first.hits.hits[0]._source; // { id: number; name: string }

const second = result.responses[1];
const doc2 = second.hits.hits[0]._source; // { title: string }

Notes:

• Responses preserve per-search typing: each responses[i] matches the corresponding header/body pair.
• responses[i] can be an error object if that search failed.

What if the library is missing a feature that you need?

Please open an issue or a PR.

If it's a type error and is urgent, you can add the types manually as you'd do without the library.

const myBrokenQuery = typedEs(client, {
    index: "my-index",
    _source: ["score", "entity_id", "*ate"],
});

const result = await (client as unknown as Client).search<TDocument, TAggregations>(myBrokenQuery); // With the `as Client` cast you are now using the native types

Limitations

• query fields and aggs fields are not typed.
• Some agg functions might be missing.
• _source fields allow any string as you can use wildcards. On the other hand, wildcards will result in the correct type in the output.
• has to use as unknown as TypedClient<Indexes> which I don't like.
• Expect index to be a string. Currently we don't support wildcard or _all for the index name.

PRs are welcome to fix these limitations.

License

MIT