@smart-llm/gemini
v0.2.0
Published
smart-llm provider adapter for Gemini (Developer API and Vertex AI) through smart-genai
Readme
@smart-llm/gemini
Gemini adapter for smart-llm: connects the Gemini Developer API (AI Studio) and Vertex AI through smart-genai.
Install
bun add @smart-llm/gemini @smart-llm/core
# or
npm install @smart-llm/gemini @smart-llm/coreESM package with TypeScript types for Node.js >= 22.12 (require() supported) and Bun >= 1.3.
Usage
import { createLlm } from '@smart-llm/core';
import { createGeminiProvider } from '@smart-llm/gemini';
const llm = createLlm({
providers: {
gemini: {
adapter: createGeminiProvider(), // env GEMINI_API_KEY, or createGeminiProvider({ vertex: true })
models: {
flash: {
model: 'gemini-3.8-flash',
capabilities: {
images: true,
tools: true,
json: true,
webSearch: { supported: true },
reasoning: { supported: true, default: 'low' },
},
modelLimits: { rpm: 1_000, tpm: 1_000_000 },
},
vec: {
model: 'gemini-embedding-2',
capabilities: { embeddings: true },
modelLimits: { rpm: 100 },
},
},
},
},
});
const answer = await llm.flash('Summarize quantum tunneling in two sentences.', { web_search: true });
const vectors = await llm.embed({ model: 'vec', input: ['First paragraph', 'Second paragraph'], dimensions: 768 });Options
createGeminiProvider(clientOrOptions?: GenaiClient | GeminiOptions) accepts an existing smart-genai client or GeminiOptions (Omit<GenaiClientOptions, 'defaults'>). Passing defaults or setting both apiKey and vertex throws TypeError; set request defaults in createLlm. A passed client created with defaults keeps applying them.
| Option | Type | Default | Description |
|---|---|---|---|
| apiKey | string | env GEMINI_API_KEY | Developer API key (GEMINI_API_KEY_ENV); Developer API only |
| vertex | boolean | GenaiVertexOptions | undefined | true or { project?, location?, bucket?, credentialsFile?, authClient?, storageUrl? } for Vertex AI |
| baseUrl | string | official endpoint | Custom API endpoint URL |
| env | Readonly<Record<string, string \| undefined>> | process.env | Environment map for the API key and the Vertex AI project, location, bucket, and credentials file |
| timeoutMs | number | 120_000 | HTTP timeout per call in milliseconds (2 minutes) |
| batchWaitSec | number | 10 | Collection window in seconds for batch calls (1 to 600) |
Vertex AI also reads GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION (default 'global'), VERTEX_BATCH_BUCKET, and GOOGLE_APPLICATION_CREDENTIALS (else gcloud ADC). Missing or invalid credentials throw TypeError.
Extra Options
extra (provider, model, or call layer) accepts only GeminiExtra:
| Key | Type | Description |
|---|---|---|
| config | GenerateContentConfig | Gemini request configuration passed to smart-genai |
- Merge:
configmerges across provider, model, and call layers per top-level field (later layers win; nested objects such astoolConfigare replaced whole).thinkingConfigmerges per member;thinkingBudgetandthinkingLevelcount as one setting, so the last layer that sets either replaces both from earlier layers. - Validation: other
extrakeys, a non-objectconfig, own__proto__keys, aconfig.toolsthat is neither an array nornull, andabortSignal/httpOptions/automaticContinuationinsideconfigare rejected (TypeErrorat setup,invalid_requestper call).
Capabilities
GEMINI_SUPPORT: { images: true, tools: true, webSearch: true, reasoning: true, batch: true, json: 'native', embeddings: true }.
- Messages:
stoptakes up to 5 sequences andseedmust be a 32-bit signed integer (invalid_requestotherwise). - Images: base64 data URLs;
detailis not sent. - Tools: missing call IDs become
call_plus 32 hex characters. Thought signatures are kept intoolCall.extra_content.google.thought_signatureand replayed on the next turn (assistantMessagekeeps them).stricton tool schemas is not sent;tool_choiceis stripped by@smart-llm/corewhentoolsis omitted (even ifextra.config.toolsdeclares functions).toolsandweb_search: trueappend afterextra.config.toolsand fail withinvalid_requestongemini-2.*models. - JSON (
native): sends the cleaned JSON Schema asresponseJsonSchema. On Vertex AI batch,requiredis stripped andresponseSchemaonly supports the typed-schema subset (unsupported keywords fail withinvalid_request).@smart-llm/corevalidates the parsed answer. - Reasoning: maps
reasoning_effort(none,minimal,low,medium,high,xhigh,max):gemini-2.*models:thinkingBudget(0fornone; at least128ongemini-2.5-pro)- Gemma models:
thinkingLevelMINIMALfornonetolow,HIGHformediumtomax - other models:
thinkingLevel;noneandminimalsendMINIMALon Gemini 3.0 to 3.6 Flash and Flash-Lite andLOWon others;xhighandmaxsendHIGH - thought summaries return in
data.reasoning:includeThoughts: trueis sent wheneverreasoning_effortis set, except whennonesendsthinkingBudget: 0; turn off withextra: { config: { thinkingConfig: { includeThoughts: false } } } - assistant
reasoningin the history is not sent
- Web search:
web_search: trueenables Google Search grounding; citations return indata.sources({ url, title? }). - Batch:
batch: truegroups calls that arrive withinbatchWaitSecand sharemodel,config,response_format, andtoolsinto one Gemini Batch API job; Vertex AI needsvertex.bucketor envVERTEX_BATCH_BUCKET. Jobs are polled every 30 s for up to 72 hours, then fail with non-retryabletimeout. A group above the payload limit is split into several jobs; a single request above it fails withinvalid_request. - Embeddings:
task('query','document','similarity','classification','clustering') maps to Gemini task types; per-inputtitlerequirestask: 'document';dimensionsis supported.usage.inputTokensis alwaysnull.- up to
100texts per call (maxEmbedInputs: 100) on the Developer API, and on Vertex AI forgemini-embedding-001and non-Gemini (non-MaaS) models - Vertex AI with other Gemini models (including
gemini-embedding-2) and MaaS models: more than1text per call returnsunsupported; setcapabilities: { embeddings: { supported: true, maxInputs: 1 } }sollm.embedsends one text per call - models other than
gemini-embedding-2take onetitleper call; different titles fail withinvalid_request
- up to
- Model listing:
llm.listModels('gemini')returns{ id, displayName?, inputTokenLimit?, outputTokenLimit? }[](Vertex AI publisher models populateidonly).
Errors
| Code | Retryable | When |
|---|---|---|
| rate_limited | yes | HTTP 429 (retry-after honored) |
| http | yes / no | HTTP errors (status and retryAfterMs preserved) |
| network | yes | Connection failure |
| timeout | yes | Request timed out |
| timeout | no | Batch job not finished after 72 hours |
| aborted | no | Caller signal aborted |
| blocked / truncated | no | Safety block / token limit; carries error.output |
| invalid_request | no | Invalid extra.config, request limits under Capabilities, tools with web_search: true on gemini-2.*, oversized batch request, or unsupported Vertex batch schema keywords |
| unsupported | no | Multi-text embed on Vertex AI for models limited to one text, or batch: true on Vertex AI without vertex.bucket / VERTEX_BATCH_BUCKET |
| unexpected | yes | Malformed function call (MALFORMED_FUNCTION_CALL) |
| unexpected | no | Missing candidate, failed batch job, other tool finish errors, or non-TypeError exceptions |
An invalid_json answer is parsed and validated by @smart-llm/core (or forwarded as invalid_json if no output was returned).
Exports
| Export | Kind | Description |
|---|---|---|
| createGeminiProvider | function | Creates the adapter |
| GEMINI_SUPPORT | constant | LlmProviderSupport of the adapter |
| GEMINI_API_KEY_ENV | constant | 'GEMINI_API_KEY' (re-exported from smart-genai) |
| GeminiOptions | type | Options of createGeminiProvider |
| GeminiExtra | type | extra accepted by the adapter |
Configuration, requests, rate limits, and error handling of @smart-llm/core are documented in the smart-llm README.
License
Apache-2.0, see LICENSE.
