@infersec/conduit

Conduit agent for connecting local LLMs to the Infersec cloud.

Usage

npx @infersec/conduit inference start --engine <type> --key <api-key> --source <source-id>

Commands

| Command | Description | | ----------------------- | ------------------------------------------------------------------------------------------------------ | | inference start | Start the inference agent | | inference group start | Start an inference agent for a source group (registers an ephemeral source) | | engine create | Create or update an engine resource | | models create | Create or update a registered model resource | | models list | List cached models and disk usage | | models clear | Remove cached models | | models fit | Detect local hardware and suggest models that will run | | source create | Create or update an inference source resource | | endpoint create | Create or update an inference endpoint resource | | tool connect <toolID> | Connect a local tool (local-filesystem, local-mysql) to the API. Type is resolved from the server. |

Note: The resource management commands (engine create, models create, source create, endpoint create) and models fit have no environment variable fallbacks - all options are provided as flags. The inference and tool commands keep their env var equivalents.

Flags

| Flag | Required | Default | Notes | | --------------- | -------- | ------------------------------ | -------------------------------------------------------------------------------------- | | --engine | Yes | - | Engine type (matches ENGINE). | | --engine-port | No | 9700 | Engine port (matches ENGINE_PORT). | | --key | Yes | - | API key (matches API_KEY). | | --port | No | 9600 | Port to listen on (matches PORT). | | --root | No | $HOME/.cache/infersec/iagent | Root directory (matches ROOT_DIRECTORY). | | --source | Yes | - | Inference source ID (matches SOURCE). | | --api-url | No | https://api.infersec.ai | API base URL (matches API_URL). | | --start-mode | No | auto | Startup mode (matches START_MODE): auto starts engine, idle leaves conduit idle. |

inference group start flags

Same flags as inference start, except --group replaces --source:

| Flag | Required | Default | Notes | | --------- | -------- | ------- | ------------------------------------- | | --group | Yes | - | Source group ID (matches GROUP_ID). |

The agent registers an ephemeral source under the group (absorbing its model/engine settings), then runs the standard inference flow. Ephemeral sources are removed automatically after being offline for 1 hour.

Resource management commands

engine create, models create, source create and endpoint create manage Infersec resources via the public REST API. They all share these flags (no environment variable fallbacks):

| Flag | Required | Default | Notes | | ----------- | --------------- | ------- | ------------------------------------------------------------- | | --api-url | Yes | - | API base URL. | | --key | Yes | - | API key. | | --name | Yes | - | Resource name (used for --update matching). | | --update | No | false | Update an existing resource matched by --name, else create. | | --id | With --update | - | Target an existing resource by ID instead of name matching. |

Per-command flags:

| Command | Flags | | ----------------- | --------------------------------------------------------------------------------------------------------------------------- | | engine create | --type <llama.cpp\|vllm\|sglang\|tensorrt-llm\|mlx-lm\|exllamav3> (required), --arg <flag> (repeatable raw engine flag) | | models create | --slug <owner/repo> (required), --format <gguf\|safetensors\|...> (required) | | source create | --engine <id> (required), --model <id> (required), --quant <label>, --context-length <n> | | endpoint create | --source <id> (repeatable, at least one), --group <id> (repeatable), --routing-method, --enabled |

models fit

Detects local hardware (GPU compute, memory, engines including Docker), downloads a small probe model (LFM2.5-350M, served from the Infersec asset mirror - LFM Open License v1.0) and suggests models that fit. Prints a runnable command chain to deploy the top suggestion.

Backend resolution order: local llama.cpp (GGUF), local vLLM (safetensors), then Docker vLLM images when Docker is available. Without any backend, falls back to rule-based suggestions.

| Flag | Required | Default | Notes | | --------------- | -------------------- | ------------------------------ | ---------------------------------------------- | | --api-url | No | https://api.infersec.ai | API used to resolve probe model download URLs. | | --interactive | No | - | Agent session that deploys suggestions. | | --key | With --interactive | - | API key for the interactive session. | | --json | No | - | Machine-readable JSON output. | | --probe-model | No | LiquidAI/LFM2.5-350M | Probe model slug requested from the API. | | --root | No | $HOME/.cache/infersec/iagent | Root directory for probe model files. |

tool connect flags

Common flags (all tool types):

| Flag | Required | Default | Notes | | ----------- | -------- | ------------------------- | --------------------------------------- | | <toolID> | Yes | - | Tool ID to connect (matches TOOL_ID). | | --key | Yes | - | API key (matches API_KEY). | | --api-url | No | https://api.infersec.ai | API base URL (matches API_URL). |

local-filesystem flags:

| Flag | Required | Default | Notes | | ------------- | -------- | ----------------- | ----------------------------------------------------------------------- | | --path | No | . (current dir) | Filesystem base path (matches TOOL_PATH). Paths are confined to this. | | --read-only | No | false | Expose only read operations (matches READ_ONLY). |

local-mysql flags (MySQL/MariaDB). Defaults to read-only; use --allow to enable writes:

| Flag | Required | Default | Notes | | -------------------- | --------------- | -------- | ------------------------------------------------------------------------------------------------------ | | --url | One of url/host | - | Connection URL mysql://user:pass@host:3306/db (matches MYSQL_URL). Prefer MYSQL_URL for secrets. | | --host | One of url/host | - | Host (matches MYSQL_HOST). | | --port | No | 3306 | Port (matches MYSQL_PORT). | | --user | With host | - | User (matches MYSQL_USER). | | --password | With host | - | Password (matches MYSQL_PASSWORD). Prefer MYSQL_PASSWORD to avoid shell history exposure. | | --database | No | - | Database name (matches MYSQL_DATABASE). | | --allow | No | (none) | Write operations to enable (csv) or all (matches MYSQL_ALLOW). Read-only if unset. | | --pool-max | No | 5 | Pool connection limit (matches MYSQL_POOL_MAX). | | --max-rows | No | 1000 | Max rows returned per query (matches MYSQL_MAX_ROWS). | | --query-timeout-ms | No | 30000 | Per-query timeout in ms (matches MYSQL_QUERY_TIMEOUT_MS). |

Examples

| Scenario | Command | | ------------------ | --------------------------------------------------------------------------------------------------------------------------------- | | Required only | npx @infersec/conduit inference start --engine vllm --key <api-key> --source <source-id> | | Custom root/port | npx @infersec/conduit inference start --engine vllm --key <api-key> --source <source-id> --root /data/infersec --port 9601 | | Custom engine/port | npx @infersec/conduit inference start --engine vllm --key <api-key> --source <source-id> --port 9601 --engine-port 9701 | | Source group | npx @infersec/conduit inference group start --key <api-key> --group <group-id> | | Connect FS tool | npx @infersec/conduit tool connect <tool-id> --key <api-key> --path ./ | | Read-only FS tool | npx @infersec/conduit tool connect <tool-id> --key <api-key> --path ./ --read-only | | Connect MySQL tool | MYSQL_URL=mysql://user:pass@host:3306/db npx @infersec/conduit tool connect <tool-id> --key <api-key> | | MySQL + writes | MYSQL_URL=mysql://user:pass@host:3306/db npx @infersec/conduit tool connect <tool-id> --key <api-key> --allow insert,update,sql |

Credentials are best supplied via the MYSQL_* environment variables (or MYSQL_URL) rather than --password/--url flags, to avoid exposing them in process listings and shell history.

Environment variables

| Variable | Required | Default | Notes | | ---------------- | -------- | ------------------------------ | ---------------------------------------------------------------------------- | | AUTO_PORTS | No | false | Auto-allocate conduit and engine ports. Conflicts with PORT/ENGINE_PORT. | | ENGINE | Yes | - | Engine type (matches --engine). | | ENGINE_PORT | No | 9700 | Engine port (matches --engine-port). | | API_KEY | Yes | - | API key (matches --key). | | GROUP_ID | No | - | Source group ID for inference group start (matches --group). | | SOURCE | Yes | - | Inference source ID (matches --source). | | API_URL | No | https://api.infersec.ai | API base URL (matches --api-url). | | PORT | No | 9600 | Port to listen on (matches --port). | | ROOT_DIRECTORY | No | $HOME/.cache/infersec/iagent | Root directory (matches --root). | | START_MODE | No | auto | Startup mode (matches --start-mode). |

tool connect environment variables

Common:

| Variable | Required | Default | Notes | | --------- | -------- | ------------------------- | --------------------------------------------------- | | API_KEY | Yes | - | API key (matches --key). | | TOOL_ID | Yes | - | Tool ID to connect (matches <toolID> positional). | | API_URL | No | https://api.infersec.ai | API base URL (matches --api-url). |

local-filesystem:

| Variable | Required | Default | Notes | | ----------- | -------- | ------- | ---------------------------------------------------- | | TOOL_PATH | No | . | Filesystem base path (matches --path). | | READ_ONLY | No | false | Expose only read operations (matches --read-only). |

local-mysql:

| Variable | Required | Default | Notes | | ------------------------ | --------------- | -------- | ------------------------------------------------------- | | MYSQL_URL | One of url/host | - | Connection URL (matches --url). | | MYSQL_HOST | One of url/host | - | Host (matches --host). | | MYSQL_PORT | No | 3306 | Port (matches --port). | | MYSQL_USER | With host | - | User (matches --user). | | MYSQL_PASSWORD | With host | - | Password (matches --password). | | MYSQL_DATABASE | No | - | Database name (matches --database). | | MYSQL_ALLOW | No | (none) | Write ops csv / all (matches --allow). | | MYSQL_POOL_MAX | No | 5 | Pool connection limit (matches --pool-max). | | MYSQL_MAX_ROWS | No | 1000 | Max rows per query (matches --max-rows). | | MYSQL_QUERY_TIMEOUT_MS | No | 30000 | Per-query timeout in ms (matches --query-timeout-ms). |

CLI flags override environment variables when both are provided.