✨ ComfyUI SDK ✨

A robust and meticulously crafted TypeScript SDK 🚀 for seamless interaction with the ComfyUI API. This SDK significantly simplifies the complexities of building, executing, and managing ComfyUI workflows, all while providing real-time updates and supporting multiple instances. 🖼️

🧭 Table of Contents

• 🌟 Key Features 🌟
• 📦 Installation 📦
• 🚀 Getting Started 🚀
  • 🎬 Basic Usage
  • 🔄 Managing Multiple Instances with ComfyPool
  • 🔑 Authentication
• 📚 API Reference 📚
  • ComfyApi
  • CallWrapper
  • PromptBuilder
  • ComfyPool
  • 🗂️ Enums
  • 🗄️ Types
  • 🧩 Features
• 📂 Examples
• 🤝 Contributing
• 📜 License

🌟 Key Features 🌟

• 💎 TypeScript Powered: Enjoy a fully typed codebase, ensuring enhanced development, maintainability, and type safety. 🛡️
• 🏗️ Workflow Builder: Construct and manipulate intricate ComfyUI workflows effortlessly using a fluent, intuitive builder pattern. 🧩
• 🤹 Multi-Instance Management: Handle a pool of ComfyUI instances with ease, employing flexible queueing strategies for optimal resource utilization. 🌐
• ⚡ Real-Time Updates: Subscribe to WebSocket events for live progress tracking, image previews, and error notifications. 🔔
• 🔧 Custom WebSocket Support: Supply your own WebSocket implementation for greater flexibility in different environments, with robust reconnection handling and fallback options. 🔄
• 🔑 Authentication Flexibility: Supports Basic Auth, Bearer Token, and Custom Authentication Headers, catering to diverse security requirements. 🔒
• 🔌 Extension Support: Seamlessly integrate with ComfyUI Manager and leverage system monitoring through the ComfyUI-Crystools extension. 🛠️
• 🔀 Flexible Node Bypassing: Strategically bypass specific nodes in your workflows during generation, enabling advanced customization. ⏭️
• 📚 Comprehensive Examples: Includes practical examples for Text-to-Image (T2I), Image-to-Image (I2I), and complex multi-node workflows. 📝
• 🚨 Robust Error Handling: Provides detailed error messages to facilitate debugging and graceful handling of API failures. 🐛
• 📝 Automatic Changelog: Automatically generates a changelog with each release, utilizing auto-changelog for transparent version tracking. 🔄

📦 Installation 📦

bun add @saintno/comfyui-sdk

or

npm i @saintno/comfyui-sdk

🚀 Getting Started 🚀

🎬 Basic Usage

Here's a simplified example to quickly get you started:

import { ComfyApi, CallWrapper, PromptBuilder, TSamplerName, TSchedulerName, seed } from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";

const api = new ComfyApi("http://localhost:8189").init();
const workflow = new PromptBuilder(
  ExampleTxt2ImgWorkflow,
  ["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
  ["images"]
)
  .setInputNode("checkpoint", "4.inputs.ckpt_name")
  .setInputNode("seed", "3.inputs.seed")
  .setInputNode("batch", "5.inputs.batch_size")
  .setInputNode("negative", "7.inputs.text")
  .setInputNode("positive", "6.inputs.text")
  .setInputNode("cfg", "3.inputs.cfg")
  .setInputNode("sampler", "3.inputs.sampler_name")
  .setInputNode("sheduler", "3.inputs.scheduler")
  .setInputNode("step", "3.inputs.steps")
  .setInputNode("width", "5.inputs.width")
  .setInputNode("height", "5.inputs.height")
  .setOutputNode("images", "9")
  .input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
  .input("seed", seed())
  .input("step", 6)
  .input("cfg", 1)
  .input<TSamplerName>("sampler", "dpmpp_2m_sde_gpu")
  .input<TSchedulerName>("sheduler", "sgm_uniform")
  .input("width", 1024)
  .input("height", 1024)
  .input("batch", 1)
  .input("positive", "A picture of cute dog on the street");

new CallWrapper(api, workflow)
  .onFinished((data) => console.log(data.images?.images.map((img: any) => api.getPathImage(img))))
  .run();

🔍 Breakdown

• Import essential components from the SDK.
• Create and initialize the ComfyApi instance.
• Use PromptBuilder to define the workflow structure and set input nodes.
• Set specific input values, including the checkpoint path, seed, and prompt.
• Execute the workflow and log the generated image URLs using the CallWrapper.

🔄 Managing Multiple Instances with ComfyPool

import {
  ComfyApi,
  CallWrapper,
  ComfyPool,
  EQueueMode,
  PromptBuilder,
  seed,
  TSamplerName,
  TSchedulerName
} from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";

const ApiPool = new ComfyPool(
  [new ComfyApi("http://localhost:8188"), new ComfyApi("http://localhost:8189")],
  EQueueMode.PICK_ZERO
)
  .on("init", () => console.log("Pool in initializing"))
  .on("add_job", (ev) => console.log("Job added at index", ev.detail.jobIdx, "weight:", ev.detail.weight))
  .on("added", (ev) => console.log("Client added", ev.detail.clientIdx));

const generateFn = async (api: ComfyApi, clientIdx?: number) => {
  const workflow = new PromptBuilder(
    ExampleTxt2ImgWorkflow,
    ["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
    ["images"]
  )
    .setInputNode("checkpoint", "4.inputs.ckpt_name")
    .setInputNode("seed", "3.inputs.seed")
    .setInputNode("batch", "5.inputs.batch_size")
    .setInputNode("negative", "7.inputs.text")
    .setInputNode("positive", "6.inputs.text")
    .setInputNode("step", "3.inputs.steps")
    .setInputNode("width", "5.inputs.width")
    .setInputNode("height", "5.inputs.height")
    .setInputNode("cfg", "3.inputs.cfg")
    .setInputNode("sampler", "3.inputs.sampler_name")
    .setInputNode("scheduler", "3.inputs.scheduler")
    .setOutputNode("images", "9")
    .input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
    .input("seed", seed())
    .input("step", 6)
    .input("width", 512)
    .input("height", 512)
    .input("batch", 2)
    .input("cfg", 1)
    .input<TSamplerName>("sampler", "dpmpp_2m_sde_gpu")
    .input<TSchedulerName>("scheduler", "sgm_uniform")
    .input("positive", "A close up picture of cute Cat")
    .input("negative", "text, blurry, bad picture, nsfw");

  return new Promise<string[]>((resolve) => {
    new CallWrapper(api, workflow)
      .onFinished((data) => {
        const url = data.images?.images.map((img: any) => api.getPathImage(img));
        resolve(url as string[]);
      })
      .run();
  });
};

const jobA = ApiPool.batch(Array(5).fill(generateFn), 10).then((res) => {
  console.log("Batch A done");
  return res.flat();
});

const jobB = ApiPool.batch(Array(5).fill(generateFn), 0).then((res) => {
  console.log("Batch B done");
  return res.flat();
});

console.log(await Promise.all([jobA, jobB]).then((res) => res.flat()));

🔍 Breakdown

• Create a ComfyPool with multiple ComfyApi instances.
• Set up event listeners for pool initialization, job additions, and client connections.
• Define an async function (generateFn) that creates a workflow, sets its inputs, and executes it with a CallWrapper.
• Use ApiPool.batch to run multiple jobs and wait for all batches to complete.

🔑 Authentication

import { ComfyApi, BasicCredentials, BearerTokenCredentials, CustomCredentials } from "@saintno/comfyui-sdk";

// Basic Authentication
const basicAuth = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "basic", username: "username", password: "password" } as BasicCredentials
}).init();

// Bearer Token Authentication
const bearerAuth = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "bearer_token", token: "your_bearer_token" } as BearerTokenCredentials
}).init();

// Custom Header Authentication
const customAuth = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "custom", headers: { "X-Custom-Header": "your_custom_header" } } as CustomCredentials
}).init();

🔍 Breakdown

• Import the necessary types from the SDK.
• Create ComfyApi instances using the corresponding credential types: BasicCredentials, BearerTokenCredentials, and CustomCredentials..

🔌 Custom WebSocket Implementation

import { ComfyApi, WebSocketInterface } from "@saintno/comfyui-sdk";
import CustomWebSocket from "your-custom-websocket-library";

// Create a ComfyApi instance with a custom WebSocket implementation
const api = new ComfyApi("http://localhost:8189", "node-id", {
  credentials: { type: "basic", username: "username", password: "password" },
  customWebSocketImpl: CustomWebSocket as WebSocketInterface
}).init();

🔍 Breakdown

• Import the necessary types and your custom WebSocket implementation.
• Pass the custom WebSocket implementation to the ComfyApi constructor using the customWebSocketImpl option.
• The SDK will use your implementation instead of the default one, allowing for greater flexibility in environments where the standard WebSocket implementation might not be available or suitable.

📝 Benefits

• Environment Flexibility: Run in environments where the standard WebSocket might not be available or optimal.
• Enhanced Stability: The SDK includes robust reconnection logic with exponential backoff and jitter to handle connection issues gracefully.
• Fallback Mechanism: Automatically falls back to HTTP polling if WebSocket connections fail, ensuring your application remains functional.
• Custom Protocol Support: Implement custom protocols or security features through your WebSocket implementation.

📚 API Reference 📚

ComfyApi

🏗️ Constructor

constructor(host: string, clientId: string, opts?: { forceWs?: boolean, wsTimeout?: number, credentials?: BasicCredentials | BearerTokenCredentials | CustomCredentials; })

• host: The base URL of your ComfyUI server.
• clientId: A unique ID for WebSocket communication (optional). Defaults to a generated ID.
• opts: Optional settings:
  • forceWs: Boolean to force WebSocket usage.
  • wsTimeout: Timeout for WebSocket connections (milliseconds).
  • credentials: Optional authentication credentials.

⚙️ Methods

• init(maxTries?: number, delayTime?: number): Initializes the client and establishes connection.
• on<K extends keyof TComfyAPIEventMap>(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: AddEventListenerOptions | boolean): Attach an event listener.
• off<K extends keyof TComfyAPIEventMap>(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: EventListenerOptions | boolean): Detach an event listener.
• removeAllListeners(): Detach all event listeners.
• fetchApi(route: string, options?: FetchOptions): Fetch data from the API endpoint.
• pollStatus(timeout?: number): Polls the ComfyUI server status.
• queuePrompt(number: number | null, workflow: object): Queues a prompt for processing.
• appendPrompt(workflow: object): Adds a prompt to the workflow queue.
• getQueue(): Retrieves the current state of the queue.
• getHistories(maxItems?: number): Retrieves the prompt execution history.
• getHistory(promptId: string): Retrieves a specific history entry by ID.
• getSystemStats(): Retrieves system and device statistics.
• getExtensions(): Retrieves a list of installed extensions.
• getEmbeddings(): Retrieves a list of available embeddings.
• getCheckpoints(): Retrieves a list of available checkpoints.
• getLoras(): Retrieves a list of available Loras.
• getSamplerInfo(): Retrieves sampler and scheduler information.
• getNodeDefs(nodeName?: string): Retrieves node object definitions.
• getUserConfig(): Get user configuration data.
• createUser(username: string): Create new user.
• getSettings(): Get all setting values for the current user.
• getSetting(id: string): Get a specific setting for the current user.
• storeSettings(settings: Record<string, unknown>): Store setting for the current user.
• storeSetting(id: string, value: unknown): Store a specific setting for the current user.
• uploadImage(file: Buffer | Blob, fileName: string, config?: { override?: boolean; subfolder?: string }): Uploads an image file.
• uploadMask(file: Buffer | Blob, originalRef: ImageInfo): Uploads a mask file.
• freeMemory(unloadModels: boolean, freeMemory: boolean): Frees memory by unloading models.
• getPathImage(imageInfo: ImageInfo): Returns the path to an image.
• getImage(imageInfo: ImageInfo): Returns the blob data of image.
• getUserData(file: string): Get a user data file.
• storeUserData(file: string, data: unknown, options?: RequestInit & { overwrite?: boolean, stringify?: boolean, throwOnError?: boolean }): Store a user data file.
• deleteUserData(file: string): Delete a user data file.
• moveUserData(source: string, dest: string, options?: RequestInit & { overwrite?: boolean }): Move a user data file.
• listUserData(dir: string, recurse?: boolean, split?: boolean): List a user data file.
• interrupt(): Interrupts the execution of the running prompt.
• reconnectWs(opened?: boolean): Reconnects to the WebSocket server.

CallWrapper

🏗️ Constructor

constructor(client: ComfyApi, workflow: PromptBuilder<I, O, T>)

• client: An instance of the ComfyApi client.
• workflow: An instance of PromptBuilder defining the workflow.

⚙️ Methods

• onPreview(fn: (ev: Blob, promptId?: string) => void): Set callback for preview events.
• onPending(fn: (promptId?: string) => void): Set callback when job is queued.
• onStart(fn: (promptId?: string) => void): Set callback when the job is started.
• onOutput(fn: (key: keyof PromptBuilder<I, O, T>["mapOutputKeys"], data: any, promptId?: string) => void): Sets a callback for when an output node is executed.
• onFinished(fn: (data: Record<keyof PromptBuilder<I, O, T>["mapOutputKeys"], any>, promptId?: string) => void): Set callback when the job is finished.
• onFailed(fn: (err: Error, promptId?: string) => void): Set callback when the job failed.
• onProgress(fn: (info: NodeProgress, promptId?: string) => void): Set callback for progress updates.
• run(): Executes the workflow.

PromptBuilder

🏗️ Constructor

constructor(prompt: T, inputKeys: I[], outputKeys: O[])

• prompt: The initial workflow data object.
• inputKeys: An array of input node keys.
• outputKeys: An array of output node keys.

⚙️ Methods

• clone(): Creates a new PromptBuilder instance with the same configuration.
• bypass(node: keyof T | (keyof T)[]): PromptBuilder<I, O, T>: Marks node(s) to be bypassed at generation.
• reinstate(node: keyof T | (keyof T)[]): PromptBuilder<I, O, T>: Unmarks node(s) from bypass at generation.
• setInputNode(input: I, key: DeepKeys<T> | Array<DeepKeys<T>>): Sets input node path for a key.
• setRawInputNode(input: I, key: string | string[]): Sets raw input node path for a key.
• appendInputNode(input: I, key: DeepKeys<T> | Array<DeepKeys<T>>): Appends a node to the input node path.
• appendRawInputNode(input: I, key: string | string[]): Appends a node to the raw input node path.
• setOutputNode(output: O, key: DeepKeys<T>): Sets output node path for a key.
• setRawOutputNode(output: O, key: string): Sets raw output node path for a key.
• input<V = string | number | undefined>(key: I, value: V, encodeOs?: OSType): Sets an input value.
• inputRaw<V = string | number | undefined>(key: string, value: V, encodeOs?: OSType): Sets a raw input value with dynamic key.
• get workflow: Retrieves the workflow object.
• get caller: Retrieves current PromptBuilder object.

ComfyPool

🏗️ Constructor

constructor(clients: ComfyApi[], mode: EQueueMode = EQueueMode.PICK_ZERO)

• clients: Array of ComfyApi instances.
• mode: The queue mode using EQueueMode enum values.

⚙️ Methods

• on<K extends keyof TComfyPoolEventMap>(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: AddEventListenerOptions | boolean): Attach an event listener.
• off<K extends keyof TComfyPoolEventMap>(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: EventListenerOptions | boolean): Detach an event listener.
• addClient(client: ComfyApi): Adds a new client to the pool.
• removeClient(client: ComfyApi): Removes a client from the pool.
• removeClientByIndex(index: number): Removes a client by index.
• changeMode(mode: EQueueMode): Changes the queue mode.
• pick(idx?: number): Picks a client by index.
• pickById(id: string): Picks a client by ID.
• run<T>(job: (client: ComfyApi, clientIdx?: number) => Promise<T>, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] }): Run a job with priority on an available client.
• batch<T>(jobs: Array<(client: ComfyApi, clientIdx?: number) => Promise<T>>, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] }): Run multiple jobs concurrently.

🗂️ Enums

• EQueueMode:
  • PICK_ZERO: Selects the client with zero remaining queue.
  • PICK_LOWEST: Selects the client with the lowest remaining queue.
  • PICK_ROUTINE: Selects clients in a round-robin manner.

🗄️ Types

• OSType:
  • POSIX: For Unix-like systems.
  • NT: For Windows systems.
  • JAVA: For Java virtual machine.
• TSamplerName: A union type of all available sampler names.
• TSchedulerName: A union type of all available scheduler names.

🧩 Features

• ManagerFeature: Provides methods to manage ComfyUI Manager Extension.

  const api = new ComfyApi("http://localhost:8189").init();
  await api.waitForReady();
  
  if (api.ext.manager.isSupported) {
    await api.ext.manager.getExtensionList().then(console.log);
    // Check api.ext.manager for more methods
  }

• MonitoringFeature: Provides methods to monitor system resources using ComfyUI-Crystools Extension.

  const api = new ComfyApi("http://localhost:8189").init();
  await api.waitForReady();
  
  if (api.ext.monitor.isSupported) {
    // For subscribing to system monitor events
    api.ext.monitor.on("system_monitor", (ev) => {
      console.log(ev.detail);
    });
  
    // For getting current monitor data
    console.log(api.ext.monitor.monitorData);
  }

Note: Features require respective extensions (ComfyUI-Manager and ComfyUI-Crystools) to be installed.

📂 Examples

The examples directory contains practical demonstrations of SDK usage:

• example-i2i.ts: Demonstrates image-to-image generation.
• example-pool.ts: Demonstrates how to manage multiple ComfyUI instances using ComfyPool.
• example-pool-basic-auth.ts: Demonstrates how to use ComfyPool with HTTP Basic Authentication.
• example-t2i.ts: Demonstrates text-to-image generation.
• example-t2i-upscaled.ts: Demonstrates text-to-image generation with upscaling.
• example-img2img-workflow.json: Example workflow for image-to-image.
• example-txt2img-workflow.json: Example workflow for text-to-image.
• example-txt2img-upscaled-workflow.json: Example workflow for text-to-image with upscaling.

🤝 Contributing

Contributions are always welcome! Feel free to submit pull requests or create issues for bug reports and feature enhancements. 🙏

📜 License

This project is licensed under the MIT License - see the LICENSE file for more details. 📄