@compeso/node-red-contrib-imap-email

Node-RED nodes for externally triggered IMAP email processing with bounded cursor-window fetch and at-least-once ACK handling.

Nodes

The package registers these Node-RED types:

Flow type           Palette label        Purpose
imap-email account  imap email account   shared IMAP account configuration
imap-email in       imap email in        externally triggered bounded cursor-window fetch
imap-email ack      imap email ack       batched acknowledgement and UID actions

Requirements

• Node.js >=22.0.0
• Node-RED >=4.0.0

Node-RED 5.x can require a stricter Node.js patch level than this package declares. Check the Node-RED runtime requirement when upgrading Node-RED itself.

Installation

From the npm registry:

cd ~/.node-red
npm install @compeso/node-red-contrib-imap-email

From GitHub during development:

cd ~/.node-red
npm install github:Harpau/node-red-contrib-imap-email

From a local checkout:

cd /path/to/node-red-contrib-imap-email
npm install
npm link

cd ~/.node-red
npm link @compeso/node-red-contrib-imap-email

Restart Node-RED after installation.

For development or release testing, pack the checkout and install the resulting tarball in an isolated Node-RED test instance. See local test instructions.

Example Flow

Import examples/basic-at-least-once-flow.json in Node-RED, open the imap email account config node, and enter your IMAP username and password. The example tab is disabled by default, the Inject node does not run automatically, and the ACK path only marks messages as seen. The visible palette labels use spaces; the stored Flow-JSON types use the imap-email ... prefix.

Minimal flow:

Inject / scheduler / HTTP trigger
  -> imap-email in
      -> your successful processing
          -> imap-email ack

Only wire messages to imap-email ack after all processing that must succeed has actually succeeded.

imap-email ack can be configured multiple times in one flow. Typical modes:

delete       delete the mail by UID and complete it; requires IMAP UIDPLUS
move         optionally update flags, move the mail and complete it; requires native IMAP MOVE
copy         copy the mail to a target folder, then optionally update source flags
flag         set or clear flags, keep the mail and complete it
set by msg.imap.ackAction
             read action, target folder and flags from the message

The ack node can set or clear \Seen, \Answered, \Flagged, other IMAP system flags such as \Draft, and custom keywords such as $Processed in flag, move and copy mode. With move, flags are updated on the source message before it is moved. With copy, the message is copied first and the configured ACK flag changes are then applied only to the source message. They are not applied to the target copy, though the IMAP server may copy flags that already existed on the source message. Use delete to delete mail; setting \Deleted as a raw flag is an advanced flag operation and does not replace the delete action.

Connection Check on Start

Available since version 1.1.0.

Each active imap email in and imap email ack node automatically checks its account when it starts, without needing an input message. This includes restart, full Deploy and any partial Deploy that restarts the node, including changes to its account, credentials or other settings. Nodes that continue running are not checked again. With Modified Flows, unchanged nodes in a restarted flow are also checked. Unused accounts and accounts used only by disabled nodes or flows create no check connection.

Nodes using the same account instance share a check while it is running. The check uses the account's current connection settings and stored credentials; a static access token takes precedence over a password. The connection is short-lived and closes after the check. A later node start requests a fresh check; there is no periodic retry or continuous monitoring.

The following status appears on the input or ACK node:

| Status | Meaning | | --- | --- | | Yellow ring: checking connection | The startup check is running. | | Green dot: connected | The last startup check succeeded; this is not a persistent connection. | | Red ring: missing account, missing host, missing username, missing credentials | Complete the account configuration. | | Red ring: authentication failed | The server rejected authentication. | | Red ring: host not found, connection refused, connection lost | The server could not be reached or the connection was lost. | | Red ring: TLS certificate error, TLS connection error | Certificate verification or the TLS connection failed. | | Red ring: connection timeout, connection failed | A time limit or another connection error ended the check. |

The whole check has a fixed 30-second limit, including connection setup, authentication and cleanup. Shorter account timeouts still apply. This limit does not change the timeouts of normal fetch or ACK connections.

Success confirms a server-accepted authenticated session. A server that sends IMAP PREAUTH has already authenticated that session and does not challenge the configured password or token again. The check does not read messages, change flags or test access to a mailbox or permission to perform an ACK action.

Deploy and normal processing can continue while the check runs. A failed check does not block later fetch or ACK attempts. Normal processing statuses and ACK configuration errors take priority over the check result. Closing or redeploying a node cancels its pending result. The check emits no output or stats messages; Node-RED Status nodes can observe its status changes. Check failures produce at most one warning per shared check using fixed error categories and safe technical codes. Deliberate cancellation does not produce a warning.

Large Mailboxes

imap-email in is designed for mailboxes that may contain many messages. It does not run an unbounded mailbox-wide search. Instead, each trigger reads one or more bounded windows and emits at most the configured batch size. Each window is streamed and discarded before another window is read; only selected candidate UIDs up to the remaining batch/inflight capacity are kept in memory.

Important settings:

Batch size       maximum messages emitted per trigger
Front window     maximum messages inspected per bounded window
Max inflight     maximum emitted but not-yet-ACKed messages tracked in memory
Retry after ms   time after which an un-ACKed message may be emitted again
Scan time ms     initial cursor-window soft time budget; 0 means exactly one cursor window
UIDs/command     maximum UID count per IMAP command chunk
Max bytes        maximum RFC822 bytes per message, 0 means unlimited
Chunk bytes      streamed IMAP download chunk size

Selection settings:

Deleted   Any | Only with flag | Only without flag
Seen      Any | Only with flag | Only without flag
Answered  Any | Only with flag | Only without flag
Flagged   Any | Only with flag | Only without flag

The defaults are Deleted = Only without flag and all other flags set to Any. These filters are applied only inside bounded windows. A selective filter may emit fewer messages than Batch size; it never causes a full-mailbox scan to fill the batch. When the cursor reaches the end of the mailbox, it wraps back to the first sequence number. The cursor is volatile and resets when Node-RED restarts or when IMAP UIDVALIDITY changes.

The node always uses an adaptive scan strategy. After restart or UIDVALIDITY reset it starts in the cursor-window phase with full Front window sized sequence windows. Empty windows are discarded and the node updates its status after every read window. The phase stops when the batch/capacity is filled, the mailbox end is reached, or Scan time ms expires; 0 means only one cursor window. If a window contains more selectable messages than the remaining batch/capacity can hold, the scan cursor is kept on that window so a later trigger can continue draining it instead of leaving messages behind. This is especially useful when the ACK node deletes or moves processed mails: after those mails leave the mailbox, remaining messages shift into the held sequence window and can be drained without being skipped. An empty mailbox with a valid UIDNEXT is treated as already at the mailbox end.

Once the mailbox end has been reached without candidate overflow, the node records the current UIDNEXT. Later triggers enter the new-uid-priority phase: they first read newly arrived UIDs up to a per-trigger UIDNEXT snapshot, then read one cyclic backlog window if capacity remains. In this phase the new-UID and backlog windows each use about half of Front window, with the larger half assigned to new UIDs. The logical new-UID window is still bounded by Front window, and its UID fetches are additionally split into commands of at most UIDs/command UIDs. New UIDs are emitted before backlog messages and in ascending UID order, but the node does not guarantee globally oldest unread delivery across the whole mailbox. Backlog windows are also held on candidate overflow. If the new-UID window already covers all messages that currently exist in the mailbox, the redundant backlog window is skipped. New-UID windows advance to the first UID that did not fit into the current batch, or to the next UID after the last actually read command chunk.

Stats report phase as the current operating phase (cursor-window or new-uid-priority). windowPhasesRead contains the ordered unique window types read during the trigger (cursor, new-uid, backlog); the last entry is the last window type read.

Output messages include the server flags as an array:

msg.imap.flags // for example ["\\Seen", "\\Flagged"]
msg.imap.flagState // { deleted: false, seen: true, answered: false, flagged: true }

Parsed mail headers are emitted in msg.email.header as a JSON-serializable object without an Object prototype. Use Object.hasOwn(msg.email.header, key) instead of msg.email.header.hasOwnProperty(key) when checking header presence. Headers with prototype-sensitive names such as __proto__, constructor or prototype are preserved under neutralized names.

Message bodies are downloaded as streams after the bounded front window has selected candidate UIDs. Attachments are drained without buffering unless Attachments is enabled. Raw source intentionally buffers the full RFC822 message in msg.raw; keep it disabled for very large messages. Set Max bytes to a positive value to reject oversized messages on output 2 with msg.imap.ackToken instead of parsing them.

Delivery Semantics

The package provides at-least-once delivery.

ACK action succeeded = successfully processed and ACKed
ACK action failed    = not successfully ACKed
Duplicate delivery   = possible
Exactly once         = not guaranteed

The inflight registry is volatile process memory. If Node-RED restarts after a message was emitted but before it was ACKed, the message remains in the mailbox and may be emitted again.

Completion guards for successfully ACKed inflight generations are also kept only in process memory and are bounded by a per-queue TTL and hard cap. After that bounded best-effort window, an extremely old fetch generation may be eligible for re-marking again. This keeps memory use bounded and preserves the package's at-least-once model; it is not an exactly-once guarantee.

A reported IMAP action failure is not acknowledged as success. In that case output 2 receives the original message with msg.imapAck.ok = false, and the inflight entry remains available for a later retry.

To avoid unsafe IMAP fallback behavior, delete requires server support for UIDPLUS and move requires native MOVE. Without those capabilities the ACK action fails closed on output 2. copy keeps the source message, copies it to the target mailbox first, and then applies any configured flag changes to the source message only.

Since version 1.1.0, ACK delete and input Expunge window explicitly confirm setting \Deleted, require a successful delete result and then search only the UIDs in the same bounded chunk to confirm that none remain. Only a successful search with an empty UID result confirms removal. The connection, selected mailbox path and UIDVALIDITY must stay valid throughout. This guards against a historical ImapFlow false-success case; see the DELETE history and fix. It adds no mailbox-wide scan. The safeguard adds two commands per deletion chunk: a checked STORE and a UID-constrained SEARCH without message content. Existing UIDs/command limits still apply; the query uses neither ALL nor a wildcard UID range.

A rejected initial flag update stops before EXPUNGE. Once that update succeeds, any later failure is partial: flags or deletions may already have taken effect, and no rollback is attempted. ACK retains inflight for the affected messages and stops later chunks in that group. Input cleanup aborts the current fetch cycle on a partial result or connection failure; it does not count unconfirmed UIDs as expunged or remove them from the registry. Earlier confirmed chunks remain applied. A later input trigger can attempt cleanup again.

ACK tokens are opaque signed bearer capabilities scoped to the configured IMAP account and to the current in-memory inflight generation. Pass msg.imap.ackToken from imap-email in to imap-email ack unchanged. Do not build, edit, log or expose tokens externally. If a token is missing required fields, is unsigned, has been modified, has already completed, or no longer matches the current inflight generation, the ACK node rejects it on output 2 without creating an IMAP client. The internal queueKey value is not a stable public API.

For dynamic decisions, configure imap-email ack to set by msg.imap.ackAction and set msg.imap.ackAction:

msg.imap.ackAction = {
  action: "move",               // delete, move, copy, flag
  targetMailbox: "Archive/Processed",
  flags: {
    seen: "set",                // ignore, set, clear
    answered: "ignore",
    flagged: "clear",
    add: ["$Processed"],
    remove: ["\\Draft"]
  }
};

Successful completions add msg.imapAck with fields such as action, disposition, mailbox, targetMailbox, uid, uidValidity, flags, range and completed. Failed completions may include msg.imapAck.partial = true when a state-changing IMAP step already succeeded before a later step failed. For delete, this includes failure after confirmed setting of \Deleted, even when final removal could not be confirmed. Retaining inflight cannot restore removed messages; remaining \Deleted messages are excluded by the default input filter. For copy, a successful copy followed by a failed source flag update is partial; because Inflight is kept for retry, a retry may create another copy in the target mailbox.

Current Limits

• OAuth2 token acquisition and refresh are not implemented. The account node supports username/password and an optional static access token field.

Development Checks

npm install
npm audit --omit=dev
npm test
npm run pack:check
git diff --check

See the release checklist for strict minimum-version installs, actual-library tests and the required isolated Node-RED deploy tests. External provider tests and current GitHub CI results are additional release requirements; historical results do not establish readiness for a new version.

Do not publish this package to npm or flows.nodered.org without explicit human approval.