@push.rocks/smartimap

A TypeScript IMAP client and TLS-capable server with OAuth authentication, streaming message access, and pluggable principal-scoped storage.

Install

Install @push.rocks/smartimap with pnpm:

pnpm add @push.rocks/smartimap

Usage

@push.rocks/smartimap is designed to facilitate the connection to an IMAP server, filter email messages and handle them in a streaming, event-driven manner. Below is a comprehensive guide that covers setting up the environment, features of the module, and usage scenarios.

Setting Up Your Environment

Before starting with @push.rocks/smartimap, ensure you have Node.js installed. You may also need environment variables for the IMAP server settings, which can be conveniently managed using a .env file.

Importing the Required Modules

Begin by importing the ImapClient class and any other necessary modules:

import { ImapClient, type ImapClientConfig } from '@push.rocks/smartimap';

Configuration Object

Define a configuration object ImapClientConfig for the IMAP connection. This configuration object will include details such as the host, port, and authentication details:

const imapConfig: ImapClientConfig = {
  host: 'imap.example.com',
  port: 993, // Secure port for IMAP
  secure: true, // Connect securely using SSL/TLS
  auth: {
    user: '[email protected]',
    pass: 'password123',
  },
  mailbox: 'INBOX', // Default mailbox to access
  filter: { seen: false }, // Default filter for unseen messages
};

For an endpoint that requires an explicit TLS upgrade, select mandatory STARTTLS instead of the implicit-TLS default:

const starttlsConfig: ImapClientConfig = {
  host: 'imap.example.com',
  port: 143,
  tlsMode: 'starttls',
  auth: { user: '[email protected]', pass: 'password123' },
};

tlsMode: 'starttls' fails when the server does not advertise STARTTLS. The legacy secure boolean remains available for compatibility.

For submission endpoints that require an explicit upgrade, select mandatory STARTTLS instead of the implicit-TLS default:

const starttlsConfig: ImapClientConfig = {
  host: 'imap.example.com',
  port: 143,
  tlsMode: 'starttls',
  auth: { user: '[email protected]', pass: 'password123' },
};

tlsMode: 'starttls' fails when the server does not advertise STARTTLS. The legacy secure boolean remains available for compatibility.

OAuth2 (XOAUTH2) Authentication

Instead of a password, you can authenticate with an OAuth2 access token. The auth option accepts either shape:

import { ImapClient } from '@push.rocks/smartimap';

// Password login
const passwordClient = new ImapClient({
  host: 'imap.example.com',
  auth: {
    user: '[email protected]',
    pass: 'password123',
  },
});

// OAuth2 login (XOAUTH2 / OAUTHBEARER, e.g. for Gmail or Office 365)
const oauthClient = new ImapClient({
  host: 'imap.gmail.com',
  auth: {
    user: '[email protected]',
    accessToken: 'ya29.a0AfH6...',
  },
});

Note: there is no token refresh hook — the underlying imapflow connection authenticates once and has no re-authentication path. When an access token expires, create a new ImapClient with a fresh token and connect again.

For offline testing, the bundled ImapServer supports AUTHENTICATE XOAUTH2. Register valid tokens per user:

import { ImapServer } from '@push.rocks/smartimap';

const imapServer = new ImapServer();
imapServer.addUser('[email protected]', 'password123');
imapServer.createInbox('[email protected]', 'INBOX');
imapServer.addOAuthToken('[email protected]', 'valid-access-token');
const port = await imapServer.start(0); // resolves with the bound port

Production IMAP server

ImapServer can expose an application mail store through a principal-scoped IImapMailBackend. A custom backend requires an authentication callback and implicit TLS unless allowInsecureAuth is explicitly enabled for an isolated test. Custom backends default to XOAUTH2 and OAUTHBEARER; password LOGIN and PLAIN are not advertised unless the application opts into them.

Consumers that may resolve multiple smartimap versions at runtime can verify the production backend contract before constructing a listener:

import { IMAP_SERVER_BACKEND_API_VERSION } from '@push.rocks/smartimap';

if (IMAP_SERVER_BACKEND_API_VERSION !== 2) {
  throw new Error('This application requires smartimap server backend API v2.');
}

import { readFile } from 'node:fs/promises';
import {
  ImapServer,
  type IImapMailBackend,
} from '@push.rocks/smartimap';

declare const backend: IImapMailBackend;
declare function validateAccessToken(
  username: string,
  token: string,
): Promise<{ userId: string } | null>;

const server = new ImapServer({
  backend,
  hostname: 'imap.example.com',
  tls: {
    key: await readFile('/run/secrets/imap-tls-key.pem'),
    cert: await readFile('/run/secrets/imap-tls-cert.pem'),
  },
  authenticate: async (request) => {
    if (request.mechanism !== 'XOAUTH2' && request.mechanism !== 'OAUTHBEARER') {
      return null;
    }
    const session = await validateAccessToken(request.username, request.secret);
    return session ? { id: session.userId, username: request.username } : null;
  },
  maxConnections: 500,
  maxLiteralBytes: 25 * 1024 * 1024,
});

await server.start(993, '0.0.0.0');

// Stop accepting connections and drain active sessions during shutdown.
await server.stop();

Every backend method receives the authenticated principal id. Treat mailbox and message ids as untrusted protocol input and enforce ownership again in the backend. Persist stable, monotonically increasing UIDs per mailbox and never reuse a mailbox's uidValidity value for a different UID history.

The production backend contract is:

| Method | Required behavior | | --- | --- | | listMailboxes / getMailboxByName | Return only principal-owned mailboxes and their delimiter, attributes, subscription state, uidValidity, and uidNext. | | getMessages | Return the selected mailbox in stable ascending UID order; the server derives sequence numbers from this exact ordering. | | appendMessage | Persist the raw RFC 5322 bytes, flags, internal date, and a newly allocated mailbox UID. | | updateFlags | Apply add/remove/replace semantics only to the requested principal/mailbox UIDs. | | copyMessages / moveMessages | Allocate destination UIDs and return every source-to-destination UID mapping. A move also removes the source entries. | | expungeMessages | Remove \\Deleted messages, optionally restricted to requested UIDs, and return the removed UIDs. | | createMailbox / deleteMailbox / renameMailbox / setSubscribed | Persist mailbox lifecycle and subscription changes with ownership checks. | | optional subscribeChanges | Notify exists, flags, expunge, or mailbox changes; it may return an unsubscribe callback. |

Throw ImapBackendError for protocol-visible failures. Supported response codes are ALREADYEXISTS, CANNOT, INUSE, LIMIT, NONEXISTENT, NOPERM, SERVERBUG, and TRYCREATE.

ImapServer accepts these operational controls:

| Option | Purpose | | --- | --- | | backend, authenticate | Principal-scoped storage and authentication. A custom backend requires authenticate. | | authMechanisms | Explicit subset of LOGIN, PLAIN, XOAUTH2, and OAUTHBEARER. Custom backends default to OAuth-only. | | tls, allowInsecureAuth | Implicit TLS configuration; insecure custom-backend auth is an isolated-test escape hatch only. | | host, hostname | Default bind address and advertised server hostname. | | maxConnections | Concurrent accepted-session ceiling. | | socketTimeoutMs | Idle socket deadline. | | maxCommandBytes, maxLiteralBytes, maxQueuedCommands | Per-connection command, message literal, and queued-work bounds. | | maxAuthAttempts | Failed authentication ceiling before the session is closed. | | onError | Receives listener, session, unexpected backend/session, and asynchronous notification errors. Protocol-visible ImapBackendError failures are rendered to the client instead. |

Creating an Instance

Create an instance of the ImapClient class using the configuration object:

const smartImap = new ImapClient(imapConfig);

Connecting to the IMAP Server

Use the connect method to establish a connection with the IMAP server:

smartImap.connect().catch(console.error);

Handling Events

@push.rocks/smartimap is event-driven and emits various events such as message, error, connected, and disconnected. These events can be handled to perform appropriate actions.

Handling the message Event

The message event is emitted when a new message is fetched and parsed. Here's an example of how to handle this event:

smartImap.on('message', (parsedMessage) => {
  console.log('New message received:');
  console.log('From:', parsedMessage.from?.text);
  console.log('Subject:', parsedMessage.subject);
  console.log('Text:', parsedMessage.text);
});

Handling the error Event

The error event is emitted when an error occurs:

smartImap.on('error', (error) => {
  console.error('Error occurred:', error);
});

Handling the connected Event

The connected event is emitted when the connection to the IMAP server is successfully established:

smartImap.on('connected', () => {
  console.log('Successfully connected to the IMAP server');
});

Handling the disconnected Event

The disconnected event is emitted when the connection to the IMAP server is closed:

smartImap.on('disconnected', () => {
  console.log('Disconnected from the IMAP server');
});

Fetching and Parsing Messages

Upon connecting to the IMAP server, the ImapClient instance will automatically start fetching and parsing messages based on the provided filter criteria. The parsed messages are then emitted through the message event. Below is a detailed example:

smartImap.on('message', (parsedMessage) => {
  console.log('From:', parsedMessage.from?.text);
  console.log('Subject:', parsedMessage.subject);
  console.log('Text:', parsedMessage.text);
});

The parsedMessage object contains various properties such as from, subject, text, etc., derived using the mailparser module internally.

Modifying the Message Filter

The message filter criteria can be dynamically altered using the setFilter method:

smartImap.setFilter({ seen: false, from: '[email protected]' });

Mailbox and Mutation Operations

The client serializes operations so temporary mailbox switches cannot race with message fetching:

const mailboxes = await smartImap.listMailboxes();
const inbox = await smartImap.getMailboxStatus('INBOX');

await smartImap.updateFlags(42, ['\\Seen'], 'add', 'INBOX');
await smartImap.updateFlags(42, ['\\Flagged'], 'replace', 'INBOX');

const moved = await smartImap.moveMessage(42, 'Archive', 'INBOX');
if (moved.destinationUid !== undefined) {
  await smartImap.deleteMessage(moved.destinationUid, 'Archive');
}

const appended = await smartImap.appendMessage(
  'Drafts',
  'From: [email protected]\r\nTo: [email protected]\r\nSubject: Draft\r\n\r\nHello',
  ['\\Draft'],
  new Date(),
);

listMailboxes() returns paths, names, delimiters, special-use flags, server flags, and subscription state. getMailboxStatus() returns message, recent, UIDNEXT, UIDVALIDITY, unseen, and (when supported) HIGHESTMODSEQ values. updateFlags() accepts add, remove, or replace; moveMessage() returns the destination UID when the server supplies COPYUID; and appendMessage() returns the assigned UID when the server supplies APPENDUID.

Disconnecting from the Server

To safely disconnect from the IMAP server, use the disconnect method:

smartImap.disconnect().catch(console.error);

Full Example

Below is a comprehensive example that showcases creating an instance, connecting to the server, listening for messages, and handling errors and other events:

import { ImapClient, type ImapClientConfig } from '@push.rocks/smartimap';

const imapConfig: ImapClientConfig = {
  host: 'imap.example.com',
  port: 993,
  secure: true,
  auth: {
    user: process.env.IMAP_USER || '[email protected]',
    pass: process.env.IMAP_PASSWORD || 'password123',
  },
  mailbox: 'INBOX',
  filter: { seen: false },
};

const smartImap = new ImapClient(imapConfig);

smartImap.connect().catch(console.error);

smartImap.on('message', (parsedMessage) => {
  console.log('----New Message----');
  console.log('From:', parsedMessage.from?.text);
  console.log('Subject:', parsedMessage.subject);
  console.log('Text:', parsedMessage.text);
});

smartImap.on('error', (error) => {
  console.error('Error:', error);
});

smartImap.on('connected', () => {
  console.log('Connected to the IMAP server');
});

smartImap.on('disconnected', () => {
  console.log('Disconnected from the IMAP server');
});

// Changing filter to show only unseen messages from a specific sender
smartImap.setFilter({ seen: false, from: '[email protected]' });

// Disconnect from the server safely
setTimeout(() => {
  smartImap.disconnect().catch(console.error);
}, 30000); // Timeout after 30 seconds

Advanced Usage

In this section, we delve into more advanced scenarios and functionalities.

Searching for Specific Messages

You may sometimes need to fetch messages based on more specific search criteria. @push.rocks/smartimap lets you customize the search queries by modifying the filter. Below is an example:

import { ImapClient, type ImapClientConfig } from '@push.rocks/smartimap';

const imapConfig: ImapClientConfig = {
  host: 'imap.example.com',
  port: 993,
  secure: true,
  auth: {
    user: process.env.IMAP_USER || '[email protected]',
    pass: process.env.IMAP_PASSWORD || 'password123',
  },
  mailbox: 'INBOX',
  filter: { seen: true },
};

const smartImap = new ImapClient(imapConfig);

smartImap.connect().catch(console.error);

smartImap.on('message', (parsedMessage) => {
  console.log('Filtered Message:', parsedMessage.subject);
});

smartImap.on('connected', () => {
  console.log('Connected');

  // Adjust filter as needed; the next fetch cycle applies it
  smartImap.setFilter({ subject: 'Invoice' });
});

smartImap.on('error', (error) => {
  console.error('Error:', error);
});

Streamlined Processing with Event Emitters

For more complex workflows, you can use event emitters to segregate processing tasks. Below is an example of a MessageProcessor class using event emitters:

import { ImapClient, type ImapClientConfig } from '@push.rocks/smartimap';
import EventEmitter from 'events';

class MessageProcessor extends EventEmitter {
  constructor(private imapConfig: ImapClientConfig) {
    super();
    const smartImap = new ImapClient(imapConfig);

    smartImap.on('message', (parsedMessage) => {
      this.emit('processMessage', parsedMessage);
    });

    smartImap.on('error', (error) => {
      console.error('IMAP Error:', error);
    });

    smartImap.connect().catch(console.error);
  }

  startProcessing() {
    this.on('processMessage', this.processMessage.bind(this));
  }

  private processMessage(parsedMessage: any) {
    console.log('Processing message from:', parsedMessage.from?.text);

    // Additional processing logic here...

    this.emit('messageProcessed', parsedMessage);
  }
}

const messageProcessor = new MessageProcessor(imapConfig);
messageProcessor.startProcessing();

messageProcessor.on('messageProcessed', (message) => {
  console.log('Message processed:', message.subject);
});

Additional Features and Usage

Handling Attachments

The mailparser library used by @push.rocks/smartimap allows you to process email attachments. Here's how you can handle attachments:

smartImap.on('message', (parsedMessage) => {
  if (parsedMessage.attachments && parsedMessage.attachments.length > 0) {
    parsedMessage.attachments.forEach((attachment) => {
      console.log('Attachment:', attachment.filename);
      // Process the attachment as needed
    });
  }
});

Advanced Filtering

You can utilize complex filter criteria as allowed by the IMAP protocol. For example, filtering by date range:

smartImap.setFilter({
  since: new Date('2023-10-01'),
  before: new Date('2023-10-10'),
});

Error Handling Strategies

Proper error handling is crucial for robust applications. Note that an ImapClient instance cannot reconnect: the underlying imapflow connection authenticates once and has no re-authentication path. To recover from a dropped connection, create a fresh instance:

let smartImap: ImapClient;

const startClient = () => {
  smartImap = new ImapClient(imapConfig);

  smartImap.on('message', (parsedMessage) => {
    console.log('Subject:', parsedMessage.subject);
  });

  smartImap.on('error', (error) => {
    if (error.message.includes('Connection closed')) {
      // Reconnect with a fresh instance
      startClient();
    } else {
      console.error('Unexpected error:', error);
    }
  });

  smartImap.connect().catch(console.error);
};

startClient();

Full Documentation and API Reference

The above examples demonstrate how to use @push.rocks/smartimap comprehensively. For detailed API documentation, refer to the project's documentation generated with tools like TypeDoc, ensuring you cover all methods and properties available.

Conclusion

With @push.rocks/smartimap, handling IMAP emails in a Node.js environment becomes efficient and straightforward. Utilize the detailed examples and advanced features to build tailored solutions for your email processing needs.

Enjoy coding with @push.rocks/smartimap!

Issue Reporting and Security

For reporting bugs, issues, or security vulnerabilities, please visit community.foss.global/. This is the central community hub for all issue reporting. Developers who sign and comply with our contribution agreement and go through identification can also get a code.foss.global/ account to submit Pull Requests directly.

License and Legal Information

This repository contains open-source code that is licensed under the MIT License. A copy of the MIT License can be found in the license file within this repository.

Please note: The MIT License does not grant permission to use the trade names, trademarks, service marks, or product names of the project, except as required for reasonable and customary use in describing the origin of the work and reproducing the content of the NOTICE file.

Trademarks

This project is owned and maintained by Task Venture Capital GmbH. The names and logos associated with Task Venture Capital GmbH and any related products or services are trademarks of Task Venture Capital GmbH and are not included within the scope of the MIT license granted herein. Use of these trademarks must comply with Task Venture Capital GmbH's Trademark Guidelines, and any usage must be approved in writing by Task Venture Capital GmbH.

Company Information

Task Venture Capital GmbH
Registered at District court Bremen HRB 35230 HB, Germany

For any legal inquiries or if you require further information, please contact us via email at [email protected].

By using this repository, you acknowledge that you have read this section, agree to comply with its terms, and understand that the licensing of the code does not imply endorsement by Task Venture Capital GmbH of any derivative works.