queueway
v0.2.0
Published
Queueway - Zero-Config Job Queue
Readme
Status: Early access (v0.2.0). In-Memory, SQLite, PostgreSQL, Redis and RabbitMQ are all dev+prod tested, including multi-worker safety, duplicate protection and outage recovery.
✨ Features
- Zero-config by default —
npm install queueway && npx queueway init && npx queueway startand you have a working queue, dashboard, and API - Pluggable brokers — In-Memory, Redis and RabbitMQ, all production-tested
- Pluggable stores — In-Memory (tested), SQLite (tested), PostgreSQL (tested)
- Automatic setup —
queueway initfinds the PostgreSQL/Redis you already run, or starts containers for you; it never touches your existing databases - Survives outages — if the database or broker goes away, the process stays up, health reports which part is down, and everything reconnects on its own
- Safe with multiple workers — workers heartbeat, so a restart never re-runs a job another worker is still processing
- Automatic retry with exponential backoff — jobs are genuinely redelivered, not just delayed
- Dead Letter Queue (DLQ) for jobs that exceed max retries, with one-click retry from the dashboard
- Crash-recovery — if the process dies mid-job (SQLite/Postgres), the job resumes on the next start instead of vanishing
- Auto-heal —
queueway startruns your server under a watchdog that automatically restarts it if it crashes (dev or prod, same command) - Background mode — run detached from your terminal with one prompt, check on it anytime with
queueway status - Secured dashboard — real signup/login (bcrypt-hashed passwords, session cookies), not an open door
- Real health checks — the dashboard's health panel actually pings your broker/database; it doesn't just say "healthy" no matter what
- Dark/light mode — via
next-themes, matching system preference by default - Structured logging — leveled, JSON, written to a real log file — crash-proof by design (a logging failure never takes your app down)
- Full TypeScript support
🚀 Quick Start
For production, run the setup wizard first — this is required, not optional (skipping it means you're stuck with In-Memory defaults, and no data survives a restart):
npm install queueway
npx queueway init # pick a store (SQLite recommended) and broker (In-Memory)
npx queueway start # boots the queue engine + REST API + dashboard, all on one portOpen http://localhost:4287 — you'll be asked to create a one-time dashboard account (email + password), then land on the live dashboard.
For dev / quick testing, you can skip init entirely and just start using it — zero setup:
const { queue } = require("queueway"); // defaults to In-Memory broker + storeUse it as a library in your own app
const { queue } = require("queueway"); // auto-configured from queueway.config.js, if present
queue.subscribe("email.welcome", async (job) => {
console.log("Sending email to:", job.data.to);
});
async function main() {
await queue.start();
await queue.publish("email.welcome", { to: "[email protected]" });
}
main();queue.subscribe(eventName, handler) registers what should happen when a job of that type runs. queue.publish(eventName, data) enqueues one. Both calls need to be in the same running process while using the In-Memory broker — see Brokers & Stores for why, and how Redis lifts that limit.
Want the dashboard + REST API running alongside your own app too, without the CLI? Pass { withServer: true }:
await queue.start({ withServer: true, port: 4287 });This does everything npx queueway start does — including auto-loading queueway.jobs.js and printing reachable URLs (localhost, LAN, and public IP if detected) — except background mode and auto-heal, which need a separate supervisor process watching this one (see the table below).
Which command should I use?
| | queue.start() (embedded in your app) | npx queueway start (standalone CLI) |
| ----------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
| Runs inside your app's own process | ✅ — direct access to your app's own variables/functions from job handlers | ❌ — runs as a separate process, isolated |
| HTTP API + dashboard | Only with { withServer: true } | ✅ always |
| Auto-heal (restarts itself if it crashes) | ❌ — use PM2/systemd/Docker around your whole app instead | ✅ built in |
| Background mode | ❌ | ✅ |
| Best for | Adding a queue to an app you already have (Express, etc.) where job handlers need tight access to your app's own state | Running the queue as its own standalone service, with zero extra code |
Rule of thumb: if you're bolting a queue onto an existing app, use queue.start() (add { withServer: true } if you also want the dashboard). If you want the queue to just run and manage itself with no app code of your own, use npx queueway start.
⚠️ Don't run both for the same project at the same time. If your own script already calls
queue.start({ withServer: true })on port 4287 (or already has the SQLite files open), and you also runnpx queueway startin the same folder, they'll either collide on the port (EADDRINUSE) or contend over the same SQLite files. Pick one way to run your app, not both.
📊 Dashboard
Running queueway start automatically serves a full dashboard (Next.js, statically exported — no separate server or port) at whatever port your API runs on:
- Live job stats — pending / processing / completed / failed, auto-refreshing every 3s
- Real health panel — broker + database status, with live latency, not a hardcoded "healthy"
- Filterable job list — by status, with full payload data visible
- One-click retry for failed jobs, right from the table
- Dark/light mode toggle
Dashboard security
The dashboard requires a real login — bcrypt-hashed password, HttpOnly session cookie. The first person to visit creates the one admin account (queueway is designed as a single-admin dashboard, not a multi-tenant one); after that, everyone else sees a login screen. Every /queueway/* API route is protected the same way — there's no unauthenticated backdoor.
Email delivery (optional): to send the welcome email + password-reset links, set these environment variables (a .env file in your project root is loaded automatically):
QUEUEWAY_SMTP_HOST=smtp.gmail.com
QUEUEWAY_SMTP_PORT=465
[email protected]
QUEUEWAY_SMTP_PASS=your-app-password
[email protected] # optional — CC'd on every signup notificationIf these aren't set, signup/login still work fully — you just won't get the welcome/reset emails, and a warning is logged.
🧱 Brokers & Stores
| Broker | Status | Notes |
| --------- | -------------------- | -------------------------------------------------------------------------------------------------------------------- |
| In-Memory | ✅ Production-tested | Zero-config default. Single-process only — see note below |
| Redis | ✅ Production-tested | Lists-based (LPUSH/BRPOP). Several workers share the load; needs a shared store — see below |
| RabbitMQ | ✅ Production-tested | Topic exchange, durable queues, publisher confirms, dead-lettering. A job survives the worker running it — see below |
| Store | Status | Notes | | ---------- | -------------------- | ----------------------------------------------------------- | | In-Memory | ✅ Production-tested | Testing/dev only — data lost on restart | | SQLite | ✅ Production-tested | File-based, crash-recovery, zero external services | | PostgreSQL | ✅ Production-tested | Shared by several workers; required for multi-worker setups |
Configure via queueway.config.js (created by queueway init) or directly:
new Queueway({ broker: "redis", store: "postgres" });Connection details come from environment variables: RABBITMQ_URL, REDIS_URL, DATABASE_URL, SQLITE_PATH.
Why brokers/stores matter for scaling
A process is one running instance of your program, with its own private
memory. With the In-Memory broker, publish() and subscribe() only work
within the same process — two separate servers can't talk to each other
through it. Redis and RabbitMQ exist to solve exactly this: they run as their
own service, so any number of processes can publish and subscribe through them.
Several workers also need a store they can all reach. SQLite is a local file,
so two machines can't share it — which is why choosing Redis or RabbitMQ in
queueway init gives you PostgreSQL as the store, with no question asked.
Redis has no message acknowledgement
BRPOP removes a job from the list the moment a worker takes it. If that
worker is killed mid-handler, the job is already gone from Redis — so the
store is what brings it back: the record sits in processing, and recovery
re-publishes it once the worker stops heartbeating.
The practical consequence: Redis must be paired with PostgreSQL (or
SQLite for a single worker). redis + in-memory store would lose a job
permanently on a crash, which is why queueway init won't offer that pairing.
RabbitMQ does have acknowledgement — and that is the whole reason to pick it
A RabbitMQ delivery stays in the queue until the worker acknowledges it. Kill a worker mid-handler and the message is returned to the queue immediately, by the broker itself — no store lookup, no waiting for a heartbeat to go stale.
Worker dies mid-job:
Redis → job already removed from the list → the store re-publishes it
(recovered after the worker's heartbeat goes stale, ~30s)
RabbitMQ → message was never acknowledged → the broker requeues it
(another worker has it in under a second)Because the broker redelivers, Queueway's store recovery deliberately stands
down for processing jobs on RabbitMQ. Recovering them from the store as
well would hand the same job to two workers — and two invoices is a worse
failure than a slow one.
Known limitation: that hand-off only happens when the connection drops. A
worker that is alive but wedged — a handler stuck on a socket with no timeout —
holds its message unacked, and the store will not rescue it either. It stays
held until RabbitMQ's own consumer_timeout (30 minutes by default on 3.x).
Handler timeouts are on the roadmap; until then, give your handlers their own
timeouts if they talk to the network.
Which one should you use?
| | Redis | RabbitMQ |
| ------------------------------- | -------------------------------- | --------------------------------------------- |
| Worker dies mid-job | Store re-publishes after ~30s | Broker requeues in under a second |
| Needs a shared store to be safe | Yes, always | Yes, for retries and history |
| Load spreading | Even by nature (BRPOP) | Even once prefetch is set (Queueway sets 1) |
| Failed publish is detectable | Yes | Yes — publisher confirms |
| Ops cost | Low; you probably already run it | Higher; another service to operate |
If Redis is already in your stack and a job losing 30 seconds is survivable, Redis is the simpler choice. If a lost or delayed job is expensive, RabbitMQ's acknowledgement is worth the extra service.
Queue settings are declared by Queueway on first use: a durable topic
exchange queueway, durable queues queueway.<event>, and a dead-letter
exchange queueway.dlx feeding queueway.dead.<event>. Queues created by
Queueway 0.0.x predate dead-lettering and will be refused with a 406;
let them drain, then rabbitmqctl delete_queue queueway.<event> once.
Queueway declares classic durable queues. Quorum queues are a better fit for a real cluster and are on the roadmap, but they change declaration and failure semantics, so they are deliberately not used yet.
🔐 Environment variables
Every variable Queueway reads or writes is prefixed QUEUEWAY_:
| Variable | What it's for |
| ---------------------------------- | --------------------------------------------------------- |
| QUEUEWAY_DATABASE_URL | PostgreSQL connection for the job store |
| QUEUEWAY_REDIS_URL | Redis connection for the broker |
| QUEUEWAY_RABBITMQ_URL | RabbitMQ connection for the broker |
| QUEUEWAY_RABBITMQ_PREFETCH | How many messages one worker may hold unacked (default 1) |
| QUEUEWAY_RABBITMQ_MANAGEMENT_URL | Where queueway init put the RabbitMQ management UI |
| QUEUEWAY_SQLITE_PATH | Override where the SQLite file lives |
| QUEUEWAY_PORT | Dashboard/API port (default 4287) |
| QUEUEWAY_SMTP_* | Mail settings for dashboard login emails |
DATABASE_URL and REDIS_URL are among the most common names in Node
projects and usually belong to your application. Queueway never writes
them — queueway init only ever adds QUEUEWAY_* lines to your .env, and
your own configuration is left exactly as it was.
The unprefixed names are still read as a fallback, so setups from earlier versions keep working. When both are set, the prefixed one wins.
⚙️ Setup wizard
queueway init asks for a broker first, then a store, because the
broker determines which stores are honest options.
For PostgreSQL or Redis, it offers only what's actually possible on your machine:
- Use what's already running — shown only when something is listening on the default port.
- Run a container for this project — Docker.
- Cancel.
Nothing dead-ends. If a path fails you're asked what to do next, and SQLite is always there as a working fallback — it needs nothing installed and is production-tested.
Using a PostgreSQL you already have
Queueway does not touch your existing databases. It creates its own, the same way the SQLite store creates its own file:
CREATE ROLE queueway_<project> LOGIN PASSWORD '<generated>';
CREATE DATABASE queueway_<project> OWNER queueway_<project>;The role is not a superuser and has neither CREATEDB nor CREATEROLE.
Nothing is ever dropped or altered — only created. The name comes from your
package.json, so two projects on one machine never share a jobs table.
Creating a role needs administrator rights, so the wizard first tries to connect as an administrator on its own (this works on Homebrew and some Linux setups). On Windows you'll be asked for the password once — it's used at that moment and never written anywhere.
Re-running init is safe: an existing role has its password rotated, an
existing database is reused, and your jobs stay where they are.
Using Docker
The wizard writes docker-compose.queueway.yml, pulls the image, starts the
container and waits until it genuinely accepts connections.
Ports are never hardcoded. If 5432 is taken — common, since you may already
run PostgreSQL — Queueway's container takes 5433 and .env is written to
match. Your own services are left alone. The compose file is merged rather than
overwritten, so adding Redis later keeps PostgreSQL in it.
docker compose -f docker-compose.queueway.yml ps # status
docker compose -f docker-compose.queueway.yml down # stop, keep data
docker compose -f docker-compose.queueway.yml down -v # stop, delete ALL data💾 Your job data
When you re-run init and Queueway finds data from an earlier setup, it always
asks. Nothing is deleted without you choosing it, twice.
? There's job data here from an earlier setup. What should happen to it?
❯ Keep it — reconnect to the existing jobs
Start fresh — delete it and create an empty database
CancelKeep it restarts the container with the credentials in .env and
reconnects to your jobs. If the password in .env doesn't match that data,
Queueway says so and stops, leaving the data untouched.
That last case is a PostgreSQL rule, not a Queueway limitation:
POSTGRES_PASSWORD is only applied to an empty data directory, after which the
password lives inside the data. Without the original, that data can't be opened
by anyone. If you still have the old DATABASE_URL, put it back in .env and
run init again — it comes straight back.
DATABASE_URLin.envis the key to your job data. Back it up like a password. Lose it and the data in that volume can't be recovered.
Start fresh asks for confirmation, then removes only that service's volume — resetting PostgreSQL leaves Redis data alone.
🔌 When a database or broker goes down
Containers restart themselves (restart: unless-stopped). Queueway's job is to
survive the outage and reconnect:
- The process stays alive. A dropped connection is a logged warning, not a crash.
- Errors reach your code, not the process.
publish()rejects so you can catch and retry. - Health is honest and fast. Each component is checked independently with a short timeout, so one failure never hides the others — and the dashboard keeps working, showing exactly which part is down.
- It reconnects on its own. No restart needed. Jobs stranded mid-flight are picked up automatically within 30 seconds.
node scripts/regression-test.js # In-Memory + SQLite, end to end — run after every change
node scripts/postgres-test.js # schema, retry/DLQ, crash recovery, pool cleanup
node scripts/postgres-multiworker-test.js # no duplicates, no orphans lost
node scripts/redis-test.js # multi-worker delivery, durability, outage recovery
node scripts/rabbitmq-test.js # acknowledgement, redelivery, dead-lettering, reconnect
node scripts/resilience-test.js # stops and restarts your container for real👥 Running several workers
Several workers sharing one PostgreSQL is the point of the Redis and RabbitMQ
brokers. Recovery has to be careful there: a job marked processing might
belong to a worker that's alive and busy, and re-queuing it would run it twice
— two invoices, two emails.
Each worker registers in queueway_workers and heartbeats every 10 seconds:
- A job held by a live worker is never touched.
- A job whose owner stopped heartbeating (30s) is reclaimed.
- Claims use
FOR UPDATE SKIP LOCKED, so two workers recovering at the same instant can't be handed the same job. - A clean shutdown deregisters immediately — no 30-second wait after a normal restart.
🖥️ CLI Reference
| Command | What it does |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| queueway init | Interactive wizard — writes queueway.config.js and a starter queueway.jobs.js |
| queueway start | Boots the server (config + jobs auto-loaded). Asks foreground vs background; -b/-f to skip the prompt, -p <port> to set the port (default 4287) |
| queueway status | Checks whether the server is up — works for dev/prod, foreground/background |
| queueway stop | Stops a server started in the background |
| queueway health | Prints broker/database health + job stats from a running instance |
queueway.jobs.js is where you define what happens for each job type:
// queueway.jobs.js — auto-loaded by `queueway start` (and by `queue.start()` too)
module.exports = function registerJobs(queue) {
queue.subscribe("email.welcome", async (job) => {
console.log("Sending:", job.data);
});
};You're not boxed into that one file — it's just an entry point, so require() as many other files from it as you want. Or skip it entirely and use a jobs/ directory instead: every .js file inside is auto-loaded, no manual wiring needed, so you can organize handlers across as many files as you like (jobs/email.js, jobs/payments.js, ...) — the same freedom you'd have embedding Queueway directly in your own app.
🧩 Library API — do everything the dashboard does, in code
Every action the dashboard/REST API can do is also a plain method on queue — no dashboard, no HTTP calls needed, if you'd rather manage jobs directly from your own code:
const { queue } = require('queueway');
await queue.publish(eventName, data); // enqueue a job
queue.subscribe(eventName, async (job) => {}); // register a handler
await queue.getStats(); // { jobs: { pending, processing, completed, failed, retrying, archived }, total }
await queue.getJob(jobId); // single job, or null
await queue.getJobs(status?, limit?); // list jobs, optionally filtered by status
await queue.getDLQ(limit?); // failed jobs currently in the dead-letter queue
await queue.retryJob(jobId); // re-queue a job (resets attempts to 0)
await queue.deleteJob(jobId); // permanently delete a job record
await queue.getHealth(); // real broker/database health checkThese are the exact same methods the dashboard and /queueway/* REST routes call internally — so anything you can click in the dashboard, you can also do directly in your own scripts, cron jobs, or admin tooling.
📡 REST API
All routes below require a logged-in session (see Dashboard security).
| Method | Route | Description |
| ------ | ------------------------------- | ----------------------------------------------------------- |
| GET | /queueway/health | Broker + database + API status (200 if healthy, 503 if not) |
| GET | /queueway/stats | Job counts by status |
| GET | /queueway/jobs?status=&limit= | List jobs, optionally filtered |
| GET | /queueway/jobs/:id | Get one job |
| GET | /queueway/dlq?limit= | List failed (dead-lettered) jobs |
| POST | /queueway/jobs/:id/retry | Re-queue a job (resets attempts to 0) |
| DELETE | /queueway/jobs/:id | Permanently delete a job record |
Auth routes (always public, obviously): POST /auth/signup, POST /auth/login, POST /auth/logout, GET /auth/me, GET /auth/status, POST /auth/forgot-password, POST /auth/reset-password.
📦 Monorepo Structure
packages/
core/ -> the "queueway" npm package: engine, brokers, stores, auth, REST API, CLI, dashboard assets
src/cli/ -> CLI commands (init, start, status, stop, health)
src/auth/ -> signup/login/session/email
src/logging -> crash-proof structured logger
dashboard/ -> Next.js dashboard source (statically exported and bundled into core/public at build time)
examples/
basic-server.ts -> minimal library-usage exampleThere is only one npm package to install (queueway) — the CLI, library, and dashboard assets all ship together.
🧹 Uninstalling
npm uninstall queueway removes the package itself, but not the files it created while running — this is deliberate, since those files may contain real data (dashboard accounts, job history) you might not want silently deleted just because you removed a dependency:
.queueway/— dashboard login (auth.db), job data if using SQLite (queueway.db), and logsqueueway.config.js/queueway.jobs.js— your config and job handlers (created byqueueway init)docker-compose.queueway.ymland its containers/volumes, if you used the Docker optionDATABASE_URL/REDIS_URLin.env
If you're removing Queueway for good and want a clean slate, delete these yourself:
docker compose -f docker-compose.queueway.yml down -v # only if you used Docker
npm uninstall queueway
rm -rf .queueway queueway.config.js queueway.jobs.js docker-compose.queueway.ymlA PostgreSQL role and database created inside your own PostgreSQL are left in place — Queueway never drops anything. Remove them yourself if you want to:
DROP DATABASE queueway_<project>;
DROP ROLE queueway_<project>;🗺️ Roadmap
- [x] CORE queue engine — In-Memory + SQLite, retry, DLQ, crash-recovery
- [x] REST API, dashboard, CLI (init/start/status/stop/health)
- [x] Dashboard authentication (signup/login/reset), structured logging
- [x] PostgreSQL dev+prod testing pass — worker-aware recovery, outage resilience, automatic setup
- [x] Redis dev+prod testing pass — multi-worker distribution, durability, outage recovery
- [x] RabbitMQ dev+prod testing pass — acknowledgement, redelivery without duplicates, reconnect, dead-lettering
- [ ] Handler timeouts (so a wedged worker can't hold a message indefinitely)
- [ ] Quorum queue support for clustered RabbitMQ
- [ ] Community (Discord, contributor program)
- [ ] PRO plugins (AI error analyzer, circuit breaker, SSO, compliance reports)
- [ ] Cloud SaaS
Modestick
Queueway is built and maintained by Modestick — a creative and technology studio building AI agents & automation, custom software, mobile apps, IoT solutions, cloud infrastructure, and brand/design work for clients who care about quality.
Contributing
See CONTRIBUTING.md
License
MIT — see LICENSE
