Happy House - Ecommerce Docs
Developer ResourcesArchitecture

BullMQ Worker Wiring

The one-worker-per-queue rule, why @Processor name options do not route jobs, and the dispatcher pattern.

Audience: Backend developers — anyone adding a job, a queue, or a worker Scope: How BullMQ workers are wired in this repo, and why the shape is mandatory

BullMQ Worker Wiring

1. The rule

Exactly one class per queue carries @Processor. That class is a dispatcher: it holds a Record<JobEnum, (job) => Promise<unknown>> and routes on job.name. Every actual handler is a plain @Injectable() whose method the dispatcher calls.

This is not a style preference. It is the only arrangement that guarantees a job is executed by the handler it was meant for.

2. Why @Processor(Q, { name: SOME_JOB }) is wrong

@nestjs/bullmq builds one BullMQ Worker per class decorated with @Processor. WorkerOptions.name is a monitoring label — BullMQ stores it on processed jobs so you can see which worker handled what. It does not filter. Two workers on one queue compete for every job on it.

Measured, with two workers differing only in opts.name, 20 jobs of each name:

workerA (opts.name="job.alpha") received: {"job.alpha":17, "job.beta":4}
workerB (opts.name="job.beta")  received: {"job.beta":16, "job.alpha":3}

A rule with a reproduction survives a refactor; one without does not.

3. What went wrong here

Four queues were arranged that way — MAINTENANCE with five workers, and OUTBOX, PRODUCTS and CATALOG with two each. None of those handlers checked job.name, so a mis-delivered job ran the wrong handler's body against the wrong payload rather than returning.

The severe case: DeleteUserAccountPayload and PermanentDeleteUserAccountPayload both carry userId, and both handlers read it and act. Neither threw on the other's payload — so a soft-delete request handled by the permanent-delete worker anonymised the account irreversibly, and reported success. On OUTBOX, a dispatch job consumed by the cleanup handler meant nothing was relayed to any queue at all.

4. Do not "fix" a future occurrence with a guard

Adding if (job.name !== X) return converts wrong-handler-runs into job-silently-eaten-and-marked-completed. Better, still broken. The inventory module shipped in exactly that state before this wiring was fixed.

5. The dispatcher shape

@Injectable()
@Processor(QueueName.MAINTENANCE, { concurrency: 5 })
export class MaintenanceQueueProcessor extends WorkerHost {
	private readonly handlers: Record<MaintenanceJob, (job: Job) => Promise<unknown>> = {
		[MaintenanceJob.DELETE_USER_ACCOUNT]: (job) => deleteAccount.process(job),
		[MaintenanceJob.PERMANENT_DELETE_USER_ACCOUNT]: (job) => permanentDelete.process(job),
		// ...
	};

	async process(job: Job): Promise<unknown> {
		const handler = this.handlers[job.name as MaintenanceJob];
		if (!handler) throw new Error(`Unknown job name: ${job.name}`);
		return handler(job);
	}
}

Two properties do the real work:

  1. Record<JobEnum, handler> makes a missing handler a compile error. A new MaintenanceJob member without a handler fails tsc — the runtime symptom of a missing handler is a job that silently completes having done nothing.
  2. An unknown job name throws instead of returning quietly. Returning quietly is what let the original defect live its entire lifetime undetected.

The gap a type cannot close: [ProductJob.IMPORT_PRODUCTS]: exportProcessor.process type-checks perfectly, so transposing two handlers in a Record produces zero compile errors. A per-dispatcher routing spec asserts each job name reaches its own handler.

6. The current wiring

QueueDispatcherJob names
MAINTENANCEmodules/maintenance/workers/maintenance-queue.processor.ts5 — delete + permanent-delete account, cache revalidation, catalog + product stalled-job sweeps. concurrency: 5
OUTBOXmodules/outbox/workers/outbox-queue.processor.ts2 — dispatch, cleanup. concurrency: 2
PRODUCTSmodules/products/workers/product-queue.processor.ts2 — import, export
CATALOGmodules/catalog/workers/catalog-queue.processor.ts2 — import, export
INVENTORYmodules/inventory/workers/inventory-queue.processor.ts5 — import stock, export stock, sweep expired reservations, reconcile, project activity
BANNERS, NOTIFICATIONSone processor eachalready correct, unchanged

Two notes on the table:

  • MAINTENANCE's five handlers are provided by four different feature modules, so the queue gets its own MaintenanceWorkerModule — nothing imports it except the application root, so the four imports form a diamond, not a cycle.
  • OUTBOX gets concurrency: 2 because the two former workers each defaulted to 1; MAINTENANCE gets 5, the highest of the merged values. Concurrency is a Worker option, and there is now one worker, so it is set once on the dispatcher.

7. The two gates, and what they cannot see

GateCatches
Record<JobEnum, …> in the dispatcherA job name with no handler — at compile time
services/bullmq/queue-worker-uniqueness.spec.tsA second @Processor on any queue, anywhere in apps/api/src
services/bullmq/queue-dispatch-routing.spec.tsA handler wired to the wrong job name, or the wrong method on the right object

What none of them see: the routing spec passes probes positionally while Nest injects by type, so transposing two constructor parameter types would not be caught. The current wiring is verified correct; closing that gap needs a container-level test.

8. Registered queues

services/bullmq/bull.module.ts's REGISTERED_QUEUES contains BANNERS, MAINTENANCE, NOTIFICATIONS and PRODUCTS. OUTBOX, CATALOG and INVENTORY are registered by their own modules and are not in that array — so bull-board only shows the four registered ones, and they carry no env-configured defaultJobOptions. PRODUCTS is in the array deliberately: its jobs are enqueued exclusively through the outbox with no per-call options, and registering it is what gives them real defaultJobOptions (attempts/backoff) instead of BullMQ's bare attempts: 1.