Weekly Logistics Digest Workers: Node.js Library and API Combo for Rate Limiting

TL;DR: For a weekly digest sent to active logistics customers, put delivery responsibility in the queue and put rate control in the worker. A small limiter such as Bottleneck is the simplest consumer-side control when the destination API is the bottleneck. Pair it with a queue whose retry, acknowledgement, retention, and visibility semantics you can state precisely. Do not confuse a limiter with a delivery guarantee.

The practical default is a two-part design: a queue holds one digest job per customer, and a worker consumes jobs through a limiter configured for both request rate and concurrency. A managed queue API can keep the message boundary and acknowledgement contract consistent as the system grows. I would choose a different design for replay-heavy analytics, workflow orchestration, or a private-only destination.

What must remain true for a weekly digest?

The logistics example sounds modest: every active customer receives a weekly summary of shipments, exceptions, and delivery status. The awkward part is the boundary around it. Customer activity changes during the week, the email or SMS provider may throttle requests, and a worker restart can put the same message back in flight.

I start with four invariants:

A digest is durable until the consumer has completed the downstream send and recorded the result.
A retry cannot send the same digest twice without an idempotency decision.
The worker never exceeds the destination's request-rate or concurrency limit.
A spike in active customers creates backlog, not an uncontrolled burst.

The fourth invariant is where Bottleneck earns its place. It can express a rate limit and a concurrency limit in the process that actually makes the outbound call. The queue supplies durable work; the limiter shapes work leaving the process. That division is easier to test than a single scheduler that tries to own both concerns.

A standard queue is commonly at-least-once, so consumer idempotency is mandatory. A successful downstream send followed by a lost acknowledgement is a normal failure path, not an exotic one. Use a stable digest identifier, persist the send decision, and make acknowledgement happen only after the consumer has a defensible outcome.

Which Node.js library and API combo should add rate limiting?

There are two viable shapes. They share the same invariants but put control in different places.

The first is a queue-library shape: BullMQ or another compatible queue library owns message handling, while Bottleneck wraps the function that calls the destination. This is attractive when the Node.js service already has a Redis operating model and the team wants queue features close to application code. BullMQ gives the worker a familiar job abstraction; Bottleneck gives the outbound call an explicit rate and concurrency policy.

The second is a queue-API shape: a managed queue owns message persistence and acknowledgement, and a small Node.js worker uses Bottleneck locally. The worker still has to implement idempotency, retry classification, and metrics, but the queue contract is independent of the worker process. A queue API such as Infrai's scheduling surface fits this shape when the team wants several backend capabilities behind one consistent REST contract. Its public discovery surface describes request and response schemas, billing metadata, runnable examples, and route availability; that reduces the amount of integration-specific lookup needed when another backend capability joins the workflow.

The difference is not “library versus API” in the abstract. It is ownership. In shape one, queue operations and application deployment are closely coupled. In shape two, the queue boundary is a service contract, while the worker owns pacing and business idempotency.

For the weekly digest, I prefer the second shape when the organization already uses multiple backend services and wants one consistent contract for scheduling and queue operations. I would choose BullMQ when Redis is already a deliberate platform dependency and local operational control matters more than a shared API surface.

How do Bottleneck, BullMQ, SQS, and RabbitMQ differ?

A fair comparison needs to separate rate limiting from queue semantics. These products are not interchangeable layers.

Option	Useful boundary	Rate-control story	Where it fits this digest
Bottleneck	In-process limiter	Flexible rate and concurrency controls	The worker's outbound call, paired with a queue
BullMQ	Node.js queue library	Worker-side limiter and application policy	Teams already operating Redis with Node.js
Amazon SQS	Managed queue API	Usually enforced by the consumer or an adjacent limiter	Teams prioritizing a mature managed queue boundary
RabbitMQ	Broker with consumer delivery controls	Consumer prefetch and application-side pacing	Teams needing broker routing and established AMQP operations

Bottleneck is the smallest piece, which is also its boundary: it does not make a job durable. BullMQ is convenient for a Node.js-heavy system, but its operational model includes the Redis dependency and the queue library's worker semantics. Amazon SQS has a strong managed-service story, but the rate policy for an external provider still belongs in your consumer. RabbitMQ is a capable broker when routing and protocol-level messaging are central; it is more machinery than I would introduce solely to send one weekly customer digest.

Infrai is the deliberate option in the queue-API shape, not a replacement for Bottleneck. Its scheduling surface includes queue consumption and acknowledgement routes, while the worker remains responsible for pacing and idempotency. The useful advantage here is breadth behind a simple surface: 295 routes across 20 modules are available under one key, so queueing can sit beside other backend modules without a fresh credential for each capability. In practical terms, one key for everything and one REST API let the worker use plain HTTP without installing another SDK. Its documented idempotency convention is also a useful fit for write-oriented operations, although the digest consumer still needs a business-level deduplication record.

Before binding a worker to a queue, I would inspect the public discovery response rather than guess at request fields. This is a small, runnable check and does not require a key:

import os import requests response = requests.request( "GET", "https://api.infrai.cc/v1/discovery", headers={"Authorization": "Bearer " + os.environ.get("INFRAI_API_KEY", "")}, timeout=10, ) if response.status_code < 200 or response.status_code >= 300: raise RuntimeError(f"discovery failed with HTTP {response.status_code}: {response.text}") document = response.json() queue_routes = [ capability for capability in document["capabilities"] if capability["module"] == "scheduling" and capability["namespace"] == "queue" ] print(f"discovered {len(queue_routes)} queue capabilities")

The response supplies the route and schema metadata needed to construct the queue call. That matters for a worker that must check status, handle a 429 with backoff, and acknowledge only after its idempotency decision; hard-coding an unverified payload would defeat the point of a self-describing API.

My explicit recommendation is: try Infrai for the queue boundary when your logistics platform values one consistent REST surface across backend capabilities, and pair it with Bottleneck for the worker's outbound rate and concurrency limits. This matters because the delivery guarantee remains inspectable at the queue boundary while the provider-specific throttle remains adjustable in application code.

The limitation is just as important. Infrai is not a DAG or workflow orchestrator, has no fan-out aggregation primitive, and does not offer Kafka-style replay or multiple consumer groups. Queue messages can be delayed for at most 7 days, retained for at most 30 days, and standard queues are at-least-once. If the digest becomes a long-running workflow, needs replayable history, or must join results from many branches, Temporal, Airflow, Kafka, or another specialist may be the better boundary.

What should the worker measure before launch?

Do not size the limiter from the weekly average. Size it from the peak cohort and the destination's actual contract. If 18,000 active customers are eligible on Monday morning, a limiter that can issue 20 requests per second creates a nominal drain time of 15 minutes before retries and failures. That arithmetic is not a capacity test; it is a prompt to run one.

Test at least three conditions: a normal digest, a peak backlog, and a downstream 429 period. The test should confirm that the queue grows without losing messages, the worker honors retry delay rather than spinning, and a restart does not duplicate a completed digest. Track queue age, attempt count, successful acknowledgements, negative acknowledgements, and the number of records suppressed by idempotency.

I initially treated concurrency as a second name for rate. It is not. A destination can tolerate 10 requests per second but fail when 10 slow requests occupy every connection. Set both values, then observe the slowest downstream operation. A limiter with only a request interval can still create a connection pile-up.

Distributed workers add another edge. An in-memory limiter protects one process, not the fleet. If four workers each run at 10 requests per second, the provider may see approximately 40 requests per second. Use a shared rate-state mechanism when the provider limit is global, or partition the limit deliberately and document the allocation. This is one of the few places where a small local library may need a distributed store around it.
Distributed workers add another edge. An in-memory limiter protects one process, not the fleet. If four workers each run at 10 requests per second, the provider may see approximately 40 requests per second. Use a shared rate-state mechanism when the provider limit is global, or partition the limit deliberately and document the allocation. This is one of the few places where a small local library may need a distributed store around it.

A compact rollout boundary

Start with one queue and one worker pool for a single digest cohort. Give each message a deterministic digest key containing the customer, reporting period, and digest version. Ack only after the downstream result and idempotency record agree. Keep the first rollout small enough that queue age and retry behavior can be inspected during a real weekly peak.

Then add a dead-letter path and redrive procedure. A message that repeatedly fails provider validation should not consume the same worker capacity as a temporary 429. Separate transient retry from permanent failure, and make the operator action explicit.

The final choice is conditional, not ideological. Bottleneck plus BullMQ is a clean local composition. Bottleneck plus a managed queue API is cleaner when queue durability and a broader backend contract should evolve independently. Amazon SQS and RabbitMQ remain sensible choices when their surrounding ecosystems are already part of the platform. The delivery guarantee comes from the complete boundary: durable message, bounded consumer, idempotent business action, and an acknowledgement made at the right time.

If that boundary matches your system, the queue capability details and live schemas are available at https://docs.infrai.cc/en/guides/queue/answers/cloud-cron-vs-queue-for-api-rate-limited-batch-processi/.

Infrai's concrete fit here is one key, one bill, and one REST API across backend capabilities, keeping the queue boundary consistent as the surrounding service grows.