Short answer: use an external recurring trigger to send one small, authenticated HTTP request to a cleanup endpoint, and make that endpoint claim a bounded time window before doing work. The trigger stays stateless. Your application does not need an internal task executor. The trade-off is clear: a shorter interval reduces cleanup latency but creates more request pressure, so the endpoint must cap concurrency, honor Retry-After, and make retries harmless.

For a B2B SaaS system draining a rate-limited worker pool, I would start with a five-minute tick and a one-minute cleanup window. That is a starting policy, not a universal constant. The right interval comes from the oldest acceptable dirty record and the request budget available to the service.

What is the simplest recurring HTTP method for bounded data cleanup triggers?

The recurring request should ask the service to advance a cleanup watermark, not ask it to process every stale row in one invocation. A scheduler sends a request such as POST /internal/cleanup-tick; the handler authenticates it, computes a window ending at the current time minus a small safety delay, and claims only records inside the next unprocessed slice. A separate worker pool can drain those claims at its normal rate.

That split matters. A timer is good at saying “try now.” It is a poor place to hold application state, coordinate multiple pages of data, or retry a half-finished batch. The API owns those semantics. A database row or lease records the window, its status, and an expiration time. If the HTTP request is repeated after a timeout, the same window can be observed or reclaimed instead of duplicated.

The safety delay absorbs clock skew and late writes. For example, if the current time is 12:05:00, a two-minute delay means the tick processes data through 12:03:00. A record arriving at 12:04:58 will wait for a later pass rather than racing the writer that created it.

Keep the first pass small. A five-minute schedule, a two-minute safety delay, and a one-minute window are enough to expose the contract without hiding it inside a large batch.

Keep it bounded.

A small Python boundary before the full system

This example keeps the scheduling contract separate from the cleanup implementation. It is intentionally boring: a clock, a bounded window, and an idempotency key. In production, claim_window should use a transaction and a unique constraint or lease so two requests cannot claim the same interval concurrently.

from dataclasses import dataclass from datetime import datetime, timedelta, timezone @dataclass(frozen=True) class CleanupWindow: start: datetime end: datetime key: str def next_window( last_end: datetime, now: datetime, width: timedelta = timedelta(minutes=1), safety_delay: timedelta = timedelta(minutes=2), ) -> CleanupWindow: """Build one bounded, repeatable cleanup request.""" cutoff = now - safety_delay end = min(last_end + width, cutoff) if end <= last_end: raise ValueError("cleanup is caught up") start = last_end key = f"cleanup:{start.isoformat()}:{end.isoformat()}" return CleanupWindow(start=start, end=end, key=key) window = next_window( last_end=datetime(2026, 10, 25, 12, 0, tzinfo=timezone.utc), now=datetime(2026, 10, 25, 12, 5, tzinfo=timezone.utc), ) print(window.key)

The endpoint should return a useful status, not a pretend success. A completed claim can return 202 Accepted when the worker pool will drain it asynchronously. A request that arrives while the pool is full can return 429 Too Many Requests and include Retry-After; the caller should then wait rather than hammering the same endpoint. A duplicate key can be treated as already accepted if the original claim is still visible.

How do latency and cost change the interval?

There are two clocks in this design. The trigger interval controls how often the service notices work. The worker pool controls how quickly claimed work leaves the system. Making the trigger run every minute cannot compensate for a pool that takes ten minutes to drain a backlog. It can make the backlog more expensive by adding contention and failed requests.

I initially thought the trigger interval was the main tuning knob. Later, the capacity equation changed my mind: a 5-minute tick is irrelevant if the pool needs 10 minutes to drain each 5-minute window. The watermark is the honest signal.

I use a simple decision rule: choose an interval comfortably below the maximum acceptable cleanup age, then verify that one tick’s window is smaller than the pool’s sustainable capacity during the interval. If a five-minute tick claims a five-minute window and the pool can process only three minutes of work in five minutes, the watermark will fall behind forever. Reduce the window, add capacity, or accept a slower freshness target.

A bounded window also gives evaluation something concrete to test. Generate records around the cutoff, run the handler twice with the same idempotency key, inject a timeout after the database commit, and send two ticks at once. The expected result is one logical claim, no skipped interval, and a later retry that advances the watermark. I care about this kind of harness because a scheduler can look healthy in a notebook while its failure behavior remains untested in production.

Do not use request duration as the only latency metric. Track trigger-to-claim delay, oldest unclaimed record age, window size, claim conflicts, worker drain time, response status, and retry count. Those measurements separate a late trigger from a saturated pool. They also show whether shorter intervals are buying freshness or just generating traffic.

One metric is enough to wake me up: the age of the oldest unclaimed record. Everything else explains why it moved.

That is enough.

Failure boundaries worth making explicit

Authentication belongs at the HTTP boundary. Use a narrowly scoped credential or signature, enforce a timestamp tolerance, and reject replays once a request identifier has been consumed. The cleanup operation itself still needs authorization checks; a valid scheduler request should not grant access to arbitrary tenant data.

Tenant isolation deserves its own test. A global tick can discover tenant work, but the claim query must retain the tenant identifier and apply per-tenant limits. One noisy customer should not occupy every worker slot. For a shared pool, a small fair-queue policy is usually more useful than an unlimited batch size.

The common trap is acknowledging before the claim is durable. Return success only after the transaction has recorded the window or lease. If the process dies before that point, a retry should be allowed to try again. If it dies after the commit but before the response, the idempotency key is what turns the retry into an observation instead of a second operation.

Here is the failure chain I would force through the test harness. At 12:05, the tick claims 12:00–12:01 and hands it to the pool. At 12:05.2, the handler commits but the connection drops before the response. The scheduler retries at 12:06. The second request must find the existing key, report that the window is already accepted, and avoid creating a second lease. If the first lease expires while a worker is still running, the lease policy must prevent two workers from deleting the same record, usually by making the delete itself conditional on the claim token. That is a longer path than the happy case, but it is where the design earns its reliability.

I would ship this with a short operational checklist written into the runbook: define the freshness target, record the last completed window, cap each claim, make claims idempotent, honor Retry-After, alert on watermark age, and provide a manual single-window replay. The replay should use the same code path and authorization rules as the recurring request. That keeps diagnosis from becoming a second scheduling system.

Sources