Consider a service running with two pods. It has an HPA configured to add a pod when CPU utilization reaches 80%. This helps the application scale as CPU demand grows. But HPA does not choose a pod for each individual request: a request can be routed to any available replica.
The problem appears when several users start a long-running request at almost the same time and all their requests land on the same pod. For example, users may run reports that hold a database connection for several minutes. Kubernetes does not account for how many HikariCP connections are occupied in each pod, so requests can all reach one pod and exhaust its pool—even while the other pod still has free connections.
If each pod’s pool is limited to five connections, five reports running at the same time can occupy every connection in one replica. New requests to that pod will wait for a connection to become available. Depending on the application’s probes and failure handling, the pod may also become unhealthy or overloaded. CPU-based scaling may not help: database-pool pressure does not always cause CPU usage to rise, and HPA changes the number of replicas—it does not route incoming requests according to HikariCP pool usage.
What can you do?
Limit concurrency. Allow a fixed number of resource-intensive operations to run at once, and queue or reject the rest with a clear response. A per-pod limit protects each local pool, but a global limit must account for all application replicas.
Use a queue and background workers. The API accepts a task and returns its ID; workers run reports with controlled concurrency. This lets you scale report processing independently from user-facing requests. The trade-off is the need to manage task status, retries, failures, and possible duplicate execution.
Scale the pods. Additional replicas can help when demand grows consistently and the database can handle more concurrent queries. HPA scales the number of pods, but it does not select the least-loaded replica for each request. More pods can also increase the total number of potential database connections.
Route based on load. A load balancer or another routing layer can take active requests or connection-pool metrics into account. This is more complex than using a standard Kubernetes Service and depends on timely metrics. A request that is already running cannot be moved to another pod.
Isolate reporting workloads. A separate read replica or reporting database can reduce the impact of heavy reads on the primary workload. This adds cost and requires deciding how much data replication lag is acceptable.
| Approach | Benefit | Trade-off |
|---|---|---|
| Concurrency limits | Protect the pool and database from overload | Some requests wait or are rejected |
| Queue and workers | Controlled execution and independent scaling | Additional infrastructure and failure handling |
| Pod scaling | More application capacity during sustained demand | Does not guarantee even routing and may increase database load |
| Load-aware routing | New requests can be sent to a less busy pod | Complexity, metric delay, and risk of choosing the wrong pod |
| Query optimization or read isolation | Reduces execution time or separates reporting reads | Requires changes to queries or infrastructure |
Why not simply increase the pool size?
Increasing the HikariCP pool size may reduce waiting inside the application, but it does not necessarily increase the system’s throughput. If the database is already close to its limits, more concurrent queries can increase contention for CPU, storage, and locks. The queue may simply move from the application to the database, where it is harder to control.
Before increasing the pool size, check how much concurrency the database can actually handle and account for the total number of connections across all pods.
Choosing an approach
If resource-intensive operations are rare, start by measuring the bottleneck and limiting concurrency. If reports run regularly and take several minutes, a queue and dedicated workers usually provide more predictable control. If a specific query is the problem, optimize it or isolate reporting reads first. Add scaling or load-aware routing when metrics show that those changes address the actual bottleneck.
A long-running request can occupy more than a database connection: it may also hold a thread, an external API slot, or another limited resource. Even traffic distributed evenly across pods does not guarantee that the system will remain healthy. The important questions are which resource is occupied, how long it is held, and where the queue should form—in a controlled place rather than unpredictably inside the system.
