Warm Pool vs KubeAPI

[u/doublecore20]

We have a debate at our workplace;

We're in the process of a big refactor of a monolithic project into micro services which will be deployed with k8s on EKS (and k8s on prem). We use Traefik as our gateway (important for option #2)

Our use-case is very specific and requires us to route a user to a specific pod which does a very user-specific isolated workload. The pod serves only 1 user at a time. When the workload ends - the worker must discarded (security requirement).

We have two options: 1. Use KubebAPI directly and spin up pods on demand. Assigning a label and routing by label with custom proxy. Allowing "native" scale per user request and delete when needed with manual monitoring also via KubeAPI.

1. Having a warm pool of "workers" with HPA for elasticity with custom metric for min available workers.. Managing worker's (workload pods) state in redis (ZSET for heartbeat and O(1) allocation). Each worker has a random unique ID assigned on startup. Traefik (our Gateway) can use Redis as external provider and can create HTTP routes dynamically based on worker state (worker allocated = heartbeat creates kv in redis and this triggers an HTTP route creation). This allows us to route the user to a pod by the unique ID (Traefik route to pod IP by worker ID). Monitoring is done by querying Redis.

Option #1 is simple, easy to implement and mostly to maintain (code wise) - but couples us with k8s (cannot be deployment agnostic), sounds like a total abuse of KubeAPI specifically at larger scale.

Option #2 is more complex theoretically, but it avoids using KubeAPI for application specific needs. Decouples infrastructure from application without some high privileged RBAC policies. Allowing the infrastructure to support the application based on custom metrics and load.

The question - is option #2 really over-engineering and using KubeAPI is not as bad as is sounds? (Controllers and Operators exist for a reason, but I am not sure they are used like that)

[u/FanZealousideal1511]

Could you elaborate how option 2 satisfies the "When the workload ends - the worker must discarded" requirement? Right not it seems like this is not covered. Are you planning to kill the process once the work is done, so that K8s restarts the container? In that case, the workload isolation is weaker than when manually scheduling pods.

I agree with the previous commenter, the solution depends on your performance targets. Maybe you can mix and match to get the best of both worlds: you can let K8s start the pods ("warm pool" model), but once the job is done, you could kill the specific pod via KubeAPI. This uses a much smaller K8s API surface, and at the same time allows you to respond faster to traffic surges (as you'd have reserved capacity), and will also enable better isolation (as the pods are not recycled).

[u/doublecore20 → u/FanZealousideal1511]

You got this correctly. When the work is done, simply send exit code 0. This triggers cleanup in the process and the container dies. K8s will restart it fresh with same IP but new context waiting for work. It does creates a strange situation where a pod with X amount of restarts is normal operation. But this can be mitigated by fine-tuning alerts by specific logs

The worker needs to be able to terminate itself because it acts a middle man between the user and the target service that it needs to use (like proxy)