Spatial Dashboard Streamlit · Panel · Production
Deployment & Ops

Tuning Kubernetes HPA for Bursty Spatial Dashboard Sessions

Scale a Streamlit or Panel dashboard on a custom active_sessions metric with an asymmetric behavior block — fast scale-up, slow scale-down — instead of CPU, so bursty session traffic is absorbed by warm pods and lulls never disconnect users mid-map.

Why this matters

Spatial dashboard traffic is spiky in a way that defeats naive CPU autoscaling. A monday-morning report drop or a shared dashboard link can open 150 WebSocket sessions in under a minute, and each of those users then sits reading a choropleth for ten minutes, generating almost no CPU. If the HorizontalPodAutoscaler watches CPU, it scales up late (the burst is memory- and connection-bound, not CPU-bound) and then scales down while users are still connected, severing their sessions. The result is a dashboard that feels both slow at peak and unreliable in the trough. Getting the metric, the targets, and the stabilization windows right is what turns Kubernetes autoscaling & orchestration from a liability into a smooth capacity curve, and it depends directly on the per-pod state described in Session State Patterns.

Why session-based scaling beats CPU scaling for bursty dashboardsA timeline showing a sharp burst of active sessions. The CPU signal lags the burst and then drops during the reading lull, so CPU-based scaling both under-provisions the spike and reclaims pods too early. The active_sessions signal tracks connected users directly, so session-based scaling with a fast scaleUp window and a slow scaleDown window keeps enough warm pods for the whole session lifetime.timeloadactive_sessions — plateau while users readCPU — spikes then decaysburstCPU scale-down would drop live sessions here

Prerequisites

  • A dashboard already deployed per Kubernetes autoscaling & orchestration with a working Service, Ingress cookie affinity, and probes.
  • Kubernetes 1.27+, metrics-server, Prometheus, and Prometheus Adapter (or KEDA) so the autoscaling/v2 API can read a custom Pods metric.
  • prometheus-client>=0.20 in the dashboard image to emit the gauge.

Step-by-step solution

Step 1 — Pick a metric that reflects connected sessions

CPU tells you how hard pods are working right now; it says nothing about how many users would be disconnected if you removed a pod. For a stateful WebSocket dashboard the constraining resource is the connected session, so emit that count from every pod as a Prometheus gauge.

python
# session_metrics.py
from prometheus_client import Gauge, start_http_server
from streamlit.runtime import get_instance

ACTIVE_SESSIONS = Gauge(
    "spatial_active_sessions",
    "Live WebSocket-backed sessions currently held by this pod",
)

def refresh() -> None:
    runtime = get_instance()
    ACTIVE_SESSIONS.set(len(runtime._session_mgr.list_active_sessions()))

if __name__ == "__main__":
    start_http_server(9102)      # Prometheus scrapes :9102/metrics
    # a lightweight ticker keeps the gauge fresh between scrapes
    import threading, time
    def _loop():
        while True:
            refresh()
            time.sleep(5)
    threading.Thread(target=_loop, daemon=True).start()

For Panel, replace the session lookup with len(pn.state.session_info["sessions"]), which tracks the same connected-session set.

Step 2 — Set the target utilisation per pod

Do not guess the per-pod capacity — benchmark it. Load a single pod with held WebSocket sessions and watch p95 interaction latency until it crosses your acceptable threshold (say 400 ms for a pan or filter). If a pod comfortably serves 55 sessions before latency degrades, set the HPA target at roughly 70% of that, so the fleet has headroom for the reprojection and spatial-join spikes that momentarily slow every session.

yaml
    - type: Pods
      pods:
        metric:
          name: spatial_active_sessions
        target:
          type: AverageValue
          averageValue: "40"     # ~70% of a benchmarked 55-session ceiling

AverageValue divides the summed metric across current pods, so a target of 40 means “add a replica whenever the fleet averages more than 40 live sessions per pod.”

Step 3 — Configure stabilization windows and policies

This is where flapping is won or lost. A flat target with default behaviour oscillates on bursty traffic: the burst adds pods, the reading lull drops CPU, the controller removes pods, the next interaction spikes latency, and the cycle repeats. Break it with an asymmetric behavior block — react instantly to growth, retreat slowly.

yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: spatial-dashboard
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: spatial-dashboard
  minReplicas: 4
  maxReplicas: 24
  metrics:
    - type: Pods
      pods:
        metric:
          name: spatial_active_sessions
        target:
          type: AverageValue
          averageValue: "40"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0       # absorb the burst immediately
      policies:
        - type: Percent
          value: 100                      # allow doubling in one step
          periodSeconds: 30
        - type: Pods
          value: 4                        # or +4 pods, whichever is larger
          periodSeconds: 30
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 600     # wait 10 min through reading lulls
      policies:
        - type: Pods
          value: 1                        # shed at most 1 pod per 2 min
          periodSeconds: 120

The scaleDown.stabilizationWindowSeconds: 600 makes the controller take the highest recommended replica count over the last ten minutes, so a brief dip in sessions never triggers a shrink. The single-pod scale-down policy then drains capacity gently, giving affinity-bound sessions time to finish.

Reading a map looks exactly like demand leavingThe upper trace is the real session count over forty minutes: a burst at minute eight takes it from about eighty to three hundred and twenty sessions, after which it stays high but visibly ragged, because analysts spend a minute reading a map for every few seconds of panning and the metric dips each time. Below it, two replica traces respond to that signal. With default behaviour the controller follows every dip: it steps up to nine replicas, back to five, up to eight, back to four, six times in half an hour, and each of those scale-downs cuts affinity-bound sessions on the pod it removes. With the asymmetric block — no stabilization on the way up, six hundred seconds on the way down, one pod per two minutes — it reaches nine replicas just as fast and then holds, because the ten-minute maximum ignores the reading lulls; the single step down comes only once demand has actually left.FORTY MINUTES OF A BURSTY DASHBOARDactive sessionsburst — 80 → 320 sessionsthe ragged part is people reading, not leavingdefault behavioursix scale-downs in half an hour — each one cuts pinned sessionsasymmetric behavioursame climb, then held — the 10-minute maximum ignores every dipone step down, after demand really left

Step 4 — Set minReplicas for burst headroom

Even instant scale-up has latency: the metric must be scraped, the HPA must observe it, and the scheduler must place and start pods — 15 to 60 seconds during which a burst has nowhere to go unless warm pods already exist. Raise minReplicas so that headroom is standing by. If a typical burst adds 80 sessions and a pod holds 40, you want at least two spare warm pods above steady-state demand.

yaml
  minReplicas: 4      # steady-state ~2 pods of demand + 2 pods of burst headroom
The ninety seconds a new pod cannot help youA burst of eighty sessions arrives at time zero. The upper bar breaks down how long it takes for a newly scaled pod to accept any of them: up to fifteen seconds for the metric to be scraped, up to fifteen more for the HPA to evaluate it on its control loop, a few seconds to schedule, ten to twenty for the image pull if the layer is not already on the node, and then the same eight to twenty second geopandas import every cold pod pays — about ninety seconds before the first session can land. The lower bar is the same burst against two warm spare pods held by minReplicas: they are already running, already imported and already in the Service endpoints, so they absorb the burst in the first second. Scaling is what keeps you up for the rest of the afternoon; headroom is what covers the first minute and a half, and only one of the two can be bought after the burst arrives.A BURST OF 80 SESSIONS ARRIVES AT t=0scale up from herefirst sessionscrapeHPA loopschedimage pullgeopandas import≈ 90 s — and the burst does not wait, it queues on the pods you already hadminReplicas: 4 — two spare warm podsfirst session — already running, already imported, already in the endpointsScaling keeps you up for the rest of the afternoon; headroom covers the first ninety seconds. Only one of the two can bebought after the burst has already arrived.

Expose the metric to Kubernetes with a Prometheus Adapter rule so autoscaling/v2 can read it:

yaml
# prometheus-adapter values: rules.custom
rules:
  custom:
    - seriesQuery: 'spatial_active_sessions{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: {resource: namespace}
          pod: {resource: pod}
      name:
        as: "spatial_active_sessions"
      metricsQuery: 'sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)'

Verification

Confirm the HPA reads the session metric and reacts with the right asymmetry:

bash
kubectl get hpa spatial-dashboard --watch
# NAME                REFERENCE                     TARGETS      MINPODS  MAXPODS  REPLICAS
# spatial-dashboard   Deployment/spatial-dashboard  18/40        4        24       4
# ...after a burst of 200 sessions...
# spatial-dashboard   Deployment/spatial-dashboard  40/40        4        24       5

kubectl describe hpa spatial-dashboard | grep -A4 "Events:"
# Normal  SuccessfulRescale  30s  horizontal-pod-autoscaler  New size: 5; reason: pods metric spatial_active_sessions above target

Drive a burst with a WebSocket holder (200 connections held for five minutes), confirm replicas climb within a minute, then confirm they do not immediately fall when the burst ends — the 10-minute window should hold the fleet steady, and no held connection should drop during the eventual gradual scale-down.

Edge cases and gotchas

  • Scale-down killing active WebSockets: the stabilization window alone is not enough. Pair it with the preStop drain hook and terminationGracePeriodSeconds from the orchestration guide and Ingress cookie affinity, or a scale-down that is eventually warranted will still cut a live session when the pod finally terminates.
  • Metric lag: the loop is only as fast as its slowest link — a 30-second Prometheus scrape interval plus a 15-second HPA sync means the controller can be 45 seconds behind reality. Tighten the scrape interval for the exporter and keep minReplicas headroom to cover the lag; do not try to compensate by lowering the target, which just causes over-provisioning.
  • Thundering herd on cache warm-up: when new pods start during a burst, every fresh pod cold-loads the same base GeoDataFrame at once, spiking backend and memory simultaneously. Warm heavy layers from a shared cache or a read replica rather than the primary database, and stagger startup with a small jittered sleep so ten new pods do not hammer the source in the same 200 ms.

FAQ

Why is CPU a bad autoscaling metric for a spatial dashboard?

A Streamlit or Panel session stays connected over a WebSocket while the user reads the map, pans, and thinks. That connected-but-idle state consumes almost no CPU yet still holds a pod’s memory and a WebSocket slot. Scaling on CPU therefore removes pods that are still serving users and disconnects them, while under-provisioning the burst itself, which is connection- and memory-bound rather than CPU-bound. An active_sessions custom metric tracks the resource that actually constrains the pod.

How do I stop the HPA from flapping on bursty traffic?

Set behavior.scaleDown.stabilizationWindowSeconds to 300–600 so the controller waits through short lulls before shrinking, and cap the scale-down policy to one pod per period. Keep scaleUp responsive with a zero-second window and a generous Percent/Pods policy. The asymmetry lets the fleet grow fast for a burst but shrink slowly, which eliminates the oscillation a flat CPU target produces on spiky session traffic.

What minReplicas should I set for burst headroom?

Set minReplicas high enough that the warm fleet absorbs a typical surge during the 15–60 seconds the autoscaler needs to observe the metric and schedule new pods. If bursts routinely add 80 sessions and a pod holds 40, keep at least two to three spare pods of headroom above steady-state demand rather than scaling from a single idle replica — the extra warm capacity is far cheaper than a wave of disconnected users.

Back to Kubernetes Autoscaling & Orchestration

Related