Spatial Dashboard Streamlit · Panel · Production
Deployment & Ops

Tuning Kubernetes HPA for Bursty Spatial Dashboard Sessions

Scale a Streamlit or Panel dashboard on a custom active_sessions metric with an asymmetric behavior block — fast scale-up, slow scale-down — instead of CPU, so bursty session traffic is absorbed by warm pods and lulls never disconnect users mid-map.

Why this matters

Spatial dashboard traffic is spiky in a way that defeats naive CPU autoscaling. A monday-morning report drop or a shared dashboard link can open 150 WebSocket sessions in under a minute, and each of those users then sits reading a choropleth for ten minutes, generating almost no CPU. If the HorizontalPodAutoscaler watches CPU, it scales up late (the burst is memory- and connection-bound, not CPU-bound) and then scales down while users are still connected, severing their sessions. The result is a dashboard that feels both slow at peak and unreliable in the trough. Getting the metric, the targets, and the stabilization windows right is what turns Kubernetes autoscaling & orchestration from a liability into a smooth capacity curve, and it depends directly on the per-pod state described in Session State Patterns.

Why session-based scaling beats CPU scaling for bursty dashboardsA timeline showing a sharp burst of active sessions. The CPU signal lags the burst and then drops during the reading lull, so CPU-based scaling both under-provisions the spike and reclaims pods too early. The active_sessions signal tracks connected users directly, so session-based scaling with a fast scaleUp window and a slow scaleDown window keeps enough warm pods for the whole session lifetime.timeloadactive_sessions — plateau while users readCPU — spikes then decaysburstCPU scale-down would drop live sessions here

Prerequisites

  • A dashboard already deployed per Kubernetes autoscaling & orchestration with a working Service, Ingress cookie affinity, and probes.
  • Kubernetes 1.27+, metrics-server, Prometheus, and Prometheus Adapter (or KEDA) so the autoscaling/v2 API can read a custom Pods metric.
  • prometheus-client>=0.20 in the dashboard image to emit the gauge.

Step-by-step solution

Step 1 — Pick a metric that reflects connected sessions

CPU tells you how hard pods are working right now; it says nothing about how many users would be disconnected if you removed a pod. For a stateful WebSocket dashboard the constraining resource is the connected session, so emit that count from every pod as a Prometheus gauge.

python
# session_metrics.py
from prometheus_client import Gauge, start_http_server
from streamlit.runtime import get_instance

ACTIVE_SESSIONS = Gauge(
    "spatial_active_sessions",
    "Live WebSocket-backed sessions currently held by this pod",
)

def refresh() -> None:
    runtime = get_instance()
    ACTIVE_SESSIONS.set(len(runtime._session_mgr.list_active_sessions()))

if __name__ == "__main__":
    start_http_server(9102)      # Prometheus scrapes :9102/metrics
    # a lightweight ticker keeps the gauge fresh between scrapes
    import threading, time
    def _loop():
        while True:
            refresh()
            time.sleep(5)
    threading.Thread(target=_loop, daemon=True).start()

For Panel, replace the session lookup with len(pn.state.session_info["sessions"]), which tracks the same connected-session set.

Step 2 — Set the target utilisation per pod

Do not guess the per-pod capacity — benchmark it. Load a single pod with held WebSocket sessions and watch p95 interaction latency until it crosses your acceptable threshold (say 400 ms for a pan or filter). If a pod comfortably serves 55 sessions before latency degrades, set the HPA target at roughly 70% of that, so the fleet has headroom for the reprojection and spatial-join spikes that momentarily slow every session.

yaml
    - type: Pods
      pods:
        metric:
          name: spatial_active_sessions
        target:
          type: AverageValue
          averageValue: "40"     # ~70% of a benchmarked 55-session ceiling

AverageValue divides the summed metric across current pods, so a target of 40 means “add a replica whenever the fleet averages more than 40 live sessions per pod.”

Step 3 — Configure stabilization windows and policies

This is where flapping is won or lost. A flat target with default behaviour oscillates on bursty traffic: the burst adds pods, the reading lull drops CPU, the controller removes pods, the next interaction spikes latency, and the cycle repeats. Break it with an asymmetric behavior block — react instantly to growth, retreat slowly.

yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: spatial-dashboard
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: spatial-dashboard
  minReplicas: 4
  maxReplicas: 24
  metrics:
    - type: Pods
      pods:
        metric:
          name: spatial_active_sessions
        target:
          type: AverageValue
          averageValue: "40"
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0       # absorb the burst immediately
      policies:
        - type: Percent
          value: 100                      # allow doubling in one step
          periodSeconds: 30
        - type: Pods
          value: 4                        # or +4 pods, whichever is larger
          periodSeconds: 30
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 600     # wait 10 min through reading lulls
      policies:
        - type: Pods
          value: 1                        # shed at most 1 pod per 2 min
          periodSeconds: 120

The scaleDown.stabilizationWindowSeconds: 600 makes the controller take the highest recommended replica count over the last ten minutes, so a brief dip in sessions never triggers a shrink. The single-pod scale-down policy then drains capacity gently, giving affinity-bound sessions time to finish.

Step 4 — Set minReplicas for burst headroom

Even instant scale-up has latency: the metric must be scraped, the HPA must observe it, and the scheduler must place and start pods — 15 to 60 seconds during which a burst has nowhere to go unless warm pods already exist. Raise minReplicas so that headroom is standing by. If a typical burst adds 80 sessions and a pod holds 40, you want at least two spare warm pods above steady-state demand.

yaml
  minReplicas: 4      # steady-state ~2 pods of demand + 2 pods of burst headroom

Expose the metric to Kubernetes with a Prometheus Adapter rule so autoscaling/v2 can read it:

yaml
# prometheus-adapter values: rules.custom
rules:
  custom:
    - seriesQuery: 'spatial_active_sessions{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: {resource: namespace}
          pod: {resource: pod}
      name:
        as: "spatial_active_sessions"
      metricsQuery: 'sum(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)'

Verification

Confirm the HPA reads the session metric and reacts with the right asymmetry:

bash
kubectl get hpa spatial-dashboard --watch
# NAME                REFERENCE                     TARGETS      MINPODS  MAXPODS  REPLICAS
# spatial-dashboard   Deployment/spatial-dashboard  18/40        4        24       4
# ...after a burst of 200 sessions...
# spatial-dashboard   Deployment/spatial-dashboard  40/40        4        24       5

kubectl describe hpa spatial-dashboard | grep -A4 "Events:"
# Normal  SuccessfulRescale  30s  horizontal-pod-autoscaler  New size: 5; reason: pods metric spatial_active_sessions above target

Drive a burst with a WebSocket holder (200 connections held for five minutes), confirm replicas climb within a minute, then confirm they do not immediately fall when the burst ends — the 10-minute window should hold the fleet steady, and no held connection should drop during the eventual gradual scale-down.

Edge cases and gotchas

  • Scale-down killing active WebSockets: the stabilization window alone is not enough. Pair it with the preStop drain hook and terminationGracePeriodSeconds from the orchestration guide and Ingress cookie affinity, or a scale-down that is eventually warranted will still cut a live session when the pod finally terminates.
  • Metric lag: the loop is only as fast as its slowest link — a 30-second Prometheus scrape interval plus a 15-second HPA sync means the controller can be 45 seconds behind reality. Tighten the scrape interval for the exporter and keep minReplicas headroom to cover the lag; do not try to compensate by lowering the target, which just causes over-provisioning.
  • Thundering herd on cache warm-up: when new pods start during a burst, every fresh pod cold-loads the same base GeoDataFrame at once, spiking backend and memory simultaneously. Warm heavy layers from a shared cache or a read replica rather than the primary database, and stagger startup with a small jittered sleep so ten new pods do not hammer the source in the same 200 ms.

FAQ

Why is CPU a bad autoscaling metric for a spatial dashboard?

A Streamlit or Panel session stays connected over a WebSocket while the user reads the map, pans, and thinks. That connected-but-idle state consumes almost no CPU yet still holds a pod’s memory and a WebSocket slot. Scaling on CPU therefore removes pods that are still serving users and disconnects them, while under-provisioning the burst itself, which is connection- and memory-bound rather than CPU-bound. An active_sessions custom metric tracks the resource that actually constrains the pod.

How do I stop the HPA from flapping on bursty traffic?

Set behavior.scaleDown.stabilizationWindowSeconds to 300–600 so the controller waits through short lulls before shrinking, and cap the scale-down policy to one pod per period. Keep scaleUp responsive with a zero-second window and a generous Percent/Pods policy. The asymmetry lets the fleet grow fast for a burst but shrink slowly, which eliminates the oscillation a flat CPU target produces on spiky session traffic.

What minReplicas should I set for burst headroom?

Set minReplicas high enough that the warm fleet absorbs a typical surge during the 15–60 seconds the autoscaler needs to observe the metric and schedule new pods. If bursts routinely add 80 sessions and a pod holds 40, keep at least two to three spare pods of headroom above steady-state demand rather than scaling from a single idle replica — the extra warm capacity is far cheaper than a wave of disconnected users.

Back to Kubernetes Autoscaling & Orchestration

Related