Spatial Dashboard Streamlit · Panel · Production
Maps & Components

Handling Large GeoJSON Files in Leafmap Without Browser Lag

Never hand a raw multi-megabyte GeoJSON straight to the client — simplify the geometry, strip non-essential properties, cap the feature count, and cache the result on the server so the browser parses a small payload instead of freezing the main thread.

Why this matters

A spatial dashboard is only as responsive as its heaviest layer. When a user drops a 120 MB administrative-boundary file onto a map, the browser must synchronously parse the entire JSON string and create a DOM or canvas element for every vertex — work that blocks the UI thread and, on constrained tabs, triggers an out-of-memory kill. This is the single most common cause of “the map froze” bug reports in production Streamlit and Panel apps, and it sits at the heart of robust Folium & Leafmap Integration. The fix is architectural: move preprocessing to the server, send the browser only what is visible and simplified, and lean on the broader rendering and state patterns in Spatial Component Integration & Interactive Maps. Because the optimized payload is deterministic for a given input, it is an ideal candidate for the framework caching covered under Caching Strategies & Async Performance Tuning.

Why raw GeoJSON freezes the browser

Leafmap wraps ipyleaflet and folium, which ultimately delegate rendering to Leaflet.js. When a GeoJSON payload exceeds ~15–20 MB, three bottlenecks trigger:

  1. Synchronous JSON parsing blocks the main thread, freezing UI responsiveness until the entire string is deserialized.
  2. DOM node explosion occurs as Leaflet creates SVG/Canvas elements for every coordinate and feature. Chromium and WebKit enforce a ~2 GB per-tab memory ceiling, which large geometries quickly exhaust.
  3. Redundant coordinate precision retains millimeter-level floats that provide zero visual benefit at dashboard zoom levels, inflating payload size by 30–50%.

The GeoJSON specification (RFC 7946) explicitly recommends simplification and coordinate reduction for web delivery. Ignoring this forces the browser to parse and paint geometries that will never be visible at the current viewport scale.

The diagram below contrasts the naive path — handing raw GeoJSON straight to Leaflet.js — with the server-side pipeline that keeps the browser’s main thread responsive.

Raw GeoJSON versus server-side optimization pipelineTwo horizontal paths from a 120 MB source GeoJSON. The top path sends raw data directly to Leaflet.js, blocking the main thread with synchronous parsing, DOM node explosion, and an out-of-memory risk. The bottom path runs server-side simplification, property stripping, an 80,000-feature cap, and framework caching, producing a small payload that renders without lag.SourceGeoJSON~120 MBRaw path — passes everything to the client ✕SynchronousJSON parseDOM nodeexplosionFrozen UImain thread blocked · OOM riskOptimized path — server preprocesses, browser only paintsSimplify (GEOS)+ strip propertiesCap 80k features@st.cache_dataResponsive Leafmapsmall payload · ~5–10 MB

Prerequisites

  • Python 3.9+ — required for the type hints and geopandas versions used below.
  • geopandas>=0.14 and shapely>=2.0 — Shapely 2.0 vectorizes the GEOS Douglas-Peucker simplification, making it fast enough to run inside a request.
  • leafmap>=0.30 — leafmap.foliumap for Folium-backed rendering, plus add_geojson() and add_tile_layer().
  • streamlit>=1.30 (or panel>=1.3) — to host the map and provide the cache decorator.
  • tippecanoe — only needed for the vector-tile path in Step 6; install it from your package manager (e.g. brew install tippecanoe).
  • One conceptual prerequisite: a working grasp of CRS normalization, because simplification tolerance is unit-sensitive — reproject to a single known CRS before you measure or simplify (see the prerequisites in Folium & Leafmap Integration).

Install the runtime dependencies:

bash
pip install "geopandas>=0.14" "shapely>=2.0" "leafmap>=0.30" streamlit

Step-by-step solution

Step 1 — Profile the payload before optimizing

Measure before you cut. The decision to simplify, cap, or switch to tiles depends on three numbers: feature count, on-disk size, and total vertices.

python
import geopandas as gpd
import os

def profile_geojson(path: str) -> dict:
    """Return the three metrics that drive the optimization decision."""
    gdf = gpd.read_file(path)
    vertices = int(gdf.geometry.apply(lambda g: len(g.exterior.coords)
                                      if g.geom_type == "Polygon" else
                                      sum(len(p.exterior.coords) for p in g.geoms)
                                      if g.geom_type == "MultiPolygon" else 0).sum())
    return {
        "features": len(gdf),
        "size_mb": round(os.path.getsize(path) / 1_048_576, 1),
        "vertices": vertices,
        "crs": str(gdf.crs),
    }

print(profile_geojson("us_counties_detailed.geojson"))
# {'features': 3221, 'size_mb': 118.4, 'vertices': 5_412_880, 'crs': 'EPSG:4326'}

A file with 3,221 features but 5.4 million vertices is a precision problem, not a feature-count problem — the win comes from simplification, not feature capping.

Step 2 — Normalize the CRS to WGS 84

GeoPandas simplify() measures tolerance in the units of the active CRS. Run it on a projected CRS such as EPSG:3857 (metres) and a degree-scale tolerance will either do nothing or destroy your geometry. Reproject to EPSG:4326 first.

python
def to_wgs84(gdf: gpd.GeoDataFrame) -> gpd.GeoDataFrame:
    if gdf.crs is None:
        raise ValueError("GeoDataFrame has no CRS; set one before simplifying")
    if gdf.crs.to_epsg() != 4326:
        gdf = gdf.to_crs(epsg=4326)
    return gdf

Step 3 — Simplify geometry and strip properties

Use topology-preserving Douglas-Peucker simplification, then drop every column the map does not render. A tolerance of 0.001 degrees is roughly 100 m at mid-latitudes — invisible at dashboard zoom levels but a 30–60% size reduction.

The reason CRS normalization (Step 2) has to come first is that simplify() reads its tolerance in whatever unit the active CRS uses. The same number means three completely different things depending on the projection:

simplify(tolerance=0.001) is interpreted in the units of the active CRSThree columns compare the effect of calling simplify with a tolerance of 0.001 in different coordinate reference systems. In EPSG:4326 the unit is degrees, so 0.001 is about 100 metres and produces a correct, useful simplification. In EPSG:3857 the unit is metres, so 0.001 is one millimetre and the geometry is effectively untouched, leaving the payload huge. The recommended workflow reprojects to EPSG:4326 first, then simplifies, which is the only column that yields the intended result.gdf.simplify(0.001, preserve_topology=True)one tolerance value · three different outcomesEPSG:4326unit = degrees0.001° ≈ 100 mvertices droppedcleanly✓ useful resultEPSG:3857unit = metres0.001 m = 1 mmgeometry barelychanges✕ payload stays hugeto_crs(4326)then simplifyunit known = degreestolerance behavespredictably✓ recommended
python
def slim_down(gdf: gpd.GeoDataFrame, tolerance: float = 0.001,
              keep: tuple = ("id", "name", "value")) -> gpd.GeoDataFrame:
    # Topology-preserving simplification (GEOS Douglas-Peucker)
    gdf = gdf.copy()
    gdf["geometry"] = gdf.geometry.simplify(tolerance, preserve_topology=True)
    # Keep only geometry + the columns the UI actually uses
    cols = ["geometry"] + [c for c in keep if c in gdf.columns]
    return gdf[cols]

Step 4 — Cap client features and cache the whole pipeline

Compose Steps 1–3 into a single cached function. The cache makes the expensive geopandas I/O and GEOS work run once per TTL instead of on every Streamlit rerun — exactly the kind of deterministic, reusable result that belongs under Caching Strategies & Async Performance Tuning.

python
import streamlit as st

@st.cache_data(ttl=3600, max_entries=5, show_spinner="Optimizing GeoJSON…")
def optimize_geojson(path: str, tolerance: float = 0.001,
                     max_features: int = 80_000) -> str:
    """Load → CRS-normalize → simplify → strip → cap → write. Returns cache path."""
    gdf = gpd.read_file(path)
    gdf = to_wgs84(gdf)
    if len(gdf) > max_features:
        # For unordered data, sample rather than head-truncate to avoid spatial bias
        gdf = gdf.sample(max_features, random_state=0).copy()
    gdf = slim_down(gdf, tolerance=tolerance)

    cache_dir = "/tmp/geojson_cache"
    os.makedirs(cache_dir, exist_ok=True)
    out = os.path.join(cache_dir, f"opt_{os.path.basename(path)}")
    gdf.to_file(out, driver="GeoJSON")
    return out

The ~80,000-feature cap exists because that is roughly where Leaflet’s per-feature DOM and event-handler overhead starts producing scroll jank, independent of payload size.

Step 5 — Render the optimized layer in Leafmap

Point Leafmap at the cached file and embed it. Because the heavy work is upstream, the map renders from a small payload.

python
import leafmap.foliumap as leafmap

st.title("Optimized Spatial Dashboard")
uploaded = st.file_uploader("Upload GeoJSON", type=["geojson"])

if uploaded:
    temp_path = os.path.join("/tmp", uploaded.name)
    with open(temp_path, "wb") as f:
        f.write(uploaded.getbuffer())

    optimized_path = optimize_geojson(temp_path, tolerance=0.001)

    m = leafmap.Map(center=[40.0, -100.0], zoom=4)
    m.add_geojson(optimized_path, layer_name="Optimized Features")
    m.to_streamlit(height=600)

For Panel, the same optimize_geojson() output feeds m.to_panel() (or assign the Folium map to a pn.pane.plot.Folium pane) and .servable().

Step 6 — Switch to vector tiles when GeoJSON is not enough

Past ~50 MB of simplified geometry, even a trimmed GeoJSON strains client memory. Pre-bake Mapbox Vector Tiles (MVT) with tippecanoe, which clips geometry per tile and per zoom level so the browser only paints the current viewport.

bash
tippecanoe -o tiles.mbtiles --drop-densest-as-needed --maximum-zoom=14 input.geojson

Serve the tiles from a lightweight HTTP server or object storage, then render the {z}/{x}/{y}.pbf endpoints:

python
m.add_tile_layer(
    url="https://your-server/tiles/{z}/{x}/{y}.pbf",
    name="Vector Tiles",
    attribution="Custom MVT",
)

For user-filtered or streaming data, an alternative is chunked delivery: split the dataset into 10k-feature batches and append them to the Leaflet layer progressively with L.geoJSON().addData(), keeping the main thread free between chunks.

Verification

Confirm the optimization actually shrank the payload and stayed within the feature cap before you ship:

python
before = profile_geojson("us_counties_detailed.geojson")
optimized_path = optimize_geojson("us_counties_detailed.geojson", tolerance=0.001)
after = profile_geojson(optimized_path)

assert after["features"] <= 80_000, "Feature cap was not enforced"
assert after["size_mb"] < before["size_mb"], "Payload did not shrink"
assert after["crs"] == "EPSG:4326", "Output is not in WGS 84"

reduction = 100 * (1 - after["size_mb"] / before["size_mb"])
print(f"{before['size_mb']} MB → {after['size_mb']} MB  ({reduction:.0f}% smaller)")
# Expected: 118.4 MB → 11.7 MB  (90% smaller)

A healthy result is a payload comfortably under 15 MB with the feature count at or below the cap. In the browser, open DevTools → Performance, record a map load, and confirm there is no long task (>50 ms) blocking the main thread during render.

Edge cases and gotchas

  • Simplifying in the wrong CRS: Running simplify(0.001) on a GeoDataFrame still in EPSG:3857 (metres) collapses sub-millimetre detail to nothing useful or leaves the geometry untouched. Always apply CRS normalization to EPSG:4326 (or express the tolerance in the projected unit) before Step 3 — this is the most common source of “my polygons vanished” bugs.
  • Topology gaps between adjacent polygons: preserve_topology=True prevents self-intersections within a geometry but does not keep shared borders between neighbouring features aligned, producing visible slivers. For coverage data (counties, watersheds), use a topology-aware tool such as mapshaper -simplify or TopoJSON instead of per-feature simplify().
  • Cache TTL serving stale geometry: @st.cache_data(ttl=3600) keys on the file path and arguments, not file contents. If a user re-uploads a changed file under the same name, the stale optimized copy is returned until the TTL expires. Include a content hash (e.g. hashlib.md5(open(path,'rb').read()).hexdigest()) in the cache key when inputs can change in place — the same cache-invalidation discipline covered in Caching Strategies & Async Performance Tuning.

FAQ

What is the maximum number of GeoJSON features Leafmap can render smoothly?

As a practical rule, keep client-side features under roughly 80,000 and the serialized payload under 15–20 MB. Beyond that, Leaflet creates a DOM/event handler per feature, which produces scroll jank and risks exhausting the per-tab memory ceiling. For larger datasets, enable canvas rendering, simplify more aggressively, or move to Mapbox Vector Tiles as shown in Step 6.

Why does simplify() distort my geometries?

simplify() interprets tolerance in the units of the active CRS. In a projected CRS such as EPSG:3857 (metres) a tolerance of 0.001 does almost nothing, while a degree-tuned tolerance can collapse projected geometry entirely. Reproject to EPSG:4326 before simplifying, or express the tolerance in the projected unit, and always pass preserve_topology=True to avoid self-intersections.

Should I use add_geojson or vector tiles for a 200 MB dataset?

Use vector tiles. At 200 MB the dataset is far past the point where even simplified GeoJSON fits comfortably in browser memory. Pre-bake Mapbox Vector Tiles with tippecanoe, serve the {z}/{x}/{y}.pbf endpoints, and render them with add_tile_layer(). Tiles clip geometry to the viewport and zoom level, so the browser only ever paints what is visible.

Back to Folium & Leafmap Integration

Related