Spatial Dashboard Streamlit · Panel · Production
Data Reference

Choosing a Spatial File Format for a Dashboard That Has to Load Fast

Every layer in a spatial dashboard is read from somewhere, and the format it is read from decides more about the dashboard’s performance than most of the code around it. A layer stored as a shapefile and a layer stored as GeoParquet are the same features; one of them takes four seconds and 900 MB to open and the other takes 300 milliseconds and reads only the columns the map draws. Nothing in the application distinguishes them.

This page is a working reference for the five formats a Python spatial dashboard actually encounters, organised by what each one is good at rather than by chronology. The organising question throughout is not “which format is best” — none of them is — but “which of these does the thing this layer needs”, where the things layers need are: being read partially, being read by column, being read by many readers at once, and being rendered by a browser without being parsed first.

What actually differs between them

Four properties separate these formats, and every performance difference downstream reduces to some combination of them.

Partial reads. Can a reader fetch the features in a bounding box without reading the whole file? A format with a spatial index and range-request support can; a format without one cannot, and a dashboard that filters to a viewport still pays for the whole country.

Columnar layout. Can a reader fetch two columns without reading the other thirty? Columnar formats make column pruning free, which as choosing dtypes shows is often the largest single reduction available.

Self-describing metadata. Does the file carry its own CRS, its own schema, and its own types? A format that does removes an entire class of the failures described in CRS & coordinate systems; one that does not makes every reader guess.

Client renderability. Can a browser draw it without a parse step that blocks the main thread? This is the property that separates tile formats from data formats, and it is why the answer for a 400,000-feature layer is never “send the file”.

What each format is actually good atFive formats are laid out against the four properties that decide dashboard performance. GeoJSON has none of them — no spatial index, no columnar layout, coordinates as decimal text, and a browser parse that blocks the main thread — but it is accepted everywhere, which is why it survives as an interchange format and not as a storage one. The shapefile has no partial read, no columnar layout, a ten-character column-name limit and a coordinate system stored in a separate file that is frequently lost, so its metadata is only nominally self-describing. FlatGeobuf carries a packed Hilbert R-tree, so a reader can fetch one bounding box over an HTTP range request without downloading the file, and it is self-describing — but it is row-oriented, so reading two columns still reads all of them. GeoParquet adds the columnar layout on top of self-describing metadata and row-group statistics that support partial reads, which makes it the strongest of the five for a cached layer. Vector tiles invert the question entirely: they are not a data format but a rendering format, pre-clipped per tile and per zoom, unreadable as a dataset and drawable by a browser without any parse of the underlying features.FIVE FORMATS, FOUR PROPERTIESpartial readcolumnarself-describingbrowser draws itGeoJSONaccepted everywhereshapefileCRS in a separate fileFlatGeobufrange requests, no serverGeoParquetthe default for a cachevector tilesa render format, not dataRead the last row carefully: vector tiles are strong on exactly the property the others lack and useless for everythingelse, which is why they sit beside a data format rather than replacing one.

Prerequisites

  • geopandas>=1.0, pyarrow>=14 for GeoParquet, and fiona>=1.9 or pyogrio for the OGR-backed formats.
  • rasterio>=1.3 if the dashboard touches imagery, for the Cloud-Optimized GeoTIFF section.
  • tippecanoe if you intend to bake vector tiles. It is a command-line tool rather than a Python library, and it is the standard answer.
  • A clear idea of which layers are reference data that changes rarely and which are result data produced per request. The right format is almost always different for the two.

The five formats

GeoJSON — the interchange format, and only that

GeoJSON is text. Every coordinate is a decimal number written out in full, every property name is repeated on every feature, and there is no index of any kind. A 400,000-feature layer is tens of megabytes before compression and takes seconds to parse in a browser, during which nothing on the page responds.

None of that makes it a bad format. It makes it an interchange format: it is readable by every tool, debuggable in a text editor, and the only thing a browser mapping library will accept without a plugin. Use it at the boundary — the response a map component consumes, the export an analyst downloads — and do not use it as storage. The single most effective thing you can do to a GeoJSON payload is reduce the coordinate precision, as converting Shapely geometries to GeoJSON covers: six decimals is finer than any screen pixel, and a float repr emits fifteen.

Shapefile — read it, do not write it

The shapefile persists because thirty years of public data is published in it. It is worth knowing three things about reading one. It is several files that must travel together, and the .prj holding the coordinate system is the one most often missing — which produces a frame with crs=None and every downstream predicate silently empty. Column names are limited to ten characters, so population_2021 arrives as populatio. And there is no reliable text encoding declaration, so non-ASCII names arrive mangled unless you specify one.

python
import geopandas as gpd

gdf = gpd.read_file("boundaries.shp", encoding="utf-8")
if gdf.crs is None:
    gdf = gdf.set_crs("EPSG:27700")     # the .prj was missing — declare it explicitly
gdf = gdf.to_crs("EPSG:4326")

Convert on ingestion and never write one. Anything you would use a shapefile to store is better stored as GeoParquet, and the conversion is one line.

FlatGeobuf — one file, spatially indexed, no server

FlatGeobuf is a binary, self-describing format with a packed Hilbert R-tree at the front of the file. That index is the interesting part: a reader that can issue HTTP range requests can fetch the index, work out which byte ranges hold the features in a bounding box, and fetch only those — so a bounding-box read of a two-gigabyte file on object storage transfers a few hundred kilobytes and needs no server at all.

python
# Reading only what a viewport needs, straight from object storage.
url = "https://storage.example.com/layers/parcels.fgb"
viewport = gpd.read_file(url, bbox=(-0.51, 51.28, 0.33, 51.69))

That makes it an excellent fit for large static reference layers served from a bucket: no PostGIS, no tile server, no application code between the browser’s request and the bytes. Its limitation is that it is row-oriented, so reading two columns still reads every column of the matching features.

GeoParquet — the default for anything you cache

GeoParquet is Parquet with a metadata convention for geometry, which means it inherits everything Parquet already does well: columnar storage, per-column compression, row-group statistics that let a reader skip whole blocks, and a schema that carries dtypes and the CRS. For a cached dashboard layer it is the strongest of the five, and the reason is column pruning rather than raw size — a frame with thirty attribute columns of which the map draws three reads a fraction of the file.

python
# Read three columns of one region, skipping the rest of the file entirely.
gdf = gpd.read_parquet(
    "s3://layers/parcels.parquet",
    columns=["parcel_id", "category", "geometry"],
    filters=[("region", "==", "camden")],       # row-group pruning
)

Partition large layers by the column you filter on most, usually region or date. A partitioned dataset lets the reader skip entire files rather than entire row groups, and the difference on a national dataset is between reading four hundred megabytes and reading eleven.

Vector tiles — a rendering format that is not a dataset

Vector tiles invert the problem. Rather than sending features and letting the client work out what to draw, the data is pre-clipped into a pyramid of tiles, one per zoom level and grid square, with geometry simplified appropriately for each zoom. The browser fetches the handful of tiles covering the viewport and draws them; it never sees the underlying dataset and never parses it.

bash
tippecanoe -o parcels.mbtiles \
  --maximum-zoom=14 --minimum-zoom=6 \
  --drop-densest-as-needed \
  --extend-zooms-if-still-dropping \
  parcels.geojson

The trade is that a tile is not a dataset. You cannot query it, join to it, or recover the original geometry from it, and rebaking after a data change takes minutes. Tiles are the right answer above roughly fifty megabytes of already-simplified geometry, where no amount of payload trimming will make a whole-layer transfer acceptable — and the wrong answer for anything that changes per request, because the pyramid is a build artefact.

Cloud-Optimized GeoTIFF — the raster equivalent of a range request

A COG is an ordinary GeoTIFF whose internal layout — tiled rather than striped, with overviews — is arranged so a reader can fetch one window over HTTP range requests without downloading the file. For a dashboard showing imagery this is the difference between a viewport read costing a few hundred kilobytes and costing the whole scene.

python
import rasterio
from rasterio.windows import from_bounds

with rasterio.open("https://storage.example.com/imagery/2026-05.tif") as src:
    window = from_bounds(*bbox_3857, transform=src.transform)
    tile = src.read(1, window=window, out_shape=(512, 512))   # overview level chosen for you

Passing out_shape matters: it lets rasterio serve the request from an overview rather than from full resolution, which is what makes a zoomed-out view cheap. A COG read without it fetches full-resolution pixels and downsamples them locally, which is correct and slow.

Three questions that pick the formatThe tree starts by asking whether the layer is produced per request or stored. Per-request results go to GeoJSON if a browser consumes them directly and to Parquet bytes if they are being cached, because a cache entry is never parsed by a browser and gains nothing from text. Stored layers are asked their size after simplification: under fifty megabytes they go to GeoParquet, which gives column pruning and row-group skipping and is read directly by GeoPandas; over fifty megabytes the question becomes who reads them. A browser reading directly wants vector tiles, pre-clipped per zoom, because no whole-layer transfer will be acceptable at that size. A Python process reading a viewport at a time wants FlatGeobuf on object storage, whose packed R-tree turns a bounding-box read into a range request and needs no server. Imagery leaves the tree at the first branch and goes to Cloud-Optimized GeoTIFF, for the same reason FlatGeobuf exists on the vector side.PICKING A FORMAT IN THREE QUESTIONSproduced per request, or stored?per requestto a browser → GeoJSON, 6 decimalsto a cache → Parquet bytes, never textstoredover ~50 MB simplified?noGeoParquetcolumns + row groupsyes — who reads it?FlatGeobufPython, per viewportvector tilesbrowser, directlyImagery leaves at the first branch and goes to Cloud-Optimized GeoTIFF — the same range-request idea, applied to pixels. What conversion costs, one wayThe same four hundred thousand feature layer is converted from a shapefile into each target format and the write time and resulting size recorded. GeoJSON takes about fifty-one seconds and produces nine hundred and forty megabytes, because every coordinate becomes decimal text and every property name is repeated per feature. FlatGeobuf takes about nine seconds and produces four hundred and ten megabytes, with the Hilbert sort and R-tree build accounting for most of the time. GeoParquet with Zstd takes about seven seconds and produces one hundred and sixty-two megabytes. Vector tiles take about four minutes, because tippecanoe is clipping and simplifying the layer once per zoom level — which is precisely why tiles are a build artefact and not something to generate per request.ONE 400,000-FEATURE LAYER — CONVERSION TIME AND RESULTGeoJSON51 s940 MBFlatGeobuf9 s · 410 MBGeoParquet + Zstd7 s · 162 MBvector tiles (z6–z14)4 mina build artefact, not a responseThe tile row is the reason tiles are baked rather than generated: four minutes is fine in a pipeline and impossiblein a request.

Converting between them safely

Conversion is where metadata is lost, and the losses are silent. Four rules cover almost every case.

Set the CRS before writing, never after. A frame written without one produces a file that every reader has to guess about, and the guess is usually EPSG:4326 whether or not that is true.

Check validity before writing to a binary format. GeoJSON will happily carry a self-intersecting polygon; some binary writers will refuse it, and others will write it and produce a file that fails to read. Repair first, as converting Shapely geometries describes.

Preserve dtypes deliberately. A round trip through GeoJSON turns every number into a float and every category into a string, because JSON has no other option. Round-tripping through Parquet preserves them exactly, which is one more reason to keep GeoJSON at the boundary.

Record what the conversion changed. A pipeline that simplifies, reprojects and prunes columns on the way into a cache is producing a derived dataset, and six months later somebody will ask why the totals differ from the source. Write the tolerance, the EPSG code and the column list into the file’s metadata, where the answer travels with the data.

Troubleshooting

gdf.crs is None after reading a shapefile. The .prj is missing from the bundle. Declare the CRS explicitly from the data’s documentation — do not guess EPSG:4326 because the numbers look like degrees, since British National Grid eastings also look plausible if you are not paying attention.

A GeoParquet file written by one library will not open in another. The geometry metadata convention has versions. Write with a recent GeoPandas and read with a recent GeoPandas, and if a third-party tool must consume it, check which version of the specification it supports before assuming the file is at fault.

A FlatGeobuf bounding-box read downloads the whole file. The server does not support HTTP range requests, or the URL is being served through something that strips the Range header. Range support is the entire point of the format; without it, it is merely a compact file.

Vector tiles show gaps between features at low zoom. Tippecanoe dropped features to fit the tile size limit. --drop-densest-as-needed is doing its job; if the gaps matter, raise the maximum tile size or aggregate the layer server-side before baking rather than expecting the tiler to solve a data-density problem.

A COG read is slow and transfers everything. The file is a GeoTIFF but not cloud-optimized — no internal tiling, no overviews. rio cogeo validate will say so, and rio cogeo create will fix it in one pass.

Performance considerations

The decision that matters most is not which format but whether the read is partial. A dashboard that reads one viewport at a time from a partitioned GeoParquet dataset or an indexed FlatGeobuf on object storage will outperform one that loads a whole layer from any format, and the gap widens with the size of the dataset rather than staying constant.

The second decision is where the simplification happens. A layer simplified once at write time into per-zoom tiers costs nothing at read time; a layer simplified per request pays for it on every interaction, and the result is identical. This is the same reasoning that produces the tiered cache in caching strategies and it applies just as directly to storage.

Finally, measure the format change rather than assuming it. Converting a shapefile to GeoParquet reliably improves read time; converting a small GeoJSON to FlatGeobuf often does not, because at a few thousand features the index costs more to read than the scan it replaces. The formats on this page are tools for specific problems, and a layer that does not have the problem does not need the tool.

Back to Spatial Data Reference.