Spatial Dashboard Streamlit · Panel · Production
Data Reference

GeoParquet vs FlatGeobuf for Dashboard Caching

Pick GeoParquet when the read is narrowed by column or attribute — it skips both without touching them. Pick FlatGeobuf when the read is a bounding box against a large file on object storage, because its packed R-tree turns that into a few range requests and needs no server.

Why this matters

Both formats are modern, binary, self-describing and fast, and both are enormous improvements on a shapefile. That makes the choice between them feel arbitrary, and teams usually settle it by whichever they read about first. It is not arbitrary: they optimise different halves of the same problem, and the wrong choice for a given access pattern gives up most of the benefit while still looking like an upgrade.

The distinction reduces to one question — what narrows the read? If it is which columns and which attribute values, the columnar format wins, and it wins by a lot on a wide table. If it is which part of the map, the spatially indexed format wins, and it wins by a lot on a large file behind a network. A dashboard usually has both kinds of layer, which is why the honest answer is often “both, for different layers” rather than a house standard.

Bytes read, for three different questionsThe same nine hundred megabyte layer of four hundred thousand features with thirty attribute columns is stored in both formats and asked three different questions. Asked for three columns of the whole layer, GeoParquet reads about seventy megabytes because it touches only those column chunks, while FlatGeobuf reads the whole file because it is row-oriented. Asked for one region by attribute value on a partitioned dataset, GeoParquet reads about eleven megabytes by skipping every other partition, while FlatGeobuf again reads everything since it has no attribute index. Asked for every column of features inside a small bounding box, the ordering reverses: FlatGeobuf's packed Hilbert R-tree resolves the query to a handful of byte ranges and transfers about four megabytes, while GeoParquet reads the row groups whose statistics overlap the box — better than everything, but coarser, at about ninety. Three questions, and the winner changes twice.ONE 900 MB LAYER — BYTES ACTUALLY READ3 columns of the whole layerGeoParquet70 MBFlatGeobuf900 MB — row-oriented, so every column comes alongone region, by attribute, partitionedGeoParquet11 MB — whole partitions skippedFlatGeobuf900 MB — no attribute index at allall columns, one small bounding box, over HTTPGeoParquet90 MB — row groups that overlap, which is coarseFlatGeobuf4 MB — the packed R-tree resolves it to a few byte ranges

Prerequisites

  • geopandas>=1.0 with pyarrow>=14 for GeoParquet, and pyogrio for FlatGeobuf.
  • Object storage that honours HTTP range requests if you intend to read either remotely. Without it the FlatGeobuf index cannot help.
  • A layer whose access pattern you have actually observed, rather than assumed. The section below is about how to observe it.

Step-by-step solution

Step 1 — Name the access pattern before choosing

Log what the dashboard asks for over a normal working day: which columns, which attribute filters, which bounding boxes. The result usually falls into one of three shapes.

A reference layer — administrative boundaries, a road network — is read whole or by region, rarely filtered by attribute, and read by every session. A wide analytical table — parcels with thirty attributes — is read by a handful of columns at a time and filtered heavily. A large static backdrop — a national point cloud, a full building footprint set — is only ever read one viewport at a time and never whole.

The first two want GeoParquet. The third wants FlatGeobuf.

Step 2 — Write GeoParquet, partitioned by what you filter on

python
import geopandas as gpd

gdf.to_parquet(
    "s3://layers/parcels",
    partition_cols=["region"],           # one directory per region
    compression="zstd",
    geometry_encoding="WKB",
    write_covering_bbox=True,            # per-row-group bbox for spatial pruning
)

# The read that pays it back: two columns, one region, nothing else touched.
view = gpd.read_parquet(
    "s3://layers/parcels",
    columns=["parcel_id", "use_class", "geometry"],
    filters=[("region", "==", "camden")],
)

write_covering_bbox is worth knowing about: it stores a bounding box column so the reader can prune row groups spatially as well as by attribute. It does not turn Parquet into a spatially indexed format — the granularity is a row group, typically tens of thousands of rows — but it closes much of the gap for moderately sized layers.

Partitioning is the larger lever. A partitioned dataset lets the reader skip entire files by looking at the directory name, which costs nothing at all, whereas row-group pruning still opens the file to read its statistics.

Step 3 — Write FlatGeobuf for the viewport-at-a-time layers

python
gdf.to_file("buildings.fgb", driver="FlatGeobuf", engine="pyogrio")

# Read one viewport straight from object storage — no server involved.
view = gpd.read_file(
    "https://storage.example.com/layers/buildings.fgb",
    bbox=(-0.51, 51.28, 0.33, 51.69),
    engine="pyogrio",
)

The write sorts features by a Hilbert curve and builds a packed R-tree at the head of the file. That ordering is what makes the range reads efficient: features near each other in space end up near each other in the file, so a bounding-box query resolves to a small number of contiguous ranges rather than to thousands of scattered single-feature reads.

Step 4 — Verify that the read is actually partial

Both formats can silently degrade to a full read, and the symptom is identical to no optimisation at all.

python
import time, requests

url = "https://storage.example.com/layers/buildings.fgb"
head = requests.head(url, timeout=10)
assert head.headers.get("accept-ranges") == "bytes", "range requests unsupported"

t0 = time.perf_counter()
view = gpd.read_file(url, bbox=SMALL_BBOX, engine="pyogrio")
print(f"{len(view):,} features in {time.perf_counter()-t0:.2f}s")

If the elapsed time for a small bounding box is close to the time for the whole file, the range reads are not happening — usually because a CDN or proxy in front of the storage is stripping the Range header, which is a configuration problem rather than a format one.

Two different ways of not reading a fileOn the left, a partitioned GeoParquet dataset is drawn as a set of directories, one per region, of which one is selected and the rest are never opened. Inside the selected file, row groups are drawn as horizontal bands and columns as vertical stripes; the read touches the intersection of the row groups whose statistics overlap the filter and the three column chunks requested, leaving most of the file untouched. On the right, a FlatGeobuf file is drawn as a header containing a packed Hilbert R-tree followed by features ordered along that curve. A bounding-box query descends the tree, which is itself read with one small range request, and resolves to two contiguous byte ranges covering the features in the box — contiguous precisely because the Hilbert ordering put spatially near features next to each other on disk. The two mechanisms are orthogonal, which is why a layer that needs both narrowings is usually better served by storing it twice than by trying to make one format do the other's job.GEOPARQUET — SKIP PARTITIONS, THEN COLUMN CHUNKSregion=camdenthe others are never openedshaded = the row groups that overlap × the columns asked forFLATGEOBUF — DESCEND THE TREE, FETCH RANGESheader: packed Hilbert R-tree — one small range readfeatures, ordered along the Hilbert curveshaded = the two contiguous ranges the box resolves toContiguous because the ordering put spatiallynear features next to each other on disk —without that, the same query would bethousands of scattered single-feature reads.The two mechanisms are orthogonal — a layer needing both narrowings is usually better stored twice than forced into one. Three layer shapes, and which format each wantsNaming the layer shape settles the format faster than benchmarking does. A reference layer of administrative boundaries is read whole or by region, rarely filtered by attribute, and read by every session — GeoParquet, partitioned by region, so a session reads one partition. A wide analytical table of parcels with thirty attributes is read three columns at a time and filtered heavily — GeoParquet again, and here the columnar pruning is doing the work rather than the partitioning. A large static backdrop of building footprints is only ever read one viewport at a time and never whole — FlatGeobuf on object storage, where the packed R-tree turns each viewport into a few range requests and no server is involved. Two of the three want the same format for different reasons, which is why the reasoning matters more than the verdict.NAME THE LAYER SHAPE, AND THE FORMAT FOLLOWSreference boundaries — read whole or by regionGeoParquet, partitioned by region: a session reads one partition rather than the countrywide analytical table — 3 of 30 columnsGeoParquet, where the columnar pruning rather than the partitioning is doing the worklarge static backdrop — one viewport at a timeFlatGeobuf on object storage: the packed R-tree resolves each viewport to a few range requestsTwo of the three want the same format for different reasons, which is why the reasoning transfers to a new layerand a benchmark of one file does not.

Verification

python
# Round trip both, then assert nothing was lost.
for path, kwargs in [("layer.parquet", {}), ("layer.fgb", {"driver": "FlatGeobuf"})]:
    gdf.to_file(path, **kwargs) if kwargs else gdf.to_parquet(path)
    back = gpd.read_file(path) if kwargs else gpd.read_parquet(path)
    assert len(back) == len(gdf), path
    assert back.crs == gdf.crs, f"{path} lost the CRS"
    assert back.geometry.geom_equals(gdf.geometry).all(), f"{path} changed geometry"

Then measure the read you actually perform, not a generic one. A benchmark that reads both files whole tells you which decompresses faster and nothing about which format suits your dashboard.

Edge cases and gotchas

  • Categoricals through FlatGeobuf. OGR-backed formats have a fixed type system, so a pandas categorical comes back as a plain string column and quietly costs several times the memory. GeoParquet preserves it exactly. If dtype fidelity matters, that alone decides it.
  • Very many small files. Partitioning GeoParquet by a high-cardinality column produces thousands of tiny files, and the per-file overhead on object storage then dominates. Partition by something coarse — region, month — not by identifier.
  • Mixed geometry types. Both formats accept them; FlatGeobuf records a single geometry type in its header and writing a mixed frame can produce a file some readers reject. Split by type or declare it generic explicitly.
  • Appending. Neither format is designed for it. Rewrite the partition rather than appending, and treat the cached file as immutable output of a build step.

FAQ

Can I just use GeoParquet for everything?

Usually, yes — and it is a defensible house standard. What you give up is the case where a browser or a lightweight client reads one viewport out of a very large file straight from a bucket, which FlatGeobuf does with no server and Parquet does coarsely. If that case does not exist in your dashboard, one format is simpler than two.

Does FlatGeobuf compress?

Not internally. The file is compact because it is binary and tightly packed, but there is no compression codec, so a FlatGeobuf file is typically larger than the equivalent Zstd-compressed GeoParquet. Transport compression will still apply if the server offers it, though that removes the ability to serve a byte range — which is the trade you would be making against the format’s main advantage.

Which should I convert my shapefiles to?

GeoParquet, unless the layer is one of the large viewport-at-a-time backdrops. It preserves dtypes and the CRS, has no ten-character column-name limit, compresses well, and is read directly by GeoPandas. The conversion is a single read and write, and it is worth doing on ingestion rather than on every load.

Back to Vector Tile & File Formats Reference.