Efficient GDAL Workflows: Reading Zipped Archives In-Place

Geodata rarely arrives as a single file. A Shapefile is at least three files (.shp, .shx, .dbf) and usually a .prj, so users, survey contractors, and public data portals zip them. Your upload endpoint receives a .zip, hands it to GDAL, and gets this:

ERROR 4: `roads.zip' not recognized as being in a supported file format. Changing the filename to /vsizip/roads.zip may help it to be recognized.

That is GDAL 3.13. Earlier versions print only the first sentence, without the hint, and before 3.10 it reads "not recognized as a supported file format". Called from code, the same failure can return None with an empty error message. Even with the fix GDAL suggests, some valid archives still refuse to open, and a path that works on your laptop fails once the service runs from a different working directory.

The short answer: prefix the path with /vsizip/, as in /vsizip/roads.zip. GDAL then reads the archive in place, from disk, memory, or object storage, without extracting anything. The rest of this guide covers doing that safely in an upload handler, using the Python bindings: reading uploads from disk or memory, validating what is inside, the archive layouts that still fail, and how to get a useful error when they do.

Quick Reference

What the upload contains	Path that opens it	Notes
One Shapefile bundle	/vsizip/upload.zip	The Shapefile driver reads the archive like a directory
Several Shapefile bundles	/vsizip/upload.zip	Each .shp becomes its own layer
A Shapefile zip named *.shp.zip or *.shz	upload.shp.zip	No prefix needed (GDAL 3.1+)
One GeoJSON file and nothing else	/vsizip/upload.zip	A zip with a single member resolves to that member; the layer is named after the zip
One GML file with its .xsd	/vsizip/upload.zip/file.gml	Two members make the zip a directory, which the GML driver rejects; name the member
Several GeoJSON files	/vsizip/upload.zip/one_of_them.geojson	Opening the archive itself fails
Bytes already in memory	/vsizip//vsimem/upload.zip	Write the bytes with gdal.FileFromMemBuffer first
A zip in an S3 bucket	/vsizip//vsis3/bucket/key.zip	Uses the standard AWS credentials; AWS_NO_SIGN_REQUEST=YES for public buckets
A zip on a web server	/vsizip//vsicurl/https://…/data.zip	Read without downloading the whole file

Prerequisites

• GDAL 3.x with its Python bindings (from osgeo import gdal must work). The examples use GDAL 3.13.3; the Python code behaves the same on 3.6, 3.8, 3.10, 3.11, and 3.12.
• Python 3.10 or later (the examples use str | None type hints).
• A few zipped uploads to test with. The examples assume these:
  • roads.zip — one Shapefile bundle (roads.shp, .shx, .dbf, .prj)
  • roads.shp.zip — the same bundle under a .shp.zip name
  • two_shapefiles.zip — two Shapefile bundles, roads and rivers
  • two_geojson.zip — two GeoJSON files, roads.geojson and rivers.geojson
  • one_geojson.zip — roads.geojson and nothing else
  • roads_gml.zip — one GML file with its .xsd schema
  • no_vector.zip — a text file and no geodata
  • finder_geojson.zip — roads.geojson plus the __MACOSX/._roads.geojson entry that macOS Finder adds

How GDAL Reads Files Inside an Archive

GDAL drivers read files; they do not unpack archives. /vsizip/ is a virtual file system, not a driver: prefix a path with it and GDAL presents the contents of the zip as a directory. The driver that opens the data (Shapefile, GeoJSON, GeoPackage) never knows an archive was involved.

That design has three consequences that explain almost every surprise in the rest of this guide:

1. Without the prefix, GDAL tries to open the .zip as a data file, no driver claims it, and the open fails.
2. With the prefix, GDAL opens the zip as a directory, so the result depends on whether the driver accepts a directory. The Shapefile driver does; the GeoJSON and GML drivers do not. The one exception is a zip with a single member at its root: GDAL resolves the archive path to that member, so /vsizip/upload.zip opens a lone GeoJSON file directly.
3. Virtual file systems chain. /vsizip/ can sit on top of /vsimem/ (a file in memory) or /vsicurl/ (a file over HTTP), which is what makes in-place reading of uploads possible.

Step 1: Open a Zipped Shapefile with /vsizip/

Without the prefix, the open fails:

$ ogrinfo roads.zip
ERROR 4: `roads.zip' not recognized as being in a supported file format. Changing the filename to /vsizip/roads.zip may help it to be recognized.
ogrinfo failed - unable to open 'roads.zip'.

With it, the Shapefile driver reads the bundle inside:

$ ogrinfo -q /vsizip/roads.zip
1: roads (Line String)

The same path works anywhere GDAL accepts a data source, including ogr2ogr. This converts one layer of a zipped upload straight to GeoPackage:

$ ogr2ogr -f GPKG rivers.gpkg /vsizip/two_shapefiles.zip rivers

In Python, the path is the only change:

from osgeo import gdal

gdal.UseExceptions()

ds = gdal.OpenEx("/vsizip/roads.zip", gdal.OF_VECTOR | gdal.OF_READONLY)
print(ds.GetLayer(0).GetName())  # roads

The .shp.zip shortcut

If you control the file name, there is a shortcut. Since GDAL 3.1, the Shapefile driver opens an archive named *.shp.zip (one or several layers) or *.shz (a single layer) directly, with no prefix:

$ ogrinfo -q roads.shp.zip
1: roads (Line String)

This only helps for Shapefiles, and only when the extension is exactly .shp.zip or .shz. For uploads, where the user picks the name, use /vsizip/ explicitly.

Step 2: Read an Upload That Has No Path with /vsimem/

If the upload already has a path, use it. A file saved by an earlier step, an object in S3 (Step 6), or a Django upload larger than FILE_UPLOAD_MAX_MEMORY_SIZE (2.5 MB by default), which arrives as a TemporaryUploadedFile with temporary_file_path(), can go straight to /vsizip/. GDAL then reads only the parts of the archive it needs, where the file already is. Reading the file into Python first, only to hand GDAL a copy, loads the whole archive into memory for nothing.

Some uploads have no path. Smaller Django uploads stay in memory as an InMemoryUploadedFile; Flask and FastAPI keep small uploads in memory and spool larger ones to anonymous temporary files, and give you a file object either way; a message queue or an HTTP client gives you bytes. For those, /vsimem/ is GDAL's in-memory file system: register the bytes under a path, then chain /vsizip/ on top of it.

import uuid
from osgeo import gdal

gdal.UseExceptions()

mem_path = f"/vsimem/{uuid.uuid4()}.zip"
gdal.FileFromMemBuffer(mem_path, data)  # data: the upload's bytes
ds = gdal.OpenEx(f"/vsizip/{mem_path}", gdal.OF_VECTOR | gdal.OF_READONLY)

Two details matter here:

• Use a unique path per request. /vsimem/ is process-wide. Two concurrent requests writing to /vsimem/upload.zip would overwrite each other's data.
• Free the buffer when you are done. Files in /vsimem/ live until you delete them. Call gdal.Unlink(mem_path) once you have finished with the dataset; the full example in Step 4 does this in a context manager.

The double slash in /vsizip//vsimem/… is not a typo: it is /vsizip/ followed by the absolute path /vsimem/…. The same holds for a file on disk: /vsizip/ followed by /srv/uploads/roads.zip is /vsizip//srv/uploads/roads.zip.

Step 3: List the Members Before You Trust the Archive

Before handing an upload to a driver, look at what is inside. gdal.ReadDirRecursive lists an archive's members through the same virtual file system, without a driver:

members = gdal.ReadDirRecursive("/vsizip//srv/uploads/two_shapefiles.zip")
# ['roads.shp', 'roads.shx', 'roads.dbf', 'roads.prj', 'rivers.shp', ...]

On the command line, gdal vsi list (GDAL 3.11 and later; 3.11 prints the list as JSON) does the same:

$ gdal vsi list /vsizip/two_geojson.zip
roads.geojson
rivers.geojson

The member list lets you reject an archive that holds no vector data with a message that says what it does hold, instead of a generic "unsupported format", and it tells you which members to try when opening the archive as a whole fails.

Step 4: Pick the Layer, or Reject the Upload

What /vsizip/upload.zip gives you depends on what is inside it. With several Shapefiles, each becomes a layer of one dataset:

$ ogrinfo -q /vsizip/two_shapefiles.zip
1: roads (Line String)
2: rivers (Line String)

With several GeoJSON files, the same path fails, because the GeoJSON driver does not accept a directory:

$ ogrinfo -q /vsizip/two_geojson.zip
ERROR 4: `/vsizip/two_geojson.zip' not recognized as being in a supported file format.
ogrinfo failed - unable to open '/vsizip/two_geojson.zip'.

Naming the member opens it:

$ ogrinfo -q /vsizip/two_geojson.zip/rivers.geojson
1: rivers (Line String)

Whether a single file needs its member named depends on what else is in the zip. roads_gml.zip holds the .gml and its .xsd, so the archive is a directory: /vsizip/roads_gml.zip fails and /vsizip/roads_gml.zip/roads.gml opens. one_geojson.zip holds nothing but roads.geojson, so GDAL resolves the archive path to that one member, and the layer takes its name from the zip rather than from the file:

$ ogrinfo -q /vsizip/one_geojson.zip
1: one_geojson (Line String)
$ ogrinfo -q /vsizip/one_geojson.zip/roads.geojson
1: roads (Line String)

Naming the member works in every single-file case and keeps the layer name predictable, so that is the path to prefer when the archive holds one vector file.

So an upload handler needs a short, ordered list of paths to try. The function below puts Steps 2–4 together. open_zip takes any /vsizip/ path, whether the archive is on disk, in memory, or in object storage. It lists the members, rejects archives with no vector data, tries the member named after the requested layer, then the only vector member if there is just one, then the archive as a whole. open_zip_bytes wraps it for uploads that exist only as bytes and frees the in-memory buffer whatever happens:

import os
import uuid
from contextlib import contextmanager

from osgeo import gdal

gdal.UseExceptions()

VECTOR_EXTS = (".shp", ".gpkg", ".geojson", ".json", ".fgb", ".kml", ".gml")


class UploadRejected(ValueError):
    pass


def _try_open(path):
    try:
        return gdal.OpenEx(path, gdal.OF_VECTOR | gdal.OF_READONLY)
    except RuntimeError:
        return None


def _stem(member):
    return member.rsplit("/", 1)[-1].rsplit(".", 1)[0]


def _is_vector(member):
    name = member.rsplit("/", 1)[-1]
    # macOS Finder adds __MACOSX/._<name> metadata files that share the data's extension
    if member.startswith("__MACOSX/") or name.startswith("._"):
        return False
    return name.lower().endswith(VECTOR_EXTS)


def open_zip(zip_path: str, layer: str | None = None):
    """Return (dataset, layer_names) for a /vsizip/ path, or raise UploadRejected."""
    members = gdal.ReadDirRecursive(zip_path)
    if members is None:
        if gdal.VSIStatL(zip_path.removeprefix("/vsizip/")) is not None:
            raise UploadRejected("not a valid zip archive")
        gdal.OpenEx(zip_path, gdal.OF_VECTOR)  # raises GDAL's reason: missing file, credentials, network
    vector_members = [m for m in members if _is_vector(m)]
    if not vector_members:
        raise UploadRejected(f"no vector file in the archive (it holds: {', '.join(members)})")

    # 1. the member named after the requested layer, 2. the only vector
    #    member, if there is one (GeoJSON, GML, a single shapefile),
    #    3. the whole archive (several shapefiles).
    candidates = [f"{zip_path}/{m}" for m in vector_members if layer and _stem(m) == layer]
    if len(vector_members) == 1:
        candidates.append(f"{zip_path}/{vector_members[0]}")
    candidates.append(zip_path)

    for path in candidates:
        ds = _try_open(path)
        if ds is not None:
            break
    else:
        raise UploadRejected(f"several vector files, pick one of: {', '.join(vector_members)}")

    names = [ds.GetLayer(i).GetName() for i in range(ds.GetLayerCount())]
    if layer is not None and layer not in names:
        raise UploadRejected(f"no layer {layer!r}, the archive has: {', '.join(names)}")
    return ds, names


@contextmanager
def open_zip_bytes(data: bytes, layer: str | None = None):
    """open_zip for an upload that exists only in memory. Use the dataset inside the block only."""
    mem_path = f"/vsimem/{uuid.uuid4()}.zip"
    gdal.FileFromMemBuffer(mem_path, data)
    try:
        yield open_zip(f"/vsizip/{mem_path}", layer)
    finally:
        gdal.Unlink(mem_path)

UploadRejected covers problems with the upload itself and maps to a 4xx response with its message. A missing file or missing storage credentials is not the user's fault, so GDAL's own RuntimeError is left to propagate. When the upload has a path, call open_zip("/vsizip/" + os.path.abspath(path)); when it is bytes or a file object, use with open_zip_bytes(upload.read()) as (ds, names): and finish with the dataset inside the block. To try it from the command line, save the code as read_upload.py with this wrapper at the bottom. The wrapper has the file's path, so it passes the path, not the file's bytes:

import sys

if __name__ == "__main__":
    try:
        ds, names = open_zip("/vsizip/" + os.path.abspath(sys.argv[1]), *sys.argv[2:3])
    except UploadRejected as e:
        sys.exit(f"rejected: {e}")
    for name in sys.argv[2:3] or names:
        lyr = ds.GetLayerByName(name)
        srs = lyr.GetSpatialRef()
        print(f"{name}: {lyr.GetFeatureCount()} feature(s), {srs.GetName() if srs else 'no CRS'}")

Run against the sample archives, it gives:

$ python read_upload.py roads.zip
roads: 1 feature(s), WGS 84
$ python read_upload.py two_shapefiles.zip rivers
rivers: 1 feature(s), WGS 84
$ python read_upload.py two_shapefiles.zip lakes
rejected: no layer 'lakes', the archive has: roads, rivers
$ python read_upload.py two_geojson.zip
rejected: several vector files, pick one of: roads.geojson, rivers.geojson
$ python read_upload.py two_geojson.zip rivers
rivers: 1 feature(s), WGS 84
$ python read_upload.py one_geojson.zip
roads: 1 feature(s), WGS 84
$ python read_upload.py roads_gml.zip
roads: 1 feature(s), WGS 84
$ python read_upload.py no_vector.zip
rejected: no vector file in the archive (it holds: notes.txt)
$ python read_upload.py finder_geojson.zip
roads: 1 feature(s), WGS 84

Every rejection names what the archive actually contains, so the user can fix the upload without a support ticket. Feeding the same archives through open_zip_bytes gives the same results, and afterwards gdal.ReadDir("/vsimem/") returns None: no buffers are left behind, including on the rejection paths.

Step 5: Get a Real Error Message Instead of an Empty One

How much GDAL tells you about a failed open depends on how you call it.

With exceptions enabled, as in every example above, the Python bindings raise a RuntimeError that carries GDAL's full message:

gdal.UseExceptions()
gdal.OpenEx("roads.zip", gdal.OF_VECTOR | gdal.OF_READONLY)
# RuntimeError: `roads.zip' not recognized as being in a supported file format.
# Changing the filename to /vsizip/roads.zip may help it to be recognized.

Without gdal.UseExceptions(), the same call returns None, and the last error message is empty:

from osgeo import gdal

ds = gdal.OpenEx("roads.zip", gdal.OF_VECTOR)
print(ds, repr(gdal.GetLastErrorMsg()))
# None ''

GDAL 3.8 and later also print a FutureWarning here (3.6 does not): "Neither gdal.UseExceptions() nor gdal.DontUseExceptions() has been explicitly called. In GDAL 4.0, exceptions will be enabled by default." Call gdal.UseExceptions() once at import time and the empty-message case disappears.

The reason is a flag on the underlying C call. GDALOpenEx only records the "not recognized" message when it is passed GDAL_OF_VERBOSE_ERROR. Opening the same archive from C, once without and once with the flag, shows the difference:

default: ds=NULL last_error=''
verbose: ds=NULL last_error='`roads.zip' not recognized as being in a supported file format. Changing the filename to /vsizip/roads.zip may help it to be recognized.'

If you call GDAL from C or C++, or through another language binding, and failed opens come back with an empty message, check whether the open passes GDAL_OF_VERBOSE_ERROR (in Python, gdal.OF_VERBOSE_ERROR).

Step 6: Read Zips Straight from S3 or a Web Server

Because virtual file systems chain, a zip does not have to be local or even in memory. A common upload design has the client send the file straight to an S3 bucket with a presigned URL, and a worker process it afterwards. With /vsis3/ under /vsizip/, the worker reads the archive where it lies, without downloading it first.

Opening and listing work the same way as for a local archive:

$ ogrinfo -q /vsizip//vsis3/my-bucket/uploads/roads.zip
$ gdal vsi list /vsizip//vsis3/my-bucket/uploads/roads.zip

open_zip from Step 4 takes any /vsizip/ path, so the worker validates the object in the bucket with the same function and nothing else changes:

ds, names = open_zip("/vsizip//vsis3/my-bucket/uploads/roads.zip")

GDAL picks up AWS credentials the usual way, from environment variables or ~/.aws/credentials. If none are found, it says exactly what is missing, and open_zip lets that error through rather than blaming the upload:

RuntimeError: InvalidCredentials: No valid AWS credentials found. For authenticated requests, you need to set AWS_SECRET_ACCESS_KEY, AWS_ACCESS_KEY_ID or other configuration options, or create a ~/.aws/credentials file. Consult https://gdal.org/en/stable/user/virtual_file_systems.html#vsis3-aws-s3-files for more details. For unauthenticated requests on public resources, set the AWS_NO_SIGN_REQUEST configuration option to YES.

For a public bucket, set AWS_NO_SIGN_REQUEST=YES (or gdal.SetConfigOption("AWS_NO_SIGN_REQUEST", "YES")) to read without credentials. For S3-compatible storage outside AWS, such as the self-hosted options in our MinIO alternatives comparison, point /vsis3/ at it with the AWS_S3_ENDPOINT configuration option (host and port, such as minio.internal:9000). Self-hosted servers usually also need AWS_VIRTUAL_HOSTING=FALSE for path-style bucket URLs, and AWS_HTTPS=NO if they serve plain HTTP. With only the endpoint set, GDAL requests https://<bucket>.<endpoint>/… and the open fails.

The same chaining works over plain HTTP with /vsicurl/, which reads a file using range requests:

ds = gdal.OpenEx("/vsizip//vsicurl/https://example.com/data/roads.zip", gdal.OF_VECTOR)

Troubleshooting & Gotchas

A relative path after /vsizip/ works locally but fails in production

The path after /vsizip/ is resolved against the process's working directory, not against anything else. The same command from a different directory fails:

$ cd / && ogrinfo /vsizip/uploads/roads.zip
ERROR 4: `/vsizip/uploads/roads.zip' does not exist in the file system, and is not recognized as a supported dataset name.

Services, task workers, and desktop apps launched from a file manager rarely run from the directory you expect. Convert to an absolute path (os.path.abspath) before adding the prefix.

/vsizip/ is set but the archive is still "not recognized"

The zip holds more than one member, so GDAL presents it as a directory, and the driver that matches the file inside does not accept one. GeoJSON and GML need the member named explicitly (/vsizip/upload.zip/file.geojson) whenever anything else is in the zip, even just an .xsd schema or a readme. List the members with gdal.ReadDirRecursive and open the member path.

A zip made on a Mac is rejected as having several vector files

Finder's "Compress" adds a __MACOSX/ folder with a ._<name> metadata file for each file in the archive, so roads.geojson comes with __MACOSX/._roads.geojson. A member filter that only checks extensions counts both as vector data. Skip members under __MACOSX/ and names starting with ._, as _is_vector in Step 4 does.

An S3 path "does not exist" although the object is there

With missing credentials GDAL says so explicitly, but with wrong credentials, a key that does not exist, or no permission on the bucket, the exception only says the path does not exist in the file system. The HTTP status appears separately, as a warning such as HTTP response code on http://…/roads.zip: 403. Log GDAL's warnings (for example with gdal.PushErrorHandler) in the worker, so a 403 is not mistaken for a missing upload.

The open returns None with an empty error message

Exceptions are not enabled, and the open was not passed the verbose-error flag. Call gdal.UseExceptions() at import time; in C or another binding, add GDAL_OF_VERBOSE_ERROR to the open flags.

Concurrent uploads read each other's data

Every request writes to the same /vsimem/ path, so a second upload overwrites the first while it is still being read. /vsimem/ is shared across the whole process; generate a unique name per request (for example with uuid.uuid4()).

Memory grows with every upload

/vsimem/ files are never freed automatically. Call gdal.Unlink() on the in-memory path in a finally block, so rejected uploads are freed too. If the upload is already on disk, skip /vsimem/ entirely and open it by path.

Conclusion

Zipped geodata does not need to be extracted to be read. /vsizip/ exposes the archive as a directory wherever it is stored (on disk, in S3, or in /vsimem/ when the upload exists only as bytes), so one handler can inspect, validate, and read a Shapefile bundle without extracting it. The remaining work is in the edge cases: naming members for drivers that do not read directories, resolving paths absolutely, and making sure failures carry a message your users can act on.

Resources

• GDAL: GDAL Virtual File Systems — /vsizip/, /vsimem/, /vsicurl/, /vsis3/ and their options
• GDAL: ESRI Shapefile driver — directory input and .shp.zip support
• GDAL: Error handling
• GDAL: Python gotchas — exceptions in the Python bindings
• u11d: MinIO alternatives for S3-compatible storage — Garage, RustFS, and SeaweedFS compared

Consultation for Geospatial Engineering

Struggling with GDAL integration or complex spatial storage? Get expert advice from the u11d engineering team to refine your upload handlers.

Contact Us