Skip to content

Table of content

Tips and Tutorials \ Compress files with gdal and rasterio

Compress files with gdal and rasterio

This tip shows how to compress a file with gdal and write it with rasterio

Tools:
Difficulty:
Type: Tip
Categories: Code Optimization and Distribution

Overview

GDAL is an open source library for processing raster and vector satellite data. The library comes with a large collection of programs able to perform numerous tasks. GDAL's default output format GeoTIFF is not compressed, meaning the value of each pixel is stored on disk without any further treatment. For large rasters, this can cause a lot of memory space consumption. A smart approach is to use specific compression algorithms allowing to reduce the size of the raster without losing any information from the original input. Popular algorithms are DEFLATE, LZW and PACKBITS.

The rasterio library is based on GDAL for reading and writing files, inheriting the same problematics, but simple solutions have been put in place to manage this configuration.

Usage example

GDAL provides many compression methods for many types of rasters. You can visit the gdal raster formats page to learn about available compression methods for your specific file format.

Default GDAL output format, GeoTIFF, supports many compression method listed here.

GDAL to compress an existing file

Here is an example showing how to compress an existing file (see gdal_translate page for more information) :

import gdal
infn = '/path/to/infile.tif'
outfn = '/path/to/outfile.tif'

ds = gdal.Translate(outfn, infn, creationOptions=["COMPRESS=LZW", "TILED=YES"])

Rasterio to write a compressed file

Every key and value available through gdal.SetconfigOption() are configurable with rasterio through the object rasterio.Env() :

import rasterio

with rasterio.Env(COMPRESS=LZW):
    with rasterio.open('example.tif', 'w',height = ds.shape[1], width = ds.shape[2],dtype=str(ds.dtype),count=ds.shape[0] ) as new_dataset :
        new_dataset.write(your_array)

rasterio.Env() returns a context manager object. It administrates GDAL configuration for a specific code block and reinitializes the configuration when the block ends for any reason, success or failure.

Contacts

    Questions ? Get help on the Forum