Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

geoenrich

geoenrich is a small, fast command-line tool that adds geospatial attributes to a table of points. Give it a file with longitude and latitude columns and it appends any of: distance to the nearest coast, bathymetric depth, the sea or ocean name, the nearest country and municipality, or the nearest location in a second table you supply.

It is written in Rust, uses no PROJ or GDAL (the geometry is hand-rolled and pure Rust), and reads and writes Parquet, CSV, TSV, and the gzip variants csv.gz and tsv.gz.

What you can do

CommandPurpose
coastDistance to the nearest shoreline (GSHHG).
depthBathymetric depth at the point (GEBCO grid).
seaSea or ocean name at the point (IHO Sea Areas).
placeNearest country and municipality (Natural Earth + GISCO).
nearestNearest location in a second table you supply, with its distance.

How it works

Every command follows the same pipeline: read the input, reduce it to unique locations with rounded coordinates (3 decimals by default), enrich those unique locations in parallel, then join the results back onto every input row. A file with millions of rows but few distinct positions is therefore cheap to enrich, because only the distinct positions are ever looked up.

Quick example

# Distance to the nearest coast, in kilometers
geoenrich coast cores.parquet \
  --data ./data/gshhg/gshhg-shp-2.3.7/GSHHS_shp/f

# Nearest fish farm to each measurement
geoenrich nearest cores.parquet --to farms.parquet --name-field farm_name

New here? Start with Installation, skim the Commands, then see the reference datasets each command needs.

Installation

Prebuilt binary

The quickest option: every release attaches prebuilt archives for Linux and macOS (x86_64 and arm64) to its GitHub release. They bundle HDF5 and netCDF, so they need no system libraries at all: download the archive for your platform, unpack it, and run the geoenrich binary inside. The helper scripts ship in the archive alongside it.

From crates.io

cargo install geoenrich

This compiles from source, so the depth command needs the HDF5 / NetCDF development headers (see System dependencies).

Build from source

geoenrich is a Rust project, so a recent stable toolchain is all you need:

git clone https://github.com/AIQC-Hub/geoenrich
cd geoenrich
cargo build --release
# binary at target/release/geoenrich

To build a self-contained binary that vendors HDF5 and netCDF (as the release archives do, needing no system libraries), add --features static-netcdf. This compiles the C libraries from source, so it needs cmake and takes longer.

System dependencies

Only the depth command needs anything beyond the Rust toolchain: it reads GEBCO NetCDF and links the HDF5 / NetCDF C libraries, so a source or cargo install build needs their development headers (the same system dependency as ctddump):

# Ubuntu / Debian
sudo apt-get install libhdf5-dev libnetcdf-dev

# macOS
brew install hdf5

The other four commands (coast, sea, place, nearest) use only pure-Rust geometry and have no system dependencies. The prebuilt binary and a --features static-netcdf build vendor the C libraries, so neither needs these.

Reference data

The datasets each command enriches from (shorelines, bathymetry, sea polygons, country and municipality boundaries) are large and are not bundled. Download the ones you need with the helper script, described under Reference datasets.

Check it works

geoenrich --help
geoenrich coast --help

Every command is self-documenting through --help at each level.

coast

Distance to the nearest shoreline, from GSHHG shoreline polygons.

geoenrich coast <INPUT> --data <GSHHG> [OPTIONS]

Appends one column, dist_to_coast (rename with --column), holding the distance from each point to the nearest coastline in kilometers (default) or meters (--unit m).

How it works

The GSHHG L1 (land / ocean) boundary segments are cropped to the region box plus a 5 degree margin, projected through the region’s Lambert Azimuthal Equal-Area (LAEA) projection into meters, and indexed in an R-tree. Each point is projected the same way and takes the planar distance to the nearest segment. Segments are dropped, never clipped, so cropping cannot invent artificial shoreline; a point whose true nearest coast lies beyond the region-plus-margin box gets an over-estimate, so widen the region for such cases.

Options

Beyond the shared options and the region options:

OptionDefaultMeaning
--data <PATH>requiredGSHHG shapefile directory (resolution f recommended) or a GSHHS_*_L1.shp file
--unit <km|m>kmDistance unit for the output column
--column <NAME>dist_to_coastOutput column name

Example

geoenrich coast cores.parquet \
  --data ./data/gshhg/gshhg-shp-2.3.7/GSHHS_shp/f \
  --unit km -o cores.coast.parquet

See Reference datasets for how to obtain GSHHG.

depth

Bathymetric depth at each point, from a GEBCO gridded NetCDF file.

geoenrich depth <INPUT> --data <GEBCO.nc> [OPTIONS]

Appends one column, bathymetry (rename with --column), holding the GEBCO elevation at each point in meters.

How it works

GEBCO is a regular lon/lat grid, so no nearest-neighbor search is needed. The lat and lon axes are read once to learn each axis origin and spacing, then every point maps straight to its nearest grid cell by arithmetic and reads the single elevation cell. Longitudes are normalized to [-180, 180), and points off the grid yield a null.

GEBCO elevation is negative below sea level. By default the value is reported as stored (negative under water); --positive flips the sign so depth reads positive under water and land reads negative.

This command needs no region: a grid lookup is global by construction.

Options

Beyond the shared options:

OptionDefaultMeaning
--data <FILE>requiredGEBCO bathymetry NetCDF file
--positiveoffReport depth as positive below sea level
--column <NAME>bathymetryOutput column name

Example

# Reading and writing gzipped CSV
geoenrich depth cores.csv.gz \
  --data ./data/gebco/GEBCO_2024_sub_ice.nc \
  --positive -o cores.depth.csv.gz

The depth command links the HDF5 / NetCDF C libraries; see Installation. See Reference datasets for how to obtain the GEBCO grid.

sea

Sea or ocean name at each point, from the IHO Sea Areas polygons.

geoenrich sea <INPUT> --data <IHO> [OPTIONS]

Appends one column, sea_name (rename with --column), holding the name of the sea or ocean the point falls in.

How it works

The IHO Sea Areas features (Marine Regions GeoJSON or shapefile) are cropped whole to the region box plus a margin, and their bounding boxes are indexed in an R-tree. Each point is resolved by an even-odd point-in-polygon test over the candidate features, with a nearest-boundary fallback for points that fall just inland (for example inside a fjord not covered by a sea polygon).

--name-field selects which feature property holds the name (default NAME); the command errors clearly if that field is absent.

Options

Beyond the shared options and the region options:

OptionDefaultMeaning
--data <PATH>requiredIHO Sea Areas polygons (GeoJSON or shapefile)
--name-field <NAME>NAMEProperty / attribute field holding the sea name
--column <NAME>sea_nameOutput column name

Example

geoenrich sea cores.parquet --region norway \
  --data ./data/iho/iho_sea_areas.geojson

See Reference datasets for how to obtain the IHO Sea Areas.

place

Nearest country and municipality, from Natural Earth countries and, optionally, Eurostat GISCO LAU municipalities.

geoenrich place <INPUT> --countries <NE> [--municipalities <GISCO>] [OPTIONS]

Appends three columns:

ColumnMeaning
countryCountry name
country_codeISO alpha-3 code (the Natural Earth -99 placeholder becomes null)
municipalityMunicipality name (empty unless --municipalities is given)

How it works

Both the country and the municipality lookups resolve a point by containment first (an even-odd point-in-polygon test over R-tree candidates) and fall back to the nearest boundary otherwise, so an offshore point still gets the closest land unit. Attribute fields are auto-detected from candidate name lists, so minor schema drift between dataset versions needs no flags.

--municipalities is optional: without it the municipality column is left empty and a note says so.

Options

Beyond the shared options and the region options:

OptionDefaultMeaning
--countries <PATH>requiredNatural Earth countries shapefile
--municipalities <PATH>noneGISCO LAU municipalities shapefile

Example

geoenrich place cores.parquet \
  --countries ./data/naturalearth/ne_10m_admin_0_countries.shp \
  --municipalities ./data/gisco/lau.shp

See Reference datasets for how to obtain Natural Earth and GISCO.

nearest

Nearest location in a second table you supply, with the distance to it.

geoenrich nearest <INPUT> --to <REFERENCE> [OPTIONS]

Unlike the other commands, the reference data is not a bundled dataset but a second table you pass with --to: any set of named locations (fish farms, ports, stations). For every input point it appends the name of the nearest reference location and the great-circle distance to it.

Appends two columns:

ColumnMeaning
nearest_nameName of the nearest reference location (rename with --name-column)
nearest_distDistance to it, in km (default) or m (rename with --dist-column)

How it works

Every reference point is mapped to a unit-sphere (x, y, z) vector and indexed in a 3D R-tree. Each input point is projected the same way and takes the R-tree’s nearest neighbor. Euclidean chord distance on the unit sphere grows monotonically with the great-circle distance, so the nearest by chord is the nearest on the globe; the chord is then converted back to meters.

The unit sphere is used instead of the region LAEA on purpose: the two sets can be anywhere and arbitrarily far apart, and a single planar projection distorts distances away from its center. The sphere is exact everywhere, so this command takes no region box or projection center. Reference rows with a missing coordinate are skipped; an empty reference set leaves every output null.

Options

Beyond the shared options:

OptionDefaultMeaning
--to <FILE>requiredReference table: the second set of locations
--to-format <FMT>inferredFormat of the reference table
--to-lon-col <NAME>longitudeLongitude column in the reference table
--to-lat-col <NAME>latitudeLatitude column in the reference table
--name-field <NAME>nameColumn in the reference table holding each location’s name
--unit <km|m>kmDistance unit for the output distance column
--name-column <NAME>nearest_nameOutput column for the nearest name
--dist-column <NAME>nearest_distOutput column for the distance

Example

# Nearest fish farm to each measurement, distance in km
geoenrich nearest cores.parquet --to farms.parquet \
  --name-field farm_name -o cores.nearest.parquet

The name column is cast to text, so numeric ids work as names too.

Regions

The coast, sea, and place commands crop their reference data to a region box (plus a margin) and, for distance work, project through a Lambert Azimuthal Equal-Area (LAEA) projection centered on that region. The depth command needs no region (a grid lookup is global), and nearest computes exact great-circle distances on the unit sphere, so it needs no region either.

Presets

Pick a region with --region <NAME>:

PresetBox (min_lon, max_lon, min_lat, max_lat)
global (default)-180, 180, -90, 90
baltic8, 31, 53, 66
norway-10, 45, 55, 85
arctic-180, 180, 60, 90
atlantic-83, 20, -60, 70
europe-25, 45, 34, 72
mediterranean-6, 37, 30, 46

The default is global (the whole globe).

Explicit box and projection center

Any preset value can be overridden with explicit bounds:

geoenrich coast cores.parquet --data ./data/gshhg/... \
  --min-lon 8 --max-lon 31 --min-lat 53 --max-lat 66

By default the LAEA projection is centered on the region box center. Override it with --proj-lon0 / --proj-lat0 if you want the center elsewhere (for example to reduce distortion over an asymmetric region).

Precedence

For the region box and projection center, later sources win:

preset / built-in default  <  config file  <  CLI flag

So a --region preset sets the box, a config file can override individual bounds, and a CLI --min-lon (etc.) overrides both.

Choosing a region

Cropping keeps the reference data small and the lookups fast, but a point whose true nearest feature lies outside the region-plus-margin box can be wrong (for coast, an over-estimate). If in doubt, widen the box or use global; the unique-location pipeline keeps even a global run affordable.

Output columns

Each command appends its columns to a copy of the input and writes the result; the input columns are preserved and their order is kept.

CommandAppended columnsType
coastdist_to_coastfloat (km or m)
depthbathymetryfloat (m)
seasea_nametext
placecountry, country_code, municipalitytext
nearestnearest_name, nearest_disttext, float (km or m)

The single-column commands rename their output with --column; nearest renames with --name-column / --dist-column.

Null values

A row gets null (or NaN for float columns) output when:

  • its longitude or latitude is null or NaN, or
  • the lookup finds nothing (for example a depth point off the GEBCO grid, or a nearest run against an empty reference set).

Clashing columns

If an output column name already exists in the input, the run fails before any enrichment, naming the clashing column(s). Pass --overwrite to replace such a column in place instead: it keeps its position but takes the output value and dtype.

Output format

By default the output file is written beside the input as <stem>.<command>.<input format>, so the output format follows the input’s (a points.csv.gz input enriches to points.coast.csv.gz). Override the path with --output or the format with --out-format. An input with an unrecognized extension defaults to Parquet.

Configuration

Shared options

Every command accepts the same input, output, and pipeline options:

OptionDefaultMeaning
-o, --output <FILE><stem>.<command>.<input format>Output file (beside the input)
--in-format <FMT>inferred, else parquetparquet, csv, tsv, csv.gz, tsv.gz, auto
--out-format <FMT>inferred, else parquetsame set
--overwriteoffReplace clashing output columns instead of failing
--lon-col <NAME>longitudeLongitude column
--lat-col <NAME>latitudeLatitude column
--decimals <N>3Rounding applied before de-duplicating
-t, --threads <N>all coresWorker threads
-c, --config <TOML>noneConfig file (CLI flags override it)

The coast, sea, and place commands also take the region options.

Formats

Input and output can be Parquet (default), CSV, TSV, and the gzip variants csv.gz and tsv.gz. The format is inferred from the file extension; an unrecognized extension falls back to Parquet. Gzip is handled directly (not via a Polars feature), so it behaves the same across Polars versions.

Config file

A TOML config file supplies the region box and projection center, sitting between the built-in default and the CLI flags (see precedence). Every field is optional:

# region.toml
region    = "baltic"   # a named preset
min_lon   = 8.0        # or explicit bounds, overriding the preset
max_lon   = 31.0
min_lat   = 53.0
max_lat   = 66.0
proj_lon0 = 19.5       # LAEA projection center (default: region center)
proj_lat0 = 59.5
geoenrich coast cores.parquet --data ./data/gshhg/... -c region.toml

Rounding and de-duplication

Coordinates are rounded to --decimals places (3 by default) before the input is reduced to unique locations. Only the unique locations are looked up, then the results are joined back onto every row. Raising --decimals distinguishes closer points at the cost of more unique lookups; lowering it collapses more rows together.

Technical notes

The shared pipeline

Every command implements one small trait (declare the output columns, compute their values for one location) and shares the rest: extract longitude and latitude (cast to float, nulls to NaN), round and de-duplicate into unique locations with integer-scaled keys (so the join never compares floats), enrich the unique set in parallel with rayon, expand the results back to one value per input row, append the columns, and write. A NaN coordinate gets no key and therefore null output.

Geometry: no PROJ, no GDAL

All geometry is hand-rolled in pure Rust, so downstream projects need no extra system libraries. Two projections do the work:

  • A spherical LAEA (Lambert Azimuthal Equal-Area) centered on the region, used for planar distances by coast (and for the nearest-boundary fallbacks in sea and place). Planar distance in that projection is accurate for the regional-scale nearest-feature query. The reference R workflow used EPSG:3035 LAEA for the same reason. A single sphere (authalic radius) is used rather than the GRS80 ellipsoid; the error is well under coastline resolution for regional work. Sub-meter accuracy, if ever needed, means an ellipsoidal LAEA.
  • The unit sphere, used by nearest. Reference points become (x, y, z) vectors on the unit sphere; nearest-by-chord equals nearest-by-great-circle, and the squared chord converts back to an exact great-circle distance in meters. This is why nearest needs no projection center and has none of a planar projection’s distortion far from a center.

The haversine_m great-circle distance is used for reference and to refine index candidates.

Spatial indexes

Nearest-feature and point-in-polygon queries use rstar R-trees:

  • coast indexes projected shoreline segments; a query takes the nearest segment’s planar distance.
  • sea and place index feature bounding boxes for the containment test and boundary segments for the nearest fallback.
  • nearest indexes reference points in 3D on the unit sphere.

Cropping keeps whole features: a feature is dropped if its bounding box misses the region-plus-margin box, but it is never clipped, so containment stays exact and cropping cannot invent geometry.

Memory and streaming

The input is read whole into memory because the join back touches every row; the enrichment set itself is only the unique locations, which stays small. For very large inputs a streamed two-pass version (collect unique locations, then append columns chunk by chunk) is the natural next step, noted in the source.

Parquet writes

Parquet is written single-threaded (set_parallel(false)), matching ctddump: the parallel column encoder in the pinned Polars version leaks memory per call, and the single-thread path is safe and deterministic.

Reference datasets

The datasets each command enriches from are large and are not bundled or shipped. Each command takes its data path by flag (--data, or --countries / --municipalities for place). The nearest command is the exception: its reference data is a table you supply with --to, not a downloaded dataset.

Download helper

scripts/download_data.sh fetches and unpacks any of the five sources into ./data/, one sub-directory per source, matching the example paths in the command pages:

# GSHHG, GEBCO, Natural Earth, and GISCO need no details
scripts/download_data.sh download gshhg gebco countries lau

# the Marine Regions (IHO) download sits behind a short form
scripts/download_data.sh --mr-name "Your Name" --mr-email you@example.org \
  --mr-country Norway download iho

Selected datasets download in parallel. Existing archives are kept (--force re-downloads), and the multi-GB GEBCO grid resumes an interrupted download. Run scripts/download_data.sh --help for all options.

Caveats baked into the script:

  • The GEBCO grid is multi-GB; the download resumes if interrupted.
  • The GISCO LAU bundle nests one zip per projection; only the EPSG 4326 (lon/lat) layer is unpacked, since the commands expect lon/lat.
  • The Marine Regions (IHO) download submits the site’s statistics form, so it requires --mr-name, --mr-email, and --mr-country. It verifies the response is a zip and fails loudly if the form rejects the request.

Sources

DatasetUsed bySource
GSHHG shorelines (ESRI shapefiles, resolution f)coasthttps://www.soest.hawaii.edu/pwessel/gshhg/
GEBCO bathymetry (gridded NetCDF)depthhttps://www.gebco.net/
IHO Sea Areas v3 (GeoJSON or shapefile)seahttps://www.marineregions.org/
Natural Earth countriesplacehttps://www.naturalearthdata.com/
Eurostat GISCO LAU (municipalities)placehttps://ec.europa.eu/eurostat/web/gisco

Helper scripts

The scripts/ directory holds two bash helpers that sit alongside the CLI: one to fetch the reference datasets, one to run several modules over an input in a single pass. Neither is required to use geoenrich, but together they cover the “get the data, then enrich” workflow end to end.

Both follow the same conventions:

  • The header comment doubles as --help, so scripts/<name> --help prints the full interface.
  • Steps are traced to stderr as they run (a timestamped RUN: line per command), so a long run shows what it is doing.
  • They run under set -euo pipefail and stop at the first error.

download_data.sh

Downloads and unpacks the reference datasets into a local data/ tree, one sub-directory per source, matching the paths the command pages use. Selected datasets download in parallel, existing archives are kept unless --force is given, and the multi-GB GEBCO grid resumes an interrupted download.

scripts/download_data.sh download gshhg gebco countries lau

See Reference datasets for the full list of sources, the Marine Regions (IHO) form details, and the caveats.

enrich.sh

Runs several modules over one input in sequence and writes a single output file carrying every selected module’s new columns. A module runs when you give its data source, each step chains onto the previous one’s output, and the intermediate files are removed when the script ends.

scripts/enrich.sh cores.parquet cores.enriched.parquet \
  --coast ./data/gshhg/gshhg-shp-2.3.7/GSHHS_shp/f \
  --depth ./data/gebco/GEBCO_2024_sub_ice.nc \
  --nearest farms.parquet --nearest-name-field farm_name

See Enrich with several modules for the module flags, the per-module and common options, and how the intermediate files are handled.

Requirements

download_data.sh needs curl and unzip on PATH. enrich.sh needs a built geoenrich binary: it uses $GEOENRICH_BIN if set, else the one on PATH, else a ./target/release or ./target/debug build in the repository (pass --bin to point elsewhere).

Enrich with several modules

scripts/enrich.sh runs several modules over one input in sequence, so the result is a single file carrying the new columns of every module you selected. Each module is chained onto the previous one’s output, its columns accumulating, and the in-between files are written to a temporary directory that is removed when the script ends. Only the final output file remains.

scripts/enrich.sh [options] <input> <output>

Selecting modules

A module runs only when you give its data source, so the flags pick both the modules and their inputs. At least one module must be selected.

FlagModuleData source
--coast PATHcoastGSHHG shapefile dir or GSHHS_*_L1.shp
--depth FILEdepthGEBCO bathymetry NetCDF
--sea PATHseaIHO Sea Areas GeoJSON or shapefile
--countries FILEplaceNatural Earth countries shapefile
--nearest FILEnearestreference table of named locations

The modules run in a fixed order (coast, depth, sea, place, nearest); the order does not matter, since each adds distinct columns.

Per-module options

OptionApplies toMeaning
--municipalities FILEplaceGISCO LAU municipalities (optional)
--coast-unit km|mcoastDistance unit (default km)
--depth-positivedepthReport depth positive below sea level
--sea-name-field STRseaFeature field with the sea name (default NAME)
--nearest-name-field STRnearestReference name column (default name)
--nearest-unit km|mnearestDistance unit (default km)

Common and other options

--region, --lon-col, --lat-col, --decimals, and --threads are passed to every module that accepts them (--region only to coast, sea, and place). --in-format describes the original input. Other options:

OptionMeaning
--bin PATHgeoenrich binary (default: $GEOENRICH_BIN, else the one on PATH, else ./target/release or ./target/debug)
-k, --keepKeep the intermediate files (default: remove them)
-n, --dry-runPrint the commands without running them

Example

scripts/enrich.sh cores.parquet cores.enriched.parquet \
  --coast ./data/gshhg/gshhg-shp-2.3.7/GSHHS_shp/f \
  --depth ./data/gebco/GEBCO_2024_sub_ice.nc \
  --nearest farms.parquet --nearest-name-field farm_name

This writes one file, cores.enriched.parquet, with the original columns plus dist_to_coast, bathymetry, nearest_name, and nearest_dist. The intermediate coast-only and coast+depth files are removed on exit.

The intermediate files are Parquet (lossless); the final file’s format follows its extension, so cores.enriched.csv.gz would be written as gzipped CSV.