ctddump
ctddump is a small, fast command-line tool for converting oceanographic CTD (Conductivity, Temperature, Depth) data from NetCDF into analysis-ready formats:
- Parquet: the observation data, as a uniform flat table.
- YAML: the file metadata (dimensions, variables, global attributes).
It is written in Rust, streams data in bounded memory, and processes whole directory trees in parallel.
Data sources
Two families of source files are supported:
| Source | Description |
|---|---|
| NRT | Near Real Time: Arctic Sea (nrt_ar), Baltic Sea (nrt_bo), Mediterranean Sea (nrt_mo), Global (nrt_gl) |
| CORA | Copernicus Ocean Reanalysis: current format (cora), legacy format (cora_legacy) |
| Your own | Any NetCDF laid out like the Copernicus files, described by a mapping file (generic); see Your own NetCDF files |
What you can do
| Command | Purpose |
|---|---|
convert | Convert a single NetCDF file to Parquet. |
batch | Convert a whole directory tree to Parquet or YAML, in parallel. |
header | Extract a single file’s metadata to YAML. |
checkpres | Find profiles whose pressure does not increase in observation order. |
concat | Merge many Parquet (or YAML) files into one. |
report | Summarise a Parquet or YAML file as a text report. |
filter | Filter a Parquet file’s profiles by a geographic bounding box. |
dropna | Drop profiles that are entirely NA in any of temp/psal/pres. |
dropqc | Drop profiles flagged bad in time_qc or position_qc. |
markdup | Mark duplicate profiles (by date + position) with an is_dup column. |
dedup | Remove duplicate profiles, keeping the one with the most observations. |
compare | Report how far each of two Parquet files covers the other’s platforms and profiles. |
sqlite | Export a Parquet file to a SQLite database (platform, profile, observation tables). |
completions | Print a shell tab-completion script for bash, zsh, fish, elvish, or powershell. |
Quick example
# One file to Parquet
ctddump convert nrt_ar input.nc output.parquet
# A whole Arctic directory, then merge into a single Parquet file
ctddump batch convert nrt_ar --output ./parquet ./netcdf
ctddump concat convert ./parquet arctic.parquet
To see what the full pipeline produces, browse the live example report site, built with the helper scripts.
New here? Start with Installation, skim the Commands, then follow an end-to-end regional workflow.