Arctic Ocean: Near Real Time, regional product (AR)
Regional Near Real Time product. Each source file holds a single platform.
File summary
Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.
- File:
report/header/nrt_ar_ar.yaml.tsv
| Metric | Count | % of files |
|---|---|---|
| Files | 204 | |
| with TEMP | 203 | 99.5% |
| with PSAL | 194 | 95.1% |
| with PRES | 202 | 99.0% |
| with DEPH | 2 | 1.0% |
| with TIME_QC | 204 | 100.0% |
| with POSITION_QC | 195 | 95.6% |
| Extra parameters | CDOM, CHLT, CNDC, DENS, DOX1, DOX2, FLU2, LGH4, OSAT, SIGT, SVEL, TUR4 |
Conversion
Counts after converting the source NetCDF files to Parquet, before any profile is dropped. These are the “original” numbers every later stage is compared against.
- File:
report/convert/nrt_ar_ar.parquet.tsv
| Metric | Count |
|---|---|
| Platforms | 204 |
| Profiles | 358,350 |
| Observations | 93,203,306 |
Cleaning
Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.
Drop bad QC
Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.
- File:
report/clean/dropqc/nrt_ar_ar.parquet.tsv
| Metric | Count | % of original | Deleted |
|---|---|---|---|
| Platforms | 204 | 100.0% | 0.0% |
| Profiles | 356,113 | 99.4% | 0.6% |
| Observations | 92,976,531 | 99.8% | 0.2% |
Drop all-NA profiles
Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.
- File:
report/clean/dropna/nrt_ar_ar.parquet.tsv
| Metric | Count | % of original | Deleted |
|---|---|---|---|
| Platforms | 194 | 95.1% | 4.9% |
| Profiles | 346,212 | 96.6% | 3.4% |
| Observations | 87,016,103 | 93.4% | 6.6% |
Filter by region
Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.
- File:
report/clean/filter/nrt_ar_ar.parquet.tsv
| Metric | Count | % of original | Deleted |
|---|---|---|---|
| Platforms | 191 | 93.6% | 6.4% |
| Profiles | 306,091 | 85.4% | 14.6% |
| Observations | 80,667,512 | 86.6% | 13.4% |
Bounding box
Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.
| Metric | Filtered | Original |
|---|---|---|
| Longitude min | -179.987 | -179.987 |
| Longitude max | 179.998 | 179.998 |
| Latitude min | 60.000 | -74.355 |
| Latitude max | 89.984 | 89.984 |
Deduplication
Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.
Mark duplicates
Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.
- File:
report/dedup/markdup/nrt_ar_ar.parquet.tsv - File:
output/dedup/markdup/nrt_ar_ar.dups.tsv
| Metric | Count | % of original | Deleted | Cleaned | % of cleaned | Deleted (cleaned) |
|---|---|---|---|---|---|---|
| Platforms | 191 | 93.6% | 6.4% | 191 | 100.0% | 0.0% |
| Profiles | 306,091 | 85.4% | 14.6% | 306,091 | 100.0% | 0.0% |
| Observations | 80,667,512 | 86.6% | 13.4% | 80,667,512 | 100.0% | 0.0% |
| Duplicate profiles | 17,578 | 4.9% | 306,091 | 5.7% |
Duplicates within a platform
Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.
| Metric | Count | % of duplicates |
|---|---|---|
| Duplicate groups | 4,267 | |
| Duplicate profiles | 12,520 | 71.2% |
| Platforms | 116 | |
| Observations | 1,908,495 | 60.8% |
Duplicates across platforms
Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.
| Metric | Count | % of duplicates |
|---|---|---|
| Duplicate groups | 2,376 | |
| Duplicate profiles | 5,058 | 28.8% |
| Platforms | 22 | |
| Observations | 1,230,506 | 39.2% |
Duplicate group sizes
How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.
| Profiles per group | Groups | Profiles | % of duplicates |
|---|---|---|---|
| 2 | 5,315 | 10,630 | 60.5% |
| 3 | 503 | 1,509 | 8.6% |
| 4 | 291 | 1,164 | 6.6% |
| 5 | 182 | 910 | 5.2% |
| 6 | 76 | 456 | 2.6% |
| 7 | 66 | 462 | 2.6% |
| 8 | 44 | 352 | 2.0% |
| 9 | 37 | 333 | 1.9% |
| 10 | 29 | 290 | 1.6% |
| 11+ | 100 | 1,472 | 8.4% |
Remove duplicates
Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.
- File:
report/dedup/dedup/nrt_ar_ar.parquet.tsv
| Metric | Count | % of original | Deleted | Cleaned | % of cleaned | Deleted (cleaned) |
|---|---|---|---|---|---|---|
| Platforms | 191 | 93.6% | 6.4% | 191 | 100.0% | 0.0% |
| Profiles | 295,156 | 82.4% | 17.6% | 306,091 | 96.4% | 3.6% |
| Observations | 79,134,055 | 84.9% | 15.1% | 80,667,512 | 98.1% | 1.9% |