Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

ctddump: CTD data summary reports

Summary reports produced by the ctddump pipeline, one page per (region, product) unit. Pick a report from the sidebar.

Observation order: when the converted files are merged (ctddump concat convert), each profile’s observations are sorted by increasing pressure and renumbered in that order, and observations without a pressure value are dropped. Every count on these pages is taken from data in that order. See Row order and numbering.

Regions in this build: arctic baltic mediterranean.

Generated on 2026-11-12 from summary.

Arctic Ocean: Near Real Time, regional product (AR)

Regional Near Real Time product. Each source file holds a single platform.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/nrt_ar_ar.yaml.tsv
MetricCount% of files
Files204
with TEMP20399.5%
with PSAL19495.1%
with PRES20299.0%
with DEPH21.0%
with TIME_QC204100.0%
with POSITION_QC19595.6%
Extra parametersCDOM, CHLT, CNDC, DENS, DOX1, DOX2, FLU2, LGH4, OSAT, SIGT, SVEL, TUR4

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/nrt_ar_ar.parquet.tsv
MetricCount
Platforms204
Profiles358,350
Observations93,203,306

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature55,624,50959.7%00.0%36,896,49739.6%682,3000.7%
Salinity55,103,36659.1%00.0%36,857,31839.5%1,242,6221.3%
Pressure56,306,68760.4%00.0%36,896,61939.6%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/nrt_ar_ar.summary.tsv
MetricCountShare
Profiles checked358,351
Profiles without pressure (not checked)0
Profiles with decreasing pressure00.0%
of which reversed00.0%
of which non-monotonic00.0%
Platforms with such profiles00.0%
Files with such profiles00.0%

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/nrt_ar_ar.parquet.tsv
MetricCount% of originalDeleted
Platforms204100.0%0.0%
Profiles356,11399.4%0.6%
Observations92,976,53199.8%0.2%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/nrt_ar_ar.parquet.tsv
MetricCount% of originalDeleted
Platforms19495.1%4.9%
Profiles346,21296.6%3.4%
Observations87,016,10393.4%6.6%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/nrt_ar_ar.parquet.tsv
MetricCount% of originalDeleted
Platforms19193.6%6.4%
Profiles306,09185.4%14.6%
Observations80,667,51286.6%13.4%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min-179.987-179.987
Longitude max179.998179.998
Latitude min60.000-74.355
Latitude max89.98489.984

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/nrt_ar_ar.parquet.tsv
  • File: output/dedup/markdup/nrt_ar_ar.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms19193.6%6.4%191100.0%0.0%
Profiles306,09185.4%14.6%306,091100.0%0.0%
Observations80,667,51286.6%13.4%80,667,512100.0%0.0%
Duplicate profiles17,5784.9%306,0915.7%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups4,267
Duplicate profiles12,52071.2%
Platforms116
Observations1,908,49560.8%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups2,376
Duplicate profiles5,05828.8%
Platforms22
Observations1,230,50639.2%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
25,31510,63060.5%
35031,5098.6%
42911,1646.6%
51829105.2%
6764562.6%
7664622.6%
8443522.0%
9373331.9%
10292901.6%
11+1001,4728.4%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/nrt_ar_ar.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms19193.6%6.4%191100.0%0.0%
Profiles295,15682.4%17.6%306,09196.4%3.6%
Observations79,134,05584.9%15.1%80,667,51298.1%1.9%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/ar_nrt_vs_cora.tsv
  • File: report/compare/ar_nrt_vs_gl.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms191
Profiles295,156
Profiles without a key0
Observations79,134,055

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
cora_ar18094.2%233,16179.0%229,7343,42798.5%
nrt_ar_gl00.0%00.0%00

Arctic Ocean: Near Real Time, global product (GL)

Global (GL) Near Real Time product restricted to this region’s bounding box. It overlaps the regional product, so a profile can appear in both and duplicates are expected.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/nrt_ar_gl.yaml.tsv
MetricCount% of files
Files560
with TEMP55298.6%
with PSAL55298.6%
with PRES35062.5%
with DEPH38067.9%
with TIME_QC560100.0%
with POSITION_QC560100.0%
Extra parametersALK3, ALKY, C13D, C14D, CDOM, CFC11, CFC113, CFC12, CHLT, CNDC, CPHL, DOX1, DOX2, DOXY, EWCT, FLU2, FLUO, HCDT, HCSP, HELD, HELM, LGH4, LGHT, NEON, NSCT, NTAW, NTRA, NTRI, NTRZ, NTZW, O18D, OSAT, PCO2, PHOS, PHOW, PHPH, SECC, SLCA, SLCW, SSAL, SVEL, TCO2, TRANSMISSIVITY, TRIW, TUR2, TUR3, TUR4, TUR6

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/nrt_ar_gl.parquet.tsv
MetricCount
Platforms560
Profiles737,465
Observations279,465,196

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature239,474,28485.7%00.0%39,988,10314.3%2,8090.0%
Salinity239,473,48085.7%00.0%39,988,10314.3%3,6130.0%
Pressure240,916,36086.2%00.0%38,548,83613.8%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/nrt_ar_gl.summary.tsv
  • File: report/convert/checkpres/nrt_ar_gl.platforms.tsv
MetricCountShare
Profiles checked753,991
Profiles without pressure (not checked)0
Profiles with decreasing pressure15,8362.1%
of which reversed4200.1%
of which non-monotonic15,4162.0%
Platforms with such profiles305.4%
Files with such profiles305.4%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
FNCM7,93045.7%1287,8022000-01-012023-04-17
FNIN2,67956.9%212,6582012-01-262022-05-16
FMCY2,20020.1%752,1252015-01-122022-02-10
FNFP1,0866.2%1968901997-09-032026-02-12
FABB98411.8%09842014-05-242019-05-25
FNOY2265.6%02261972-02-211977-12-15
SNGH2036.7%02032010-02-102015-10-19
CGBV1672.1%01671971-05-101990-10-10
CGDG700.4%0701974-03-311998-06-06
MJPX9411.2%0411997-08-122007-05-12

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/nrt_ar_gl.parquet.tsv
MetricCount% of originalDeleted
Platforms55999.8%0.2%
Profiles718,32497.4%2.6%
Observations273,240,62797.8%2.2%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/nrt_ar_gl.parquet.tsv
MetricCount% of originalDeleted
Platforms55098.2%1.8%
Profiles673,42191.3%8.7%
Observations263,333,06694.2%5.8%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/nrt_ar_gl.parquet.tsv
MetricCount% of originalDeleted
Platforms54597.3%2.7%
Profiles202,83227.5%72.5%
Observations58,186,06820.8%79.2%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min-179.980-179.999
Longitude max179.999180.000
Latitude min60.000-78.638
Latitude max90.00090.000

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/nrt_ar_gl.parquet.tsv
  • File: output/dedup/markdup/nrt_ar_gl.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms54597.3%2.7%545100.0%0.0%
Profiles202,83227.5%72.5%202,832100.0%0.0%
Observations58,186,06820.8%79.2%58,186,068100.0%0.0%
Duplicate profiles56,6637.7%202,83227.9%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups14,629
Duplicate profiles30,95754.6%
Platforms179
Observations11,543,87570.4%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups12,683
Duplicate profiles25,70645.4%
Platforms97
Observations4,861,75729.6%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
226,21052,42092.5%
38042,4124.3%
41606401.1%
5422100.4%
6251500.3%
714980.2%
812960.2%
96540.1%
103300.1%
11+365531.0%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/nrt_ar_gl.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms53896.1%3.9%54598.7%1.3%
Profiles173,48123.5%76.5%202,83285.5%14.5%
Observations50,392,17218.0%82.0%58,186,06886.6%13.4%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/ar_gl_vs_cora.tsv
  • File: report/compare/ar_nrt_vs_gl.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms538
Profiles173,481
Profiles without a key0
Observations50,392,172

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
cora_ar51796.1%134,99477.8%132,4872,50798.1%
nrt_ar_ar00.0%00.0%00

Arctic Ocean: CORA reanalysis

CORA delayed-mode reanalysis. Source files hold many platforms each, and the profiles are re-processed rather than near real time.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/cora_ar.yaml.tsv
MetricCount% of files
Files20,975
with TEMP20,975100.0%
with PSAL20,96399.9%
with PRES17,40283.0%
with DEPH20,81799.2%
with TIME_QC20,975100.0%
with POSITION_QC20,975100.0%
Extra parametersnone

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/cora_ar.parquet.tsv
MetricCount
Platforms767
Profiles499,801
Observations137,477,306

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature137,477,306100.0%00.0%00.0%00.0%
Salinity137,477,306100.0%00.0%00.0%00.0%
Pressure137,477,306100.0%00.0%00.0%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/cora_ar.summary.tsv
  • File: report/convert/checkpres/cora_ar.platforms.tsv
MetricCountShare
Profiles checked502,841
Profiles without pressure (not checked)0
Profiles with decreasing pressure2,2340.4%
of which reversed210.0%
of which non-monotonic2,2130.4%
Platforms with such profiles212.7%
Files with such profiles2,22210.6%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
684122,13717.9%212,1161953-04-091990-12-31
CGDG180.9%0181977-09-131987-08-28
DBFP180.3%0181984-11-121989-10-10
EXIC0003142.7%0141978-07-261978-10-05
DBLK90.2%091984-08-011997-08-02
LMBG910.1%092008-08-122008-08-22
UKKN614.6%061961-03-231961-04-09
TFJA40.2%041996-11-012004-11-21
UKUR36.1%031960-09-081960-12-29
EXO1053921.6%021964-04-041965-09-04

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/cora_ar.parquet.tsv
MetricCount% of originalDeleted
Platforms76299.3%0.7%
Profiles488,11197.7%2.3%
Observations135,042,91898.2%1.8%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/cora_ar.parquet.tsv
MetricCount% of originalDeleted
Platforms75898.8%1.2%
Profiles482,82396.6%3.4%
Observations134,395,96997.8%2.2%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/cora_ar.parquet.tsv
MetricCount% of originalDeleted
Platforms75898.8%1.2%
Profiles482,82396.6%3.4%
Observations134,395,96997.8%2.2%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min-179.987-179.987
Longitude max179.999179.999
Latitude min60.00060.000
Latitude max89.99889.998

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/cora_ar.parquet.tsv
  • File: output/dedup/markdup/cora_ar.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms75898.8%1.2%758100.0%0.0%
Profiles482,82396.6%3.4%482,823100.0%0.0%
Observations134,395,96997.8%2.2%134,395,969100.0%0.0%
Duplicate profiles133,09026.6%482,82327.6%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups16,900
Duplicate profiles38,61029.0%
Platforms252
Observations11,766,18129.7%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups46,234
Duplicate profiles94,48071.0%
Platforms347
Observations27,817,76070.3%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
260,054120,10890.2%
31,9775,9314.5%
44701,8801.4%
52071,0350.8%
6975820.4%
7704900.4%
8655200.4%
9524680.4%
10303000.2%
11+1121,7761.3%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/cora_ar.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms74797.4%2.6%75898.5%1.5%
Profiles412,86782.6%17.4%482,82385.5%14.5%
Observations117,998,37685.8%14.2%134,395,96987.8%12.2%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/ar_gl_vs_cora.tsv
  • File: report/compare/ar_nrt_vs_cora.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms747
Profiles412,867
Profiles without a key0
Observations117,998,376

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
nrt_ar_ar18024.1%233,16156.5%229,7343,42798.5%
nrt_ar_gl51769.2%134,99432.7%132,4872,50798.1%

Baltic Sea: Near Real Time, regional product (BO)

Regional Near Real Time product. Each source file holds a single platform.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/nrt_bo_bo.yaml.tsv
MetricCount% of files
Files753
with TEMP753100.0%
with PSAL753100.0%
with PRES57676.5%
with DEPH17723.5%
with TIME_QC753100.0%
with POSITION_QC46161.2%
Extra parametersDOX1, FLU2, LGHT, SVEL, TUR4

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/nrt_bo_bo.parquet.tsv
MetricCount
Platforms753
Profiles113,277
Observations9,392,800

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature9,392,800100.0%00.0%00.0%00.0%
Salinity9,392,799100.0%10.0%00.0%00.0%
Pressure9,392,800100.0%00.0%00.0%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/nrt_bo_bo.summary.tsv
  • File: report/convert/checkpres/nrt_bo_bo.platforms.tsv
MetricCountShare
Profiles checked113,277
Profiles without pressure (not checked)0
Profiles with decreasing pressure2,4052.1%
of which reversed00.0%
of which non-monotonic2,4052.1%
Platforms with such profiles9813.0%
Files with such profiles9813.0%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
DMU4138744.4%0871992-02-102009-09-17
DMU4038447.5%0841992-02-112009-09-16
DMU9548447.7%0841992-02-202009-09-14
DMU9258346.1%0831992-02-182009-09-16
DMU9358351.2%0831993-09-292009-09-15
DMU9058249.4%0821992-02-112009-09-16
DMU9398245.1%0821993-04-292009-09-15
DMU4186244.9%0621994-01-102009-09-17
VSJ100036010.3%0601990-03-122006-04-20
DMU10075742.2%0571994-07-122009-09-16

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/nrt_bo_bo.parquet.tsv
MetricCount% of originalDeleted
Platforms753100.0%0.0%
Profiles107,37594.8%5.2%
Observations8,590,34391.5%8.5%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/nrt_bo_bo.parquet.tsv
MetricCount% of originalDeleted
Platforms74899.3%0.7%
Profiles105,20092.9%7.1%
Observations8,569,83491.2%8.8%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/nrt_bo_bo.parquet.tsv
MetricCount% of originalDeleted
Platforms74899.3%0.7%
Profiles105,20092.9%7.1%
Observations8,569,83491.2%8.8%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min6.3006.300
Longitude max27.45327.453
Latitude min54.00054.000
Latitude max65.73065.730

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/nrt_bo_bo.parquet.tsv
  • File: output/dedup/markdup/nrt_bo_bo.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms74899.3%0.7%748100.0%0.0%
Profiles105,20092.9%7.1%105,200100.0%0.0%
Observations8,569,83491.2%8.8%8,569,834100.0%0.0%
Duplicate profiles28,77325.4%105,20027.4%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups3,079
Duplicate profiles6,28021.8%
Platforms139
Observations256,91116.5%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups6,760
Duplicate profiles22,49378.2%
Platforms106
Observations1,298,93783.5%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
25,82511,65040.5%
31,8015,40318.8%
48843,53612.3%
55722,8609.9%
63672,2027.7%
72081,4565.1%
8756002.1%
9484321.5%
10393901.4%
11+202440.8%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/nrt_bo_bo.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms74198.4%1.6%74899.1%0.9%
Profiles86,26676.2%23.8%105,20082.0%18.0%
Observations7,734,36182.3%17.7%8,569,83490.3%9.7%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/bo_nrt_vs_cora.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms741
Profiles86,266
Profiles without a key0
Observations7,734,361

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
cora_bo45461.3%49,64657.5%48,93471298.6%

Baltic Sea: CORA reanalysis

CORA delayed-mode reanalysis. Source files hold many platforms each, and the profiles are re-processed rather than near real time.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/cora_bo.yaml.tsv
MetricCount% of files
Files12,482
with TEMP12,482100.0%
with PSAL12,482100.0%
with PRES12,47399.9%
with DEPH11,97395.9%
with TIME_QC12,482100.0%
with POSITION_QC12,482100.0%
Extra parametersnone

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/cora_bo.parquet.tsv
MetricCount
Platforms907
Profiles230,250
Observations19,103,053

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature18,604,39697.4%00.0%00.0%498,6572.6%
Salinity18,604,39697.4%00.0%00.0%498,6572.6%
Pressure18,604,39697.4%00.0%00.0%498,6572.6%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/cora_bo.summary.tsv
  • File: report/convert/checkpres/cora_bo.platforms.tsv
MetricCountShare
Profiles checked234,621
Profiles without pressure (not checked)0
Profiles with decreasing pressure4,2641.8%
of which reversed10.0%
of which non-monotonic4,2631.8%
Platforms with such profiles24527.0%
Files with such profiles1,2189.8%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
58GD2634.1%02632020-03-072022-11-30
OWPB22840.3%02282002-02-042004-11-20
SNGH2036.7%02032010-02-102015-10-19
OXBH16931.6%01692020-02-282022-11-16
OHLV1101.6%01102006-05-032020-08-14
DMU4138744.4%0871992-02-102009-09-17
DMU4038447.5%0841992-02-112009-09-16
DMU9548447.7%0841992-02-202009-09-14
DMU9258346.1%0831992-02-182009-09-16
DMU9358351.2%0831993-09-292009-09-15

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/cora_bo.parquet.tsv
MetricCount% of originalDeleted
Platforms64070.6%29.4%
Profiles188,78082.0%18.0%
Observations17,100,06789.5%10.5%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/cora_bo.parquet.tsv
MetricCount% of originalDeleted
Platforms63970.5%29.5%
Profiles187,25781.3%18.7%
Observations17,024,49989.1%10.9%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/cora_bo.parquet.tsv
MetricCount% of originalDeleted
Platforms63970.5%29.5%
Profiles187,25781.3%18.7%
Observations17,024,49989.1%10.9%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min8.0008.000
Longitude max29.44729.447
Latitude min53.33353.333
Latitude max65.80865.808

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/cora_bo.parquet.tsv
  • File: output/dedup/markdup/cora_bo.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms63970.5%29.5%639100.0%0.0%
Profiles187,25781.3%18.7%187,257100.0%0.0%
Observations17,024,49989.1%10.9%17,024,499100.0%0.0%
Duplicate profiles54,50523.7%187,25729.1%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups11,095
Duplicate profiles23,89843.8%
Platforms178
Observations1,942,74949.7%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups11,479
Duplicate profiles30,60756.2%
Platforms277
Observations1,966,43650.3%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
218,59437,18868.2%
31,8075,4219.9%
48993,5966.6%
55202,6004.8%
63672,2024.0%
71801,2602.3%
8836641.2%
9454050.7%
10272700.5%
11+528991.6%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/cora_bo.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms63169.6%30.4%63998.7%1.3%
Profiles155,32667.5%32.5%187,25782.9%17.1%
Observations15,179,62679.5%20.5%17,024,49989.2%10.8%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/bo_nrt_vs_cora.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms631
Profiles155,326
Profiles without a key0
Observations15,179,626

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
nrt_bo_bo45471.9%49,64632.0%48,93471298.6%

Mediterranean Sea: Near Real Time, regional product (MO)

Regional Near Real Time product. Each source file holds a single platform.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/nrt_mo_mo.yaml.tsv
MetricCount% of files
Files79
with TEMP4759.5%
with PSAL79100.0%
with PRES79100.0%
with DEPH11.3%
with TIME_QC79100.0%
with POSITION_QC79100.0%
Extra parametersAMON, BCCW, CNDC, CPHL, DENS, DOX1, DOX2, DOXY, FLU2, LGH4, LGHT, NRAD, NTRZ, PHOS, POTENTIAL_TEMP, SLCA, TUR3, TUR4

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/nrt_mo_mo.parquet.tsv
MetricCount
Platforms10
Profiles1,652
Observations363,495

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature311,64885.7%00.0%00.0%51,84714.3%
Salinity363,495100.0%00.0%00.0%00.0%
Pressure363,495100.0%00.0%00.0%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/nrt_mo_mo.summary.tsv
  • File: report/convert/checkpres/nrt_mo_mo.platforms.tsv
MetricCountShare
Profiles checked1,655
Profiles without pressure (not checked)0
Profiles with decreasing pressure291.8%
of which reversed00.0%
of which non-monotonic291.8%
Platforms with such profiles220.0%
Files with such profiles22.5%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
29SO272.5%0272015-11-282017-07-03
EABV20.8%022014-05-272014-11-27

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/nrt_mo_mo.parquet.tsv
MetricCount% of originalDeleted
Platforms10100.0%0.0%
Profiles1,652100.0%0.0%
Observations363,495100.0%0.0%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/nrt_mo_mo.parquet.tsv
MetricCount% of originalDeleted
Platforms990.0%10.0%
Profiles1,61597.8%2.2%
Observations311,63385.7%14.3%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/nrt_mo_mo.parquet.tsv
MetricCount% of originalDeleted
Platforms990.0%10.0%
Profiles1,59396.4%3.6%
Observations309,80385.2%14.8%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min-3.204-3.204
Longitude max26.93326.933
Latitude min33.6580.000
Latitude max41.66941.669

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/nrt_mo_mo.parquet.tsv
  • File: output/dedup/markdup/nrt_mo_mo.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms990.0%10.0%9100.0%0.0%
Profiles1,59396.4%3.6%1,593100.0%0.0%
Observations309,80385.2%14.8%309,803100.0%0.0%
Duplicate profiles583.5%1,5933.6%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups15
Duplicate profiles3763.8%
Platforms4
Observations14,97287.5%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups10
Duplicate profiles2136.2%
Platforms2
Observations2,13612.5%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
2183662.1%
361831.0%
4146.9%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/nrt_mo_mo.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms990.0%10.0%9100.0%0.0%
Profiles1,56094.4%5.6%1,59397.9%2.1%
Observations303,86883.6%16.4%309,80398.1%1.9%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/mo_nrt_vs_cora.tsv
  • File: report/compare/mo_nrt_vs_gl.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms9
Profiles1,560
Profiles without a key0
Observations303,868

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
cora_mo9100.0%1,40389.9%1,3871698.9%
nrt_mo_gl00.0%00.0%00

Mediterranean Sea: Near Real Time, global product (GL)

Global (GL) Near Real Time product restricted to this region’s bounding box. It overlaps the regional product, so a profile can appear in both and duplicates are expected.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/nrt_mo_gl.yaml.tsv
MetricCount% of files
Files187
with TEMP18598.9%
with PSAL17995.7%
with PRES11461.0%
with DEPH15582.9%
with TIME_QC187100.0%
with POSITION_QC18699.5%
Extra parametersALK3, ALKY, C13D, C14D, CDOM, CF3W, CFC11, CFC12, CHLT, CNDC, CPHL, DOX1, DOX2, DOXY, EWCT, FLU2, FLUO, HCDT, HCSP, HELD, HELM, HELW, LGH4, LGHT, NEON, NEOW, NSCT, NTAW, NTRA, NTRI, NTRZ, NTZW, OSAT, PCO2, PHOS, PHOW, PHPH, SLCA, SLCW, SSAL, SVEL, TCO2, TRANSMISSIVITY, TRIW, TUR2, TUR3, TUR4, TUR5, TUR6

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/nrt_mo_gl.parquet.tsv
MetricCount
Platforms187
Profiles415,974
Observations167,629,126

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature166,573,22799.4%00.0%689,6750.4%366,2240.2%
Salinity166,740,69099.5%00.0%689,6750.4%198,7610.1%
Pressure167,599,516100.0%00.0%29,6100.0%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/nrt_mo_gl.summary.tsv
  • File: report/convert/checkpres/nrt_mo_gl.platforms.tsv
MetricCountShare
Profiles checked433,306
Profiles without pressure (not checked)0
Profiles with decreasing pressure26,2296.1%
of which reversed6340.1%
of which non-monotonic25,5955.9%
Platforms with such profiles3418.2%
Files with such profiles3418.2%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
FGTO8,93355.2%1408,7932004-09-092025-10-17
FNCM7,93045.7%1287,8022000-01-012023-04-17
FNIN2,67956.9%212,6582012-01-262022-05-16
FMCY2,20020.1%752,1252015-01-122022-02-10
FKJB1,41339.9%721,3411997-07-122023-01-11
FNFP1,0866.2%1968901997-09-032026-02-12
FABB98411.8%09842014-05-242019-05-25
FNOY2265.6%02261972-02-211977-12-15
FITA1635.8%01631972-01-191993-01-14
FZVN1192.0%21171981-04-072012-03-11

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/nrt_mo_gl.parquet.tsv
MetricCount% of originalDeleted
Platforms187100.0%0.0%
Profiles405,66397.5%2.5%
Observations164,518,95798.1%1.9%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/nrt_mo_gl.parquet.tsv
MetricCount% of originalDeleted
Platforms17895.2%4.8%
Profiles373,27489.7%10.3%
Observations155,370,56892.7%7.3%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/nrt_mo_gl.parquet.tsv
MetricCount% of originalDeleted
Platforms17794.7%5.3%
Profiles87,01820.9%79.1%
Observations32,475,56819.4%80.6%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min-5.610-180.000
Longitude max35.567180.000
Latitude min28.378-77.893
Latitude max45.75586.550

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/nrt_mo_gl.parquet.tsv
  • File: output/dedup/markdup/nrt_mo_gl.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms17794.7%5.3%177100.0%0.0%
Profiles87,01820.9%79.1%87,018100.0%0.0%
Observations32,475,56819.4%80.6%32,475,568100.0%0.0%
Duplicate profiles20,0854.8%87,01823.1%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups3,454
Duplicate profiles14,29271.2%
Platforms69
Observations2,536,78649.7%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups2,641
Duplicate profiles5,79328.8%
Platforms52
Observations2,570,75350.3%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
24,4878,97444.7%
32206603.3%
41375482.7%
5723601.8%
6935582.8%
71651,1555.8%
88536,82434.0%
9131170.6%
10202001.0%
11+356893.4%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/nrt_mo_gl.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms17593.6%6.4%17798.9%1.1%
Profiles73,02817.6%82.4%87,01883.9%16.1%
Observations29,862,72017.8%82.2%32,475,56892.0%8.0%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/mo_gl_vs_cora.tsv
  • File: report/compare/mo_nrt_vs_gl.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms175
Profiles73,028
Profiles without a key0
Observations29,862,720

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
cora_mo17197.7%65,70290.0%63,8901,81297.2%
nrt_mo_mo00.0%00.0%00

Mediterranean Sea: CORA reanalysis

CORA delayed-mode reanalysis. Source files hold many platforms each, and the profiles are re-processed rather than near real time.

File summary

Coverage of the core measurement variables and the two profile-level QC flags across the source NetCDF files. A file counts as having one if the variable is present, regardless of how many of its values are valid. “Extra parameters” lists the non-core variables seen in any file.

  • File: report/header/cora_mo.yaml.tsv
MetricCount% of files
Files7,905
with TEMP7,905100.0%
with PSAL7,904100.0%
with PRES7,84999.3%
with DEPH7,86999.5%
with TIME_QC7,905100.0%
with POSITION_QC7,905100.0%
Extra parametersnone

Conversion

Counts after converting the source NetCDF files to Parquet and merging them into one file. The merge sorts each profile’s observations by increasing pressure and drops observations without a pressure value (a profile with no pressure at all is dropped with them); no other profile is dropped yet. These are the “original” numbers every later stage is compared against.

  • File: report/convert/cora_mo.parquet.tsv
MetricCount
Platforms195
Profiles93,575
Observations33,243,462

Data mode

Observations per data mode for each parameter: real-time (R), adjusted in real time (A), or delayed mode (D), as recorded in the source files. Percentages are shares of all observations. Unknown counts observations with no data mode: a fill value in the source file, a file without data-mode information, or a parameter the file does not have.

ParameterReal-time (R)% RAdjusted (A)% ADelayed mode (D)% DUnknown% Unknown
Temperature33,243,462100.0%00.0%00.0%00.0%
Salinity33,243,462100.0%00.0%00.0%00.0%
Pressure33,243,462100.0%00.0%00.0%00.0%

Pressure order

Checks, before the merge, that pressure increases along each profile in the order the source file stores it. A profile is counted when its pressure steps down between two observations; missing pressures are skipped and a repeated pressure is fine. “Reversed” profiles are stored from the bottom up, which the merge’s sort by pressure puts right; “non-monotonic” ones go both up and down, which can mean loops, spikes or misplaced levels, and the sort hides them. Shares are of the profiles, platforms and files checked; profiles without any pressure are not checked (the merge drops them). Nothing is removed here.

  • File: report/convert/checkpres/cora_mo.summary.tsv
  • File: report/convert/checkpres/cora_mo.platforms.tsv
MetricCountShare
Profiles checked100,158
Profiles without pressure (not checked)0
Profiles with decreasing pressure14,57914.6%
of which reversed2230.2%
of which non-monotonic14,35614.3%
Platforms with such profiles2512.8%
Files with such profiles5667.2%

Platforms

The platforms with the most such profiles, at most 10; the full list, and every profile with its time and position, are in the checkpres reports.

PlatformProfiles% of its profilesReversedNon-monotonicFirstLast
FGTO4,96447.5%694,8952004-09-092023-03-08
FNCM4,55553.0%34,5522017-01-272020-10-19
FMCY2,20077.5%752,1252015-01-122022-02-10
FKJB1,41340.6%721,3411997-07-122023-01-11
FABB64214.1%06422016-03-232018-05-13
FNOY17835.4%01781972-02-211975-03-10
FITA15618.1%01561972-01-191985-06-01
FZVN1155.2%01151981-04-071992-04-26
OCL017911226.0%01121986-07-101998-11-10
FZGD5169.9%0511975-01-251975-12-22

Cleaning

Stages that drop whole profiles that are unusable or out of scope. They run in the order below, each on the output of the previous one, so the counts fall monotonically.

Drop bad QC

Drops whole profiles whose profile-level quality flags are bad. A profile is kept only if both its time and position QC are good (flag 1) or absent. Many files ship no profile-level QC, and those profiles are kept.

  • File: report/clean/dropqc/cora_mo.parquet.tsv
MetricCount% of originalDeleted
Platforms19399.0%1.0%
Profiles81,92887.6%12.4%
Observations32,366,52197.4%2.6%

Drop all-NA profiles

Drops whole profiles that carry no valid measurement at all in any one of temperature, salinity, or pressure. A profile is kept only if each of the three has at least one valid observation.

  • File: report/clean/dropna/cora_mo.parquet.tsv
MetricCount% of originalDeleted
Platforms18393.8%6.2%
Profiles78,55383.9%16.1%
Observations31,859,18495.8%4.2%

Filter by region

Keeps only the profiles whose position falls inside the region’s bounding box. A profile with a missing position counts as outside and is dropped.

  • File: report/clean/filter/cora_mo.parquet.tsv
MetricCount% of originalDeleted
Platforms18393.8%6.2%
Profiles78,54283.9%16.1%
Observations31,856,26695.8%4.2%

Bounding box

Geographic extent of the profiles that survived the filter, in decimal degrees. The “Original” column, when present, is the extent of the converted data before any cleaning stage ran, so the two columns show how much the bounding box tightened. Profiles with a missing position are ignored.

MetricFilteredOriginal
Longitude min-5.610-5.610
Longitude max35.56736.280
Latitude min28.37828.378
Latitude max45.75545.755

Deduplication

Stages that find and remove profiles measuring the same time and place. Profiles match on date and rounded longitude/latitude. The platform is not part of the key, so duplicates are found across platforms as well as within one.

Mark duplicates

Flags every profile that shares its (date, position) key with at least one other. Nothing is removed here, so the counts match the previous stage; “Duplicate profiles” is the pool the next stage picks a survivor from.

  • File: report/dedup/markdup/cora_mo.parquet.tsv
  • File: output/dedup/markdup/cora_mo.dups.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms18393.8%6.2%183100.0%0.0%
Profiles78,54283.9%16.1%78,542100.0%0.0%
Observations31,856,26695.8%4.2%31,856,266100.0%0.0%
Duplicate profiles11,77212.6%78,54215.0%

Duplicates within a platform

Duplicate groups whose profiles all carry the same platform code: one platform reporting the same cast more than once. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups2,466
Duplicate profiles6,11551.9%
Platforms71
Observations2,521,84450.2%

Duplicates across platforms

Duplicate groups whose profiles carry two or more different platform codes: the same cast reaching the archive under more than one platform, typically via overlapping products. Percentages are shares of all duplicate profiles.

MetricCount% of duplicates
Duplicate groups2,580
Duplicate profiles5,65748.1%
Platforms53
Observations2,501,11449.8%

Duplicate group sizes

How many profiles each duplicate group holds. A group of 2 is a cast recorded twice; larger groups are the same cast repeated more often. Sizes above 10 are pooled into the final row.

Profiles per groupGroupsProfiles% of duplicates
24,5979,19478.1%
31915734.9%
4863442.9%
5251251.1%
6382281.9%
7201401.2%
8252001.7%
98720.6%
10191901.6%
11+377066.0%

Remove duplicates

Keeps one profile per duplicate key and drops the rest. The survivor is the profile with the most observations, with ties broken by first appearance. Profiles with no key (missing time or position) are never treated as duplicates and always survive.

  • File: report/dedup/dedup/cora_mo.parquet.tsv
MetricCount% of originalDeletedCleaned% of cleanedDeleted (cleaned)
Platforms18192.8%7.2%18398.9%1.1%
Profiles71,81676.7%23.3%78,54291.4%8.6%
Observations29,329,43688.2%11.8%31,856,26692.1%7.9%

Comparison

How this product compares with the other products covering the same region. Each other product is measured with this one as the reference: how many of this product’s platforms and profiles it also holds, with profiles matched on date and rounded longitude/latitude. Coverage is one-directional, so a product that contains this one covers it fully while this one covers only part of it.

  • File: report/compare/mo_gl_vs_cora.tsv
  • File: report/compare/mo_nrt_vs_cora.tsv

This product

This product’s own totals, the denominators the coverage below is measured against. “Profiles without a key” have a missing time or position, so they can never be matched and are counted against coverage.

MetricCount
Platforms181
Profiles71,816
Profiles without a key0
Observations29,329,436

Coverage of other products

One row per other product. “Platform coverage” is the share of this product’s platforms the other also has; “Profile coverage” the share of its profiles. Among the profiles found in both, “Same obs” carry the same number of observations in each product and “Diff obs” differ, which separates the same profiles cleaned differently from genuinely different ones; “Obs agreement” is the share of matched profiles that are the same.

Compared withCommon platformsPlatform coverageMatched profilesProfile coverageSame obsDiff obsObs agreement
nrt_mo_gl17194.5%65,70291.5%63,8901,81297.2%
nrt_mo_mo95.0%1,4032.0%1,3871698.9%