ICOADS Web information page (Thursday, 19-May-2016 16:18:34 UTC):
Release 2.5 Data Characteristics
We have completed a major ICOADS update, Release 2.5, covering 1662-2007, and including a wide variety of newly available or improved data inputs (Figure 1).
The remaining sections of this webpage provide temporal and spatial comparisons between final Release 2.5 and Release 2.4 data for 1662-2007, information about a preliminary set of 1662-1969 data (some systematic data continuity problems, mostly during 1941-69, were found in those preliminary data that required a complete rerun), and plots illustrating duplicate elimination (dupelim) performance and quality control results.
The quality control ("trimming" flag) comparisons between decks provided in Figures 7a-7f illustrate some systematic QC problems. For example, the trimming limits (see also Figs. 1-4 on this webpage) are extensively missing, spanning different variables, for near-polar sources (e.g., whaling or expeditionary decks including 186-188, 197, 246, 733-734, 736, 761, 897, 899).
In preparation for the rerun, NODC/OCL World Ocean Database 2005 (WOD05) oceanographic data were re-translated into IMMA format using a new scheme to estimate sea surface temperature (SST) from subsurface ocean profile temperatures. The old (used for Release 2.4 and preliminary Release 2.5 data) and new (used for final Release 2.5 processing) schemes for deriving SST are discussed in this document, with detailed comparison tabulations and charts presented in this spreadsheet.
Figure 1. Major historical data sources added to Release 2.5. Horizontal green lines
illustrate the time range of the original data sources. The annual numbers of reports are
plotted as curves, blue for the previous Release 2.4, and red for Release 2.5. For clarity
the vertical scale is truncated at 9M; years 2005-07 have 13M, 15M, and 16M total
reports (not visible) in R2.5, respectively. Data coverage prior to 1800 is very sparse.
2. Release 2.5 4-panel plots
Figures 2a1 through 2i6 provide 4-panel comparison plots between Release 2.5 data and Release 2.4 data (2° enhanced monthly summaries in both cases) for 20-year periods (for details on the 4-panel plot layout see icoads.noaa.gov/panels.html; note that a minimum of 1000 nobs are required per month, or all curves are omitted to ensure meaningful data patterns).
The humidity plots are omitted for the earliest three periods due to general data sparsity.
3. Release 2.5 map comparisons
Figures 3a-3d are decadal plots (1800-2007) illustrating data additions between Releases 1, 2.4, and 2.5.
The colors show the total number of observations in a 2° box per decade separately for four primary variables (and using the enhanced monthly summaries for both Releases 2.4 and 2.5).
The amounts of data added prior to 1800 were very small, so plots for those decades have been omitted (note: Release 2.4 data began in 1784 and ended in May 2007, and January-May 2007 data are not plotted; and Release 1 data began in 1854).
4. Preliminary Release 2.5 and outstanding data continuity questions
As noted above, some systematic data continuity problems, concentrated during 1941-69, were found in the preliminary 1662-1969 data that required a complete re-run. This webpage describes the overall characteristics of the preliminary data, with links to subsidiary webpages exploring data homogeneity questions for three major variables. This additional webpage describes some outstanding data continuity questions in the final data.
5. Release 2.5 dupelim results
Numbers of reports added to, or deleted from, the output Release 2.5 data during dupelim, separately by deck, covering 1792-1877. The deck 156 reports deleted around 1860, without corresponding additions from other decks, were previously undetected duplicates (i.e., they matched other reports already in ICOADS).
As for Figure 4a, but covering 1878-1949.
As for Figure 4a, but covering 1950-69.
As for Figure 4a, but covering 1970-79.
As for Figure 4a, but covering 1980-2004.
As for Figure 4a, but covering 2005-2007
(through coads_20090304_seas_jan.gz, coads_20090219_keyed_oct.gz, coads_20090206_bufr_jan.gz, and coadsgcc200804.IMMA.r2.5.Z).
The larger plot shows total numbers of reports output in the Release 2.5 data, stratified by deck, for 1830-1969 (Tables D6a-D6c). The smaller plot shows patterns for the period 1750-1834, which is almost exclusively composed of data from decks 730 (CLIWOC) and 701 (US Maury).
Figure 5b. Total numbers of reports output in the Release 2.5 data, stratified by deck, for 1970-2007.
6. Release 2.5 quality control results
In Figures 7a through 7f, bars show the percentage frequency (left axis) of observation (a1) trimming flag value. Specifically, where a1 is the individual observation under scrutiny, g is the smoothed median, and s1 and s5 are the smoothed lower and upper median deviation:
The line (right axis) indicates the total number of extant observations per deck during the period (note: log scale).
Landlocked 2° boxes (LZ=1) were removed in a final Release 2.5 IMMA processing step (Figure 6), or during processing of earlier ICOADS Releases (for previously available data sources).
Decks containing no extant data for a given variable are missing (e.g., deck 736 "Byrd Antarctic Expedition" includes only SLP).
As an important modification to the trimming procedure for Release 2.5, the 1910-49 trimming limits for RH were used for flagging all RH data within the 1662-1909 period. This approach was used because the original 1854-1909 trimming limits for RH generally were missing (see Figs. 1-4 on this webpage, which also reveal smaller gaps in the earlier trimming limits for other variables including SLP and SST).
| E = data unusable (SF, AF, and PF, only)
| C = limits missing (ocean/coastal box)
| B = limits missing (ocean/coastal box); MEDS data correct (SF/PF only)
| 7 = greater than 4.5 sigma upper limit (a1 > g + 4.5*s5)
| 6 = less than 4.5 sigma lower limit (a1 < g - 4.5*s1)
| 5 = greater than 3.5 sigma upper limit (g + 3.5*s5 < a1 <= g + 4.5*s5)
| 4 = less than 3.5 sigma lower limit (g - 4.5*s1 <= a1 < g - 3.5*s1)
| 3 = greater than 2.8 sigma upper limit (g + 2.8*s5 < a1 <= g + 3.5*s5)
| 2 = less than 2.8 sigma lower limit (g - 3.5*s1 <= a1 < g - 2.8*s1)
| 1 = within 2.8 sigma limits (g - 2.8*s1 <= a1 <= g + 2.8*s5)
[Data and Documentation][Documentation and Software]
U.S. National Oceanic and Atmospheric Administration hosts the icoads website
Document maintained by firstname.lastname@example.org
Updated: May 19, 2016 16:18:34 UTC