Release 1c (1784-1949) Overview

1. Introduction

Final Release 1c (1784-1949) observational data were completed in February
2001, and are available in Long Marine Report Fixed-length (LMRF) format.
This page provides some background, plus comparisons against original COADS
Release 1 data for 1854-1949.  Similar comparison results using a "beta"
(preliminary) version of the Release 1c LMRF that became available in late
2000 are described here:
     icoads.noaa.gov/r1c_beta.html

Release 1c monthly summary statistics (1800-1949) first became available
in March 2001.  These were impacted by a report-rejection problem that
significantly reduced numbers of observations and spatial coverage in
early decades (especially prior to 1860).  This also impacted some users
of LMRF data.   During August-September 2001 the problem was corrected (details
here), and Figs. 2a-2f and 3a-4e on this webpage were updated.


2. Data source additions

Following is a brief discussion of the new input datasets.  Report counts are
as input to dupelim, after initial quality controls and conversion to LMR (in
some cases significant reductions in report counts occur as a result of later
dupelim processing).  Note that some counts reflect changes from earlier (raw)
report estimates (e.g., the Maury Collection was previously estimated at 1.4M
reports).

a) Blend of the UK Main Marine Data Bank (MDB) with COADS for the period
   1854-1949 (decks 201-255; 12.1M reports):  The MDB also included copies
   of several decks the UK received many years ago from NCDC in TDF-11
   format.  These decks were deleted from MDB (9.6M reports during 1854-1949)
   prior to this processing stage, since more original versions of the TDF-11
   data already exist within COADS.  Comparisons are made between the TDF-11
   data available from NCDC versus from MDB, and between two German decks (192
   in COADS, and 215 in MDB), here:
        icoads.noaa.gov/mdb_tdf11.html
   The two German decks were derived from the same punched card format, but
   both decks appear to provide some unique coverage.

b) US Maury Collection (deck 701; 1784-1863; 1.3M reports):  This deck
   provides the only data for 1784-1803, and substantial new data additions
   after that.

c) Norwegian Logbook Collection (deck 702; 1867-89; 201K reports).

d) Japanese Kobe Collection data (1998 Edition; deck 762; 1890-1932; 1M reports):
   These are data more recently keyed by Japan (decks 118-119, which are among
   the COADS Release 1 data, were keyed in the 1960s).

e) US Merchant Marine 1912-46 Collection (decks 705-707; 3.5M reports): A few
   data also included back to 1910.

f) Russian Makarov Collection (deck 731; 1804-1891; 3.5K reports): 27 ships  
   including the "Vitiaz" in two partially overlapping collections.
   
g) World Ocean Database 1998 (WOD98; deck 780; 405K reports), including sea
   surface temperature estimates derived from the uppermost layers of ocean
   profiles, and some surface meteorological fields (CTD and XBT archives
   were outside of the Release 1c period).

h) Arctic drift stations (deck 734):  For this period the deck includes two
   Norwegian ships overwintering in the Arctic:
     i) Data from the North Polar expedition of the "Fram" (1893-96, North
        of 76N; 8K reports) were obtained from Volker Wagner at the Deutscher
        Wetterdienst (German Weather Service), and with the assistance of the
        US National Snow and Ice Data Center (NSIDC).
     ii) NCDC-keyed data covering 1922-24 (7K reports) from the North Polar
         expedition with the "Maud" (1918-25) were obtained also with the
         assistance of NSIDC.

i) Russian AARI North Pole (NP) Station (manned drifting ice floe) data from
   the Polar Science Center (deck 733; NP-1 for 1937-38; 1K reports).

j) Russian MARMET (deck 732; 268K reports starting about 1888) marine
   meteorological archive (previously known as MORMET).

Figures 1a-1e are time-series plots illustrating data composition and
additions made to the Release 1c period (and for Fig. 1e also to the Release
1b 1950-79 period), in comparison to existing Release 1 data or MDB data:
     Figure 1a: Dupelim output: 1792-1853 deck composition.  Colored bars are
     used for decks new to Release 1c (only the Maury Collection provided
     visible amounts of data, and no comparison is shown with Release 1, since
     it began in 1854).
     Figure 1b: Dupelim output: 1854-1901 deck composition.  Decks originally
     used for Release 1, and also output for Release 1c, are aggregated within
     the white bar.  Colored bars are used for decks new to Release 1c.
     The line shows the total dupelim output for Release 1, for comparison.
     Figure 1c: Dupelim output: 1902-1949 deck composition (otherwise as for
     Fig. 1b).
     Figure 1d: Global source "deck" makeup of COADS Release 1c, grouped
     according to roughly national categories, plus Historical Sea Surface
     Temperature (HSST) Project data (color bars; thousands of marine reports
     per year).  Note the importance of Netherlands data, e.g., during 1860-80.
     The line shows the total dupelim output for Release 1, for comparison.
     Figure 1e: MDB regular data (dark blue bars) and MDB TDF-11 data (light
     blue bars), received from the UK for 1854-1979.  The lines compare total
     dupelim output for Releases 1 (red) versus Releases 1c and 1b (black).
     Table 1 provides a decadal comparison of the counts for these categories
     for 1854-1949, and Table 2 provides an annual comparison for 1854-1979.

Table 1.  Decadal counts (over 1854-1949) of MDB regular and MDB TDF-11 data,
and total MDB received from the UK, compared to dupelim output for Release 1
and for Release 1c.  See Table 2 for additional details.
-------------------------------------------------------------------------------
                       MDB             
           ---------------------------
Decade     Regular    TDF-11     Total       COADS Release 1     Release 1c
===============================================================================
1854-59      14182    235062    249244            258213           729473
1860-69      30203    643914    674117            700360           765807
1870-79     130305    475564    605869            707475           847090
1880-89     435045    525853    960898           1427965          1495344
1890-99     563915    459331   1023246           2193351          2219134
1900-09    1810663    483696   2294359           3801766          4195232
1910-19    2461550    812877   3274427           3796116          5406078
1920-29    2978271   1485263   4463534           4325972          6361908
1930-39    2790768   2690359   5481127           5342787          7441693
1940-49     894270   1798735   2693005           2890426          3558144
Total     12109172   9610654  21719826          25444431         33019903
-------------------------------------------------------------------------------

Figures 2a-2f are map comparisons of Release 1 and 1c coverage (based on
     untrimmed data):
     Figure 2a: Decadal totals of sea surface temperature (SST) observations
     (70°N-78°S; 68°W-68°W) for seven earlier decades (1810-19, ..., 1870-79).
     Older data available in COADS Release 1 are shown on the left; those plus
     new data added for Release 1c are shown on the right.  The colors show the
     number of observations in a 2° box per decade.  Grey indicates 1-9; red
     indicates 10-99; orange indicates 100-399; yellow indicates 400 or more.
     Figure 2b: As for Figure 2a, except for sea level pressure (SLP).
     Figure 2c: As for Figure 2a, except for scalar wind (WSPD).
     Figure 2d: As for Figure 2a, except for seven later decades (1880-89,
     ..., 1940-49).
     Figure 2e: As for Figure 2b, except for seven later decades (1880-89,
     ..., 1940-49).
     Figure 2f: As for Figure 2c, except for seven later decades (1880-89,
     ..., 1940-49).

Notes:
     i) The Release 1 dupelim output as shown in Figures 1a-1d is based on
	counts of Compressed Marine Reports (CMR), in which uncertain
	duplicates had been removed.  The Release 1c output is based on LMR,
	but the counts were reduced to account for most uncertain duplicates
	(landlocked reports, which were few in number, were retained for a
	better correspondence with CMR).  After adjustment the counts
	correspond approximately to LMRF and CMR counts.  (This approximate
	relationship is not estimated to have a major impact on item ii.)
    ii) In many cases in Figs. 1b-1c (e.g., 1868-1901), fewer reports from
        Release 1 decks were output (i.e., the line is above the tops of the
        white bars).  We think this usually indicates that decks new to
        Release 1c were selected instead.  For example, some of the UK MDB
        decks thereby replaced inferior or less complete copies of the data
        that were originally included in Release 1.


3. Data problems

a) US Maury Collection:
        i) Data for 1784-1796 are discontinuous (missing months and years;
           as listed at the following translation link).
       ii) Detailed information about the translation of this Collection
           to LMR format, including corrections made to temperatures, is
           available here:
                icoads.noaa.gov/maury.html
           Further examination of the temperatures and other data was planned
           at NCDC (e.g., to explore whether Reaumur temperatures are embedded
           among those now labeled Celsius).
      iii) Until about 1850, many ships reported only wind data.  Wind force
           terms, rather than numeric codes, frequently were reported.  Also,
           many of the early terms predated Beaufort's terminology.  Thus,
           without extensive research into the supplemental data containing
           these original terms, we were not able to establish equivalences
           to wind speeds in the LMRF format.
       iv) Indirect documentation suggests that the wind directions may be
           magnetic, but no adjustment has been made.  Based on preliminary
           inquiries with NOAA/NGDC, it may be possible someday to locate
           fields or models for historical magnetic declination.
        v) In the absence of any metadata as to instrument type, barometers
           were all assumed to be mercurial and adjusted for gravity, and
           also adjusted for temperature if attached thermometer data
           were available (a flag was set indicating whether one or both
           corrections were made).  Comparisons with other historical data
           (Todd Mitchell, personal communication) indicate that the adjusted
           pressures in the Collection may contain biases.

b) Dutch (deck 193) sea level pressures:  Pressures were recovered from the
   supplemental attachment and adjusted for gravity.  This accounts partially
   for large increases in pressure data coverage (see sec. 4) particularly
   in the 19th century.  Initially there was concern that a small percentage
   of the data were from aneroid barometers, and thus should not be adjusted
   for gravity, but Wallbrink et al. (2003) discounted this possibility.


4. Comparisons of near-global (62°N-62°S) time-series using colocated 2° boxes

Comparisons between COADS Release 1 and 1c data using year-month summaries for
2° boxes are presented in the 4-panel plots listed below, separately for two
periods: 1854-99, and 1900-49.  The Release 1 summaries were trimmed (labeled
"R1"), whereas the Release 1c summaries were untrimmed (labeled "R1c un6"; for
details see Note below).  In both the Release 1 and 1c summaries, ships and any
other available platform types are included (e.g., oceanographic data that
became available starting in the late 19th century).

A brief description of the information presented in the four panels follows
(details are available here):
     a) Departures (area-weighted): 2° boxes were included only if they
        possessed data for a given year-month from both Releases 1 (black) and
        1c (green).  This ensured a comparable grid within each monthly time
        step, but not a frozen grid through time.  Departures were calculated
        with respect to a basic 1950-79 Release 1 long-term monthly mean (LTM).
     b) Differences: between 2° monthly means for Release 1 minus 1c (black =
        area-weighted, green = non-area-weighted).  This may be a larger set of
        boxes, because there was no requirement that the LTM be available
        (plots c and d are also based on this potentially larger set of boxes).
     c) 2° boxes: number of colocated 2° boxes (black), and that number plus
        the number of non-colocated 2° boxes containing only Release 1c data
        (green).  Non-colocated boxes from Release 1, if any, are not shown.
     d) Numbers of observations (nobs): Using only the set of colocated boxes,
        for Release 1 (black) and 1c (green).  Any additional observations in
        2° boxes outside the set of colocated boxes are not shown.

     Figure 3a: 1854-99: Sea surface temperature.
     Figure 3b: 1854-99: Air temperature.
     Figure 3c: 1854-99: Scalar wind.
     Figure 3d: 1854-99: Sea level pressure.
     Figure 3e: 1854-99: Total cloudiness.

     Figure 4a: 1900-49: Sea surface temperature.
     Figure 4b: 1900-49: Air temperature.
     Figure 4c: 1900-49: Scalar wind.
     Figure 4d: 1900-49: Sea level pressure.
     Figure 4e: 1900-49: Total cloudiness.

Note: The Release 1c untrimmed summaries were calculated from LMR adjusted to
be comparable to LMRF; specifically this involved eliminating most uncertain
duplicates (i.e., retaining only DS=0-2 and 6), and landlocked reports (i.e.,
retaining LZ not equal to 1).


Reference

Wallbrink, H., F. Koek, G.P. Können, and T. Brandsma, 2003: Sea-level pressure
observations from Dutch ships 1854-1938 incorporated in COADS Release 1c
climatology.  Int. J. Climatol., 23, 471-475.

[Documentation and Software][Delayed-mode (ICOADS.DM) Archive]