![]()
| ||||||
ICOADS Web information page (Tuesday, 11-Feb-2014 19:21:56 UTC): Release 1a (1980-97) Overview1. Background An update and extension of Release 1a for 1980-97 (previously 1980-95) was completed in May 1999. This involved new data and other revisions for the previously available 1980-95 period, plus the extension through 1997 (an initial version of this update was completed in March 1999, but data for 1982-85 subsequently were re-processed to correct a GTS cloud data problem, discussed in section 3, item d). 226M input records were processed; yielding 67.7M output Long Marine Reports (LMR), and 66.7M LMRF (fixed-length format) after removal of 1% of the LMR data as "uncertain" (possible) duplicates and landlocked data. The input mixture involved large new redundancies from alternative US GTS datastreams, but most of these data were limited to test usage only during this update as we explore data quality issues in more detail. Data for 1994-97 were impacted by complex changes due to system and format changes by major data providers, which took extra time to resolve. Some problems are not yet fully resolved (as discussed in section 3). 2. Data source additions and modifications Following are major new data additions, and modifications of previously available data sources, made to the Release 1a period: a) Merger of the UK Main Marine Data Bank (MDB) with COADS for the period 1980-94 (2% of total output LMR, and 3% of 1980-89 period LMR). b) Replacement of post-1990 TOGA/TAO GTS data with "standard archive" data obtained directly from PMEL (deck 144) offering improved diurnal coverage and additional quality control. PIRATA buoys (tropical Atlantic) are now part of this archive. Also, a problem involving undetected duplicates (~24K reports) in 1989-90 deck 144 data was resolved. c) Data were added from the World Ocean Database 1998 (WOD98; deck 780), including sea surface temperature estimates derived from the uppermost layers of ocean profiles, and some surface meteorological fields. d) Correction of a problem with MEDS (deck 714) drifting buoys with misassigned (+1) day for 1980-1985, only impacting buoys reporting in the last quarter of the day. Also, for 1993-97, our processing of the MEDS data was modified to accept reports flagged doubtful by MEDS (and revised 1993 data were obtained from MEDS), resulting in large increases in available drifting buoy data for 1993-97 (this problem stemmed from changes at ARGOS; see section 3, item e for further information). e) Usage of US Air Force Global Weather Central (GWC; deck 888) GTS data (and experimental usage of US Navy GTS datastreams) to fill data gaps and advance toward a more complete and higher-quality GTS archive. Previously missing US Navy Autodin (deck 889) data for March 1980 and January 1989 were added (an additional gap, likely not resolvable, still exists in April 1988). f) Russian AARI North Pole (NP) Station (manned drifting ice floe) data from the Polar Science Center (deck 733; through 1991). g) French logbook (deck 926) data through 1988 with corrected longitudes replaced mislocated data (the problem was confined to 90E-90W across the dateline, and may have extended only back to about 1981). Tests during duplicate elimination (i.e., seeking matches with a mislocated copy of the correction data) indicated that the problem was successfully resolved. h) New Shipboard Environmental (Data) Acquisition System (SEAS; deck 874) data were included for 1995-97. i) Additional Russian MORMET (deck 732) data. 3. Additional data problems We are still exploring the extent and ramifications of some of the following problems. More information will be available at a later date, and we may consider remedial processing for selected time periods if problems are later judged serious enough: a) The quality and completeness of available GTS and US-keyed logbook data deteriorated during recent years (1994-97), stemming from a variety of system and format changes. Progress was made in comparison to previously available 1994-95 data (e.g., large numbers of buoy reports previously misidentified as reports with generic ID "SHIP" were properly assigned during 1995); however, more work is needed (note: there is a residual problem with misidentified "SHIP" reports in October 1994). Efforts are planned in the future to more fully resolve some problems by re-processing of original GTS archives (e.g., UK and NCEP). b) Extant wind speed indicators (WI; 3 = knot, estimated) in NCEP data (decks 892-896) are in error for approximately 19 April-21 October 1997, since the initial BUFR format used at NCEP during this portion of 1997 failed to carry WI. We continue to examine questions about the reliability of WI and other key indicators in ship (GTS and logbook) data sources. c) Certain temperature values in NCEP data also are subject to small (0.1C) errors, due to the method of storage in BUFR (corrected for data starting approximately 17 February 1999). d) In the initial version of the 1980-97 data completed in March 1999, US Air Force GWC (deck 888) GTS data frequently were selected over logbook or other GTS sources, during late 1982 through mid-1985, for reasons related to the handling of WMO Ship Code changes in 1982. We later discovered that many GWC reports during this period also contained erroneous (zero) total cloud amounts. Thus the 1982-85 period was re-processed in May 1999 to entirely delete deck 888. Deck 888 also was frequently selected during the initial NCEP BUFR transition period (approximately April through October 1997); we are still looking into the reasons for this more recent selection pattern. e) The change in our handling of MEDS QC flags (1993-97; section 2, item d) will result in some additional reports with genuine location or other problems being accepted. Such data may also be accepted into later COADS statistics (2-deg and 1-deg) for areas without trimming limits. Detailed background: Prior to 1993, reports were generally flagged due to unrealistic movements in the buoy trajectory detected during MEDS track checking. Starting in 1993, however, MEDS started flagging large numbers of reports because the position was calculated from a different satellite pass (i.e., the time of the buoy position did not correspond to the time of data), in response to apparent software changes at ARGOS. By retaining these reports we are now still accepting some data with non-trivial position (track check) problems. At present, we do not have enough information in the available MEDS format to determine why data were flagged, but work is underway to estimate the extent of remaining problems. f) WOD98 data were "passed through" duplicate elimination (unique data not available from other sources). However, possible duplication may arise, e.g., if oceanographers utilized data also reported from the bridge via regular synoptic ship reports. 4. Noteworthy transitions in the data a) Ship data coverage has been declining since peaking in the late 1980s. Although influenced by delays in the receipt of keyed ship logbook data (e.g., significant amounts of international logbook data became available for 1988-89 as part of this update), most of the change apparently stems from declines in global shipping or other transitions in the ship data. b) Conversely, there has been substantial growth in the numbers of drifting and moored buoy reports since 1980, including crucial TOGA/TAO arrays. Drifting buoys provide wide coverage in the global ocean (generally only temperature and pressure data, however). There was steady growth, e.g., in NDBC (deck 883) moored buoy and coastal (hourly and recently some sub-hourly) data until 1996, but a decline in 1997. c) The PMEL EPOCS data (deck 145) are all daily averages, and PMEL TOGA/TAO data prior to the early 1990s have averaging varying from 2-8 hours. d) Fishing fleet data (IATTC; deck 667) are subject to differences in observing practices compared to ordinary ship data. Due to the nature of the fishing activities, a bias toward conditions associated with calm winds has been detected in these data. e) NCEP changed procedures around the end of 1994 to accept more hourly moored buoy data (and possibly hourly data from other platform types). f) A gap existing in NCDC's NCEP archive for 2-8 November 1986 is filled from GWC data (deck 888) and other sources (missing NCEP data have been supplied by NCAR to NCDC, but a reconversion from NCEP's format is still required).
[Documentation and Software][Delayed-mode (ICOADS.DM) Archive]
U.S. National Oceanic and Atmospheric Administration hosts the icoads website privacy disclaimer Document maintained by icoads@noaa.gov Updated: Feb 11, 2014 19:21:56 UTC http://icoads.noaa.gov/r1a.html |