ICOADS Web information page (Monday, 28-Apr-2014 17:05:21 UTC):

Release 1 Trimming Limits: GrADS and ASCII Files

The processing of the "trimming" (quality control) limits, and the format of the resultant Decadal Summary Untrimming Limits (DSUL), are documented in supp. C of COADS Release 1 (Slutz et al., 1985). For easier use, we reformatted the DSUL files into data (and control; ctl) files in the Grid Analysis and Display System (GrADS) format. We also reformatted the fields into a self-describing ASCII format (see Appendix). [Update note: for ICOADS Release 2.5, the relative humidity (RH) trimming limits for 1910-49 were used for all RH data prior to 1910, as discussed further in (Woodruff et al. 2011).]

1. GrADS files

The first part of each file name indicates the variable:

air air temperature
rhum relative humidity
slp sea level pressure
sst sea surface temperature
uwnd vector wind U component
vwnd vector wind V component
and the second part of the file name indicates the trimming quantity included in the file. Specifically, each file contains one of the three quantities from DSUL listed on p. C8 of Release 1, for all three trimming periods:
l smoothed lower limit (g - 3.5 sigma1)
g smoothed median
u smoothed upper limit (g + 3.5 sigma5)
In the DSUL format there was a distinction between a landlocked 2° box, and a non-landlocked 2° box with no limits available. In the GrADS (and ASCII) files, both possibilities were converted into missing data. (Note: it is permissible for the limits for U to be missing but not those for V, or vice versa.) Following are links to the GrADS data (compressed) and control files:
air.g.Z rhum.g.Z slp.g.Z sst.g.Z uwnd.g.Z vwnd.g.Z
air.g.ctl rhum.g.ctl slp.g.ctl sst.g.ctl uwnd.g.ctl vwnd.g.ctl
air.l.Z rhum.l.Z slp.l.Z sst.l.Z uwnd.l.Z vwnd.l.Z
air.l.ctl rhum.l.ctl slp.l.ctl sst.l.ctl uwnd.l.ctl vwnd.l.ctl
air.u.Z rhum.u.Z slp.u.Z sst.u.Z uwnd.u.Z vwnd.u.Z
air.u.ctl rhum.u.ctl slp.u.ctl sst.u.ctl uwnd.u.ctl vwnd.u.ctl

Additional limits files are offered (with "4.5" in the name, after variable) that have been expanded in accordance with the ICOADS "enhanced" trimming procedure:

l4.5 smoothed lower limit (g - 4.5 sigma1)
u4.5 smoothed upper limit (g + 4.5 sigma5)
As part of this expansion, extreme values possible for l4.5 and u4.5 were adjusted to fall within the lower and upper bounds given in Table C2-3 of Release 1. Following are links to these additional GrADS data and control files:
air4.5.l.Z rhum4.5.l.Z slp4.5.l.Z sst4.5.l.Z uwnd4.5.l.Z vwnd4.5.l.Z
air4.5.l.ctl rhum4.5.l.ctl slp4.5.l.ctl sst4.5.l.ctl uwnd4.5.l.ctl vwnd4.5.l.ctl
air4.5.u.Z rhum4.5.u.Z slp4.5.u.Z sst4.5.u.Z uwnd4.5.u.Z vwnd4.5.u.Z
air4.5.u.ctl rhum4.5.u.ctl slp4.5.u.ctl sst4.5.u.ctl uwnd4.5.u.ctl vwnd4.5.u.ctl

2. ASCII files

The ASCII files are in a self-describing format, documented in the Appendix. Unlike the GrADS files, separate ASCII files were produced for the three trimming periods. Therefore, the ASCII file names are prefixed with the concluding year of the trimming period (1854-1909; 1910-49; 1950-79). Otherwise the convention used for naming the ASCII files follows that of the GrADS files. Following are links to the ASCII files:

1909air.g.Z 1909rhum.g.Z 1909slp.g.Z 1909sst.g.Z 1909uwnd.g.Z 1909vwnd.g.Z
1909air.l.Z 1909rhum.l.Z 1909slp.l.Z 1909sst.l.Z 1909uwnd.l.Z 1909vwnd.l.Z
1909air.u.Z 1909rhum.u.Z 1909slp.u.Z 1909sst.u.Z 1909uwnd.u.Z 1909vwnd.u.Z
1949air.g.Z 1949rhum.g.Z 1949slp.g.Z 1949sst.g.Z 1949uwnd.g.Z 1949vwnd.g.Z
1949air.l.Z 1949rhum.l.Z 1949slp.l.Z 1949sst.l.Z 1949uwnd.l.Z 1949vwnd.l.Z
1949air.u.Z 1949rhum.u.Z 1949slp.u.Z 1949sst.u.Z 1949uwnd.u.Z 1949vwnd.u.Z
1979air.g.Z 1979rhum.g.Z 1979slp.g.Z 1979sst.g.Z 1979uwnd.g.Z 1979vwnd.g.Z
1979air.l.Z 1979rhum.l.Z 1979slp.l.Z 1979sst.l.Z 1979uwnd.l.Z 1979vwnd.l.Z
1979air.u.Z 1979rhum.u.Z 1979slp.u.Z 1979sst.u.Z 1979uwnd.u.Z 1979vwnd.u.Z
 
1909air4.5.l.Z 1909rhum4.5.l.Z 1909slp4.5.l.Z 1909sst4.5.l.Z 1909uwnd4.5.l.Z 1909vwnd4.5.l.Z
1909air4.5.u.Z 1909rhum4.5.u.Z 1909slp4.5.u.Z 1909sst4.5.u.Z 1909uwnd4.5.u.Z 1909vwnd4.5.u.Z
1949air4.5.l.Z 1949rhum4.5.l.Z 1949slp4.5.l.Z 1949sst4.5.l.Z 1949uwnd4.5.l.Z 1949vwnd4.5.l.Z
1949air4.5.u.Z 1949rhum4.5.u.Z 1949slp4.5.u.Z 1949sst4.5.u.Z 1949uwnd4.5.u.Z 1949vwnd4.5.u.Z
1979air4.5.l.Z 1979rhum4.5.l.Z 1979slp4.5.l.Z 1979sst4.5.l.Z 1979uwnd4.5.l.Z 1979vwnd4.5.l.Z
1979air4.5.u.Z 1979rhum4.5.u.Z 1979slp4.5.u.Z 1979sst4.5.u.Z 1979uwnd4.5.u.Z 1979vwnd4.5.u.Z

3. Sample plots of smoothed median (g) values

Figures 1 and 3 show the January smoothed median (g) values for relative humidity (RH), sea level pressure (SLP), sea surface temperature (SST), and for air temperature (AT), vector wind U component (U), and vector wind V component (V), for the three trimming periods. Similarly, Figures 2 and 4 show the July g values. For these (and other) months, the RH values are nearly all missing for the earliest (1854-1909) trimming period, due to a general absence of Release 1 humidity data during this period. This posed a problem for blending new data into ICOADS, because we lacked a defined basis for trimming newly available humidity data prior to 1910, and thus for calculating trimmed 2° monthly summaries. For Release 2.5 (Woodruff et al. 2011), however, as noted above, the RH trimming limits for 1910-49 were used for all RH data prior to 1910 (as discussed further in Woodruff et al. 2011). Moreover, it is important to note that the g fields are less complete for SLP (and to a lesser extent for other variables) in the earliest period (e.g., in the Pacific basin), than they are for SST. This could tend to cause similar problems, but on a smaller scale, for the addition of newly available data in those regions prior to 1910.

Figure 1
Figure 1. January g values for RH, SLP, and SST. The values are plotted separately for the three trimming periods (1854-1909, 1910-49, 1950-79) from top to bottom.

Figure 2
Figure 2. As for Figure 1, but for July.

Figure 3
Figure 3. January g values for AT, U, and V. The values are plotted separately for the three trimming periods (1854-1909, 1910-49, 1950-79) from top to bottom.

Figure 4
Figure 4. As for Figure 3, but for July.

References

Slutz, R.J., S.J. Lubker, J.D. Hiscox, S.D. Woodruff, R.L. Jenne, D.H. Joseph, P.M. Steurer, and J.D. Elms, 1985: Comprehensive Ocean-Atmosphere Data Set; Release 1. NOAA Environmental Research Laboratories, Climate Research Program, Boulder, CO, 268 pp. (NTIS PB86-105723).

Woodruff, S.D., S.J. Worley, S.J. Lubker, Z. Ji, J.E. Freeman, D.I. Berry, P. Brohan, E.C. Kent, R.W. Reynolds, S.R. Smith, and C. Wilkinson, 2011: ICOADS Release 2.5: Extensions and enhancements to the surface marine meteorological archive. Int. J. Climatol. (CLIMAR-III Special Issue), 31, 951-967 (doi:10.1002/joc.2103).

Appendix: MANFORMAT-05 Format Description

MANFORMAT-05 is a self-describing ASCII format that was designed primarily to store gridded fields. Further information about the format, and an example Fortran program to read similar files, can be found in this README. The first seven lines of each file comprise a self-describing header (see example below):
  • Line 1: The format. The number "2" signifies the number of text lines (free form) that follow.
  • Line 2: First text-line. Lists the creating process (ASCII.01A), the date and time created, and the source filename.
  • Line 3: Second text-line. Lists the terminating year of the trimming period, and the trimming quantity and variable.
  • Line 4: Text indicates 2° box size and the latitude/longitude range, and "MONTHS LONGITUDE LATITUDE" are descriptors of the three sets of ranges that appear on line 6.
  • Line 5: "12" for the twelve timesteps in the file, 90 and 180 to indicate that there are 90x180 2° latitude by 2° longitude boxes, and the missing data indicator (-9999.00). The trailing-zero field is unused.
  • Line 6: The time and space coverage of the file. Since each file covers 12 months (encoded as 0.00, 0.08, 0.17, 0.25, 0.33, 0.42, 0.50, 0.58, 0.67, 0.75, 0.83, and 0.92), the first two values give the minimum and maximum time values (0.00 = January and 0.92 = December). The next four values give the minimum longitude, maximum longitude, maximum latitude, and minimum latitude (negative latitudes are south, and longitude is measured east from the Prime Meridian), such that latitudes and longitudes correspond to the middle of each 2° box.
  • Line 7: Each file is divided into 90 bands (2° latitude) of data: 180 data values for the 2° boxes in that band (from minimum to maximum longitude), followed by a 181st value giving the central latitude of the band. The Fortran format specification on this line will read a single latitude row of these 181 values. Note that boxes defined in the COADS Release 1 2° box system to store data exactly at the North or South Poles are not included in the files (see Release 1, supp. G).
The data follow the seven header lines. First, the encoded value for January is given on the eighth line, followed by the January data. This structure is repeated for the remaining 11 months. For example, the following shows the beginning (through first latitude band of January) and end (portion of last latitude band of December) of uncompressed file 1909air4.5.l: 

MANFORMAT-05    2
ASCII.01A  05/04/15. 16.17.01. FROM air4.5.l
ICOADS 1909 4.5 SIGMA LOWER LIMIT OF AIR TEMPERATURE (DEG. C)
2 DEGREE 89N-89S 1E-359E   MONTHS   LONGITUDE   LATITUDE
 12    90    180    -9999.00   0
       0.00         0.92      1.00000      359.000      89.0000     -89.0000
 (22(8F9.2,/),4F9.2,F10.4)
       0.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00   89.0000
.
.
.
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00 -9999.00
 -9999.00 -9999.00 -9999.00 -9999.00  -89.0000

[Documentation and Software][Links to additional]

U.S. National Oceanic and Atmospheric Administration hosts the icoads website privacy disclaimer
 Document maintained by icoads@noaa.gov
Updated: Apr 28, 2014 17:05:21 UTC
http://icoads.noaa.gov/dsul.html