=============================================================================== International Comprehensive Ocean-Atmosphere Data Set (ICOADS): Release 2.4 Monthly Summary Trimmed Groups (MSTG2) 27 September 2007 ========================================================================= Document Revision Information (previous version: 27 February 2004): Updates (heading only) for Release 2.4. ------------------------------------------------------------------------------- {1. Introduction} This document provides a technical description of the Monthly Summary Trimmed Groups (MSTG) format. This format is designed to store 2-degree latitude x 2-degree longitude monthly summaries. We have transitioned to the Monthly Summary Groups (MSG) format (described in ), which stores both 1-degree and 2-degree data and offers a number of other enhancements. See for information about the details of currently available products and time periods of data available in MSG format. Six "groups" of variables make up MSTG (Table 1a; redundant group 8 is only available starting in 1980), providing relatively compact alternatives to the full Monthly Summaries Trimmed (MST) format (ref. Release 1, supp. A). The eight statistics calculated for each variable are listed in Table 1b. Header fields, as defined below Table 1b, precede the statistics in the MSTG format. The Fortran read program for the MSTG format should provide ready access to the header fields and statistics included in each group (using the abbreviations given in Tables 1a and 1b, and for the header fields as follow Table 1b). As background, however, a technical description of the format layout (bit lengths) is given in sec. 2, and detailed information about the reconstruction of floating point data is given in sec. 3. Table 1a. Variables in MSTG. Each group contains four variables and eight selected statistics (Table 1b) for each variable. Derived variables in groups 5-8 are computed as indicated from individual observations of other variables, e.g., the wind-stress parameter "X" is the product of W and U. In addition, QS denotes saturation Q at SST. [NOTE: Groupings of variables are the same for MSG, except group 8 was created only for Release 1a MSTG and group 9 is not used (groups 1 and 2 are reserved for "untrimmed" Release 1 products).] ------------------------------------------------------------------------------- Group 3 4 5 6 7 8 =============================================================================== Sea surface temp. (S) Scalar wind (W) Total cloud. (C) D=S - A I=UA S Air temperature (A) Wind U-comp. R E=(S - A)W J=VA A Specific humidity (Q) Wind V-comp. X=WU F=QS - Q K=UQ X Relative humidity (R) Pressure (P) Y=WV G=FW L=VQ Y ------------------------------------------------------------------------------- Table 1b. Statistics in MSTG. Each statistic is assigned a number and a lower-case abbreviation (from Release 1, supp. A, Table A1-2). ------------------------------------------------------------------------------- No. Abbrev. Variable =============================================================================== 11 s3 3/6 sextile (the median) 6 m mean 5 n number of observations 15 e standard deviation estimate* 1 d mean day-of-month of observations 2 ht fraction of observations in daylight 3 x mean longitude of observations 4 y mean latitude of observations ------------------------------------------------------------------------------- * Instead of the standard deviation about the mean (s) available in the MSG and MST formats, this robust estimate is computed from the fifth and first sextiles from MST: e = (s5 - s1)/2. ------- Following is a definition of the MSTG header fields preceding the statistics in the MSTG format: RPTIN RPTID The RPTIN field is reserved for use of the RPTIN unblocking utility, where available (e.g., NCAR), and RPTID indicates the MSTG format version number (2). YEAR The year can range from 1800 to 2054. MONTH 1=January, 2=February, ..., 12=December. B2 2-degree box B10 10-degree box See Release 1, supp. G for a description of the 2-degree and 10-degree box systems, and supp. H for related software. GRP group Group number (3-8). CK checksum A checksum was computed and stored with each packed summary as a measure of reliability during storage and transmission. The Fortran read program will recalculate the checksum and compare it to the stored checksum. If disagreement is found data processing will stop and an error statement will be issued. This problem indicates that the data file is corrupted or the access software is not correctly implemented. {2. Details of Monthly Summary Trimmed Groups (MSTG)} Table 2 shows the bit layout in common to any MSTG (regardless of group number); Tables 3a through 3f show the bit layout of each of the 64-bit or 16-bit sections of groups 3-8 (note that Tables B1-1b through B1-1f in Release 1, supp. B give identical bit layouts for groups 3-7). Each variable is assigned a number and an upper-case abbreviation (from Release 1, supp. A, Table A1-1). Example of bit layout: If we denote the lower-case abbreviation for each statistic by "a" and the upper-case abbreviation for each variable by "B", group 3 contains, in order, RPTIN, RPTID, ..., CK followed by: ((aB, B = S,A,Q,R), a = s3,m,n,e,d,ht,x,y) I.e., s3 of S, s3 of A,..., s3 of R; m of S, m of A,..., m of R; ... ; y of S, y of A,..., y of R. The MSTG2 format is a slight modification to the MSTG1 format (described in Release 1, supp. B); following is a summary of changes in comparison to MSTG1: a) In MSTG1, RPTIN was defined as a 16-bit field; in MSTG2, this was redefined as a 12-bit field, with RPTID occupying the other four bits. b) Group number was included as a separate header field. This was accomplished by taking the first four bits from the formerly 12-bit "identification checksum" field. c) Calculation of the checksum differs from that of the previous identification checksum by including the group field in the summation of coded values (instead of adding a separate constant for group number) and by modulo 255 instead of 4095. d) Group 8 was also added in MSTG2 [NOTE: Applicable only to Release 1a and 1b data]. Table 2. The Monthly Summary Trimmed Group (MSTG2) format. Fields added (with respect to MSTG1) are marked by a double asterisk (**), and when the number of bits for a field has changed (*), the old MSTG1 value is shown in parentheses. ------------------------------------------------------------------------------- No. Abbrev. Description Bits =============================================================================== Header fields: RPTIN (reserved) 12*(16) RPTID (reserved) 4** YEAR 8 MONTH 4 B2 2-degree box 14 B10 10-degree box 10 GRP group 4** CK checksum 8*(12) Statistics: 11 s3 3/6 sextile (the median) 64 6 m mean 64 5 n number of observations 64 15 e standard deviation estimate 64 1 d mean day-of-month of observations 16 2 ht fraction of observations in daylight 16 3 x mean longitude of observations 16 4 y mean latitude of observations 16 total 384 = 48B ------------------------------------------------------------------------------- Table 3a. Group 3, 64-bit or 16-bit sections. ------------------------------------------------------------------------------- No. Abbr. Variable Bits Bits =============================================================================== 1 S sea surface temperature 16 4 2 A air temperature 16 4 8 Q specific humidity 16 4 9 R relative humidity 16 4 total 64 16 ------------------------------------------------------------------------------- Table 3b. Group 4, 64-bit or 16-bit sections. ------------------------------------------------------------------------------- No. Abbr. Variable Bits Bits =============================================================================== 3 W scalar wind 16 4 4 U vector wind eastward component 16 4 5 V vector wind northward component 16 4 6 P sea level pressure 16 4 total 64 16 ------------------------------------------------------------------------------- Table 3c. Group 5, 64-bit or 16-bit sections. ------------------------------------------------------------------------------- No. Abbr. Variable Bits Bits =============================================================================== 7 C total cloudiness 16 4 9 R relative humidity 16 4 14 X WU 16 4 15 Y WV (14-15 are wind stress parameters) 16 4 total 64 16 ------------------------------------------------------------------------------- Table 3d. Group 6, 64-bit or 16-bit sections. ------------------------------------------------------------------------------- No. Abbr. Variable Bits Bits =============================================================================== 10 D S - A = sea-air temperature difference 16 4 11 E (S - A)W = sea-air temperature difference*wind magnitude 16 4 12 F QS - Q = (saturation Q at S) - Q 16 4 13 G FW = (QS - Q)W (evaporation parameter) 16 4 total 64 16 ------------------------------------------------------------------------------- Table 3e. Group 7, 64-bit or 16-bit sections. ------------------------------------------------------------------------------- No. Abbr. Variable Bits Bits =============================================================================== 16 I UA 16 4 17 J VA 16 4 18 K UQ 16 4 19 L VQ (16-19 are sensible and latent heat transport parameters) 16 4 total 64 16 ------------------------------------------------------------------------------- Table 3f. Group 8, 64-bit or 16-bit sections. ------------------------------------------------------------------------------- No. Abbr. Variable Bits Bits =============================================================================== 1 S sea surface temperature 16 4 2 A air temperature 16 4 14 X WU 16 4 15 Y WV (14-15 are wind stress parameters) 16 4 total 64 16 ------------------------------------------------------------------------------- {3. Reconstruction of floating point data} Refer to for background and details, including the coded and true value ranges, the units, and the base associated with each header field, statistic, and variable. Table 4 provides information about these quantities only in the few cases where they differ in the MSTG format. The centroids of observational location in time and space are shortened in length and precision from their representation in MST, as discussed in detail in Release 1, supp. B. Table 4. Unpacking header fields and statistics in MSTG (header fields and statistics not listed are as described in ). For unpacking purposes, the standard deviation estimate (e) is treated exactly like the corresponding standard deviation (s) of each respective variable. Notation is as follows: m:n denotes m through n inclusive; @ is used as a plain text abbreviation for the degree symbol. "Units" gives the smallest increment of the data that has been encoded; thus a change of one unit in the integer coded value represents a change in the true value of one of the units shown (units of 1 are explained in the text). ------------------------------------------------------------------------------- Abbr. Description True value Units Base Coded =============================================================================== Header fields -------------------------------------------- B2 2-degree box 1:16202 1 0 same B10 10-degree box 1:648 1 0 same GRP group 3:8 1 0 same Statistics -------------------------------------------- x mean longitude of observations 0.1:1.9 0.2 -0.5 1:10 y mean latitude of observations 0.1:1.9 0.2 -0.5 1:10 -------------------------------------------------------------------------------