=============================================================================== International Comprehensive Ocean-Atmosphere Data Set (ICOADS): Release 2.1 10-deg Box Assignment Conventions at Location Discontinuities 27 February 2004 ===================================================================== Document Revision Information (previous version: 9 September 2002): Updates for Release 2.1 and ICOADS. ------------------------------------------------------------------------------- {1. Introduction} Ordinarily, 10-degree box (B10) and Marsden Square (MSQ) assignments follow the historical convention that the two inclusive boundaries of each box are those that form the corner nearest the intersection of the equator and prime meridian (0N; 0E). For example, a report (with location to the nearest tenth of degree) from 110.0W through 119.9W and from 10.0N through 19.9N would be assigned B10= 274 (MSQ=048) (see Release 1, supp. G for more information). However, reports falling directly on the Greenwich Meridian (0 longitude) or 180 longitude, or along the Equator (0 latitude) or at either Pole (+90 or -90 latitude) are subject to special problems in assignment of B10 and MSQ. We refer to these boundaries as "location discontinuities" because the aforementioned historical convention does not explicitly accommodate them. According to WMO conventions, reports along these location discontinuities are assigned WMO quadrant, applicable to current International Maritime Meteorological (IMM) logbook or Global Telecommunication System (GTS) data, according to the "observer's choice." In IMM data prior to 1 July 1996, and possibly in early GTS data, observer's choice (or a processing decision by individual countries) applied also along the additional boundaries of WMO octants enclosed within quadrants. Early ship logbook data prior to WMO's 1963 definition of the IMM exchange may have followed different or unknown conventions. A further complication is that NCDC has employed a different quadrant numbering system, for the same WMO-defined quadrant areas, for Tape Deck-11 (TD-11) data. Figure 1 illustrates the quadrant and octant numbering systems. 180W Western Hemisphere 0 Eastern Hemisphere 180E I--------------I---------------I--------------I---------------I 90N I NCDC quadrant=1 I NCDC quadrant=2 I I WMO quadrant=7 I WMO quadrant=1 I I octant=1 I octant=0 I octant=3 I octant=2 I I--------------I---------------I--------------I---------------I 0 lat I NCDC quadrant=3 I NCDC quadrant=4 I I WMO quadrant=5 I WMO quadrant=3 I I octant=6 I octant=5 I octant=8 I octant=7 I I--------------I---------------I--------------I---------------I 90S Figure 1. NCDC and WMO quadrant numbering, and WMO octant numbering. Differences in the archiving and processing of logbook versus GTS data may introduce problems along location discontinuities. For example, it appears likely that reported quadrant was preserved in logbook data (although octant choice within quadrant was not preserved in TD-11 data). In contrast, some GTS data as gathered by Operational Centers retained only pure latitude and longitude. Later ICOADS duplicate elimination processing, which currently operates on a 10-degree/1-degree box sort, may therefore be unable to locate GTS and logbook duplicates, or even logbook-logbook (or GTS-GTS) duplicates processed according to different conventions. Resolution of these problems will require a total re-sort of the database, hopefully as part of future ICOADS processing, for which we propose utilizing a new "Uniform Convention," defined in sec. 2. Reprocessing of old data should also consider the practicality of regeneration, and retention in supplemental data, of original quadrant/octant assignment. To gradually phase in the Uniform Convention for new data receipts, we are implementing it for new format translations, and hopefully as part of anticipated upcoming changes to NCDC's marine processing systems. Background is provided in sec. 3 on the conventions used for Release 1 and in currently available data, for processing of reports at the location discontinuities. Information is also provided on some past systematic assignment problems that are known to exist. {2. Uniform Convention for location discontinuities} Part a) of the Convention discusses retention of original quadrant/octant information, part b) defines a predictable box (B10 and MSQ) assignment for data along the location discontinuities, and part c) deals with assignment of NCDC quadrant in the event original quadrant/octant information is not available. a) Retention of originally reported quadrant/octant information: i) TD-11 formats (containing quadrant and MSQ): If original information is available (e.g., in IMMT data), it should be preserved by assignment of the corresponding NCDC quadrant (octant choice may not be fully preserved, because each quadrant encloses two octants). NCDC quadrant assignment therefore may or may not agree with MSQ assignment as defined in item b). (Note that future revisions of TD-11 conversion programs will thus need to accommodate matching and mismatching of quadrant and MSQ along the location discontinuities.) ii) LMR format (containing B10 and pure latitude and longitude): The LMR format lacks quadrant/octant so this information should henceforth be preserved in the supplemental attachment (as programs are written to convert new formats, or at the time of revision of existing conversion programs as appropriate). iii) For future re-conversion of already processed LMR data from formats that possessed original quadrant/octant information, we should consider the practicality of using existing B10 and any available supplemental data to reconstruct quadrant/octant for retention in a redefined supplemental attachment. However, questions of practicality arises because if the supplemental attachment for a given format is redefined, every report would need to be reconfigured (not just those along the location discontinuities). Also, for reports derived from TD-11, as discussed in sec. 3, questions arise along 180 longitude of whether the input MSQ or the mismatching quadrant reflects the original quadrant/ octant assignment. b) Assignment of B10/MSQ: i) If exactly on either 0 or 180 longitude, assign to the Western Hemisphere, where B10 (MSQ) assignment is based on latitude. ii) If exactly on the Equator, assign to the Northern Hemisphere, where B10 (MSQ) assignment is based on longitude. iii) If exactly +90 or -90 degrees latitude, use longitude to determine B10 (MSQ) assignment, i.e., to B10=1-36 (MSQ=901-936) or to B10=613-648 (MSQ=588-623). c) Assignment of NCDC quadrant without original quadrant/octant information (in conformance with part b): i) If exactly on either 0 or 180 longitude, assign to the Western Hemisphere, where quadrant (1 or 3) is based on latitude. ii) If exactly on the Equator, assign to the Northern Hemisphere, where quadrant (1 or 2) is based on longitude. iii) If exactly +90 or -90 degrees latitude, use longitude to determine quadrant (1, 2, 3, or 4). For technical and historical reasons, the Uniform Convention makes B10 or MSQ assignment of data along the Equator into the Northern Hemisphere, and along both the Greenwich Meridian and 180 longitude into the Western Hemisphere. Full implementation of the Convention will ensure that duplicates along the location discontinuities receive predictable B10 or MSQ assignments. However, subsequent selections of data for users from a B10- or MSQ-sorted archive will also require attention to the Convention. For example, a request for global data covering 10S latitude up to and including the Equator must include access to B10=289-342 (MSQ=001-036). Furthermore, generation of 1-degree or 2-degree latitude x longitude statistics may be impacted by the Convention (assignments of data to 2-degree or 1-degree boxes have thus far been determined by using the 2- or 1-degree box enclosed by the pre-assigned B10). Table 1 provides a set of representative examples illustrating the Uniform Convention. Table 1. Representative examples of the Uniform Convention for assignment of B10, MSQ, and NCDC quadrant (Q), for data including, or not including, original WMO quadrant information (or WMO octant information mapped into quadrant). =============================================================================== Lon Lat B10 MSQ Original WMO quadrant information: No information: (deg (deg Original = Resultant Resultant E) +N, -S) WMO-Q = NCDC-Q NCDC-Q ------------------------------------------------------------------------------- 0 90 33 901 7 = 1 1 = 2 1 0 0 321 001 7 = 1 1 = 2 5 = 3 3 = 4 1 0 -90 645 588 5 = 3 3 = 4 3 90 90 7 927 1 = 2 2 90 0 295 027 1 = 2 3 = 4 2 90 -90 619 614 3 = 4 4 180 90 16 918 7 = 1 1 = 2 1 180 0 304 018 7 = 1 1 = 2 5 = 3 3 = 4 1 180 -90 628 605 5 = 3 3 = 4 3 270 90 24 910 7 = 1 1 270 0 312 010 7 = 1 5 = 3 1 270 -90 636 597 5 = 3 3 ------------------------------------------------------------------------------- {3. Handling of location discontinuities for Releases 1 and 2.1} Table 2 and Figure 2 show that different conventions were used for Release 1 and currently available delayed-mode (ICOADS.DM) data. Table 2. Release 1 and ICOADS.DM conventions for assignment of B10 at location discontinuities (a-f in Figure 2), depending on whether the input data were with or without quadrant/octant (Q/O). Quadrant used here is NCDC quadrant (see Figure 1). =============================================================================== Release 1* ICOADS.DM** ------------------------------ ------------------------------- lat lon #1) with Q/O #2) without Q/O #3) with Q/O #4) without Q/O ------------------------------------------------------------------------------- a) 90N any B10=1 B10=1 per lon per lon b) any 0 B10 in Q/O larger B10 (q=2/4) B10 in Q/O B10 in Q=1/3 c) any 180 B10 in Q/O larger B10 (q=1/3) B10 across Q/O& B10 in Q=1/3& d) 0 any B10 in Q/O larger B10 (q=3/4) B10 in Q/O B10 in Q=1/2 e) 0 0 B10 in Q/O larger B10=358 (q=4) B10 in Q/O B10 in Q=1 f) 0 180 B10 in Q/O larger B10=340 (q=3) B10 across Q/O& B10 in Q=1& g) 90S any B10=648 B10=648 per lon per lon ------------------------------------------------------------------------------- * Note that the input Q/O was not retained in Attm4, although this information should have been retained indirectly (except at the poles) because of B10 selection in accordance with the stated rules (i.e., "in Q/O"). However, see for a discussion of the "field 104" problem. The HSST Exchange format contained octant; all other Release 1 inputs were TD-11 formats containing quadrant. Thus the "without Q/O" rules (Release 1, p. G1) probably were never invoked. ** Note that any input Q/O was not retained in Attm4, although this information should have been retained indirectly through B10 selection in accordance with the stated rules. & Only along 180 lon, the current NCDC algorithm for calculation of MSQ from quadrant, lat, and lon automatically produces a mismatch between the input quadrant and MSQ. To achieve better duplicate matching for Release 2.1 between data processed by NCDC and data processed by Boulder (e.g., TD-11 versus IMMT), the results of this algorithm were duplicated in calculation of B10. Thus for reports possessing an input quadrant/octant, the algorithm selects the B10 across the dateline from the input Q/O (e.g., at 0 lat, 180 lon, B10=303, 304, 339, 340 when quadrant is 1, 2, 3, 4, respectively). For reports at 180 lon lacking input Q/O, quadrant 2/4 was arbitrarily selected, resulting in actual B10 selection in quadrant 1/3 (see Figure 2). It is not known whether NCDC has followed this procedure for all past TD-11 data. ---------- 180W Western Hemisphere 0 Eastern Hemisphere 180E I------------------------------I----------------a-------------I 90N I NCDC quadrant=1 I NCDC quadrant=2 I I WMO quadrant=7 QMb* WMO quadrant=1 QcM* I QM I I I---d--------------------------e------------------------------f 0 lat I * NCDC quadrant=3 I* NCDC quadrant=4 I* I WMO quadrant=5 I WMO quadrant=3 I I I I I------------------------------I----------------g-------------I 90S Figure 2. Table 2 lists B10 assignment for reports at North and South Poles (a, g), 0 and 180 lon (b, c), and the equator (d, e, f). For original data lacking quadrant/octant information, this figure illustrates the direction of TD-11 quadrant (Q) and Marsden Square (M) assignment, or of (hypothetical, since it may never have been applied) B10 assignment (*) according to Release 1, p. G1.