=============================================================================== International Comprehensive Ocean-Atmosphere Data Set (I-COADS): Release 2.0 Translation Specifications: UK Marine Data Bank (Flatfile Data) 6 Sep 2002 =============================================================== {EDIT IN PROGRESS} Document Revision Information (previous version: 18 August 1998 ): Updates for Release 2.0 (1784-1997) and I-COADS. Authors: Jim Arnott and Mike Jackson (Hadley Centre, UK Met Office), and Scott Woodruff (NOAA/CDC) ------------------------------------------------------------------------------- {1. Introduction} These specifications document the translation performed at the UK Met Office of historical data from the UK Marine Data Bank (MDB) into the format for Long Marine Reports (LMR6). "Flatfile" formats 1, 1A, and 1B were originally prepared to transfer the MDB from an obsolete database management system (DBMS) to a Relational DBMS. The translation was made from these, plus from a new Flatfile format 1C for recently keyed UK historical data. Data in the UK Met.Office Marine Data Bank (MDB) are best-estimate values; quality control (QC) was carried out in two stages: a) Manual QC when obviously-erroneous values, including position, were corrected subjectively; original data were not retained (except where available in original cardimages). b) Automatic QC when suspect values were removed from the record but stored in a QC file to which a reference was stored with the remains of the record on the MDB. All of the Flatfile formats comprise three distinct Parts (Figure 1), with Part 1 always containing 156 4-byte integers, Part 2 containing either an additional set of 156 4-byte integers, or original cardimage and associated decoded data used when attempting to match cardimages and original Flatfiles, and Part 3 containing fields for ship number in one such integer, and call sign as eight ebcdic characters packed into two such integers. The contents and layout of each format further vary as follows: Flatfile 1: Data taken only from MDB. Cardimage not matched or not available (a card image should have been available, but none matched; or no cardimages were archived for this deck). Part 1 (the first 156-integer section) holds the data held in the MDB itself, and the second repeats those data with any rejected values retrieved from the QC file replaced. Flatfile 1A: Data from MDB with original cardimage data available. Part 1 holds MDB data as above. Part 2 has a "decode" section consisting of the results of decoding selected cardimage fields in the first 25 integers, followed by "scorecard" flags in the 26th. The decode section is followed by the 80-character card image, and 440 blanks. Flatfile 1B: Cardimage available but no corresponding Flatfile 1 record found from MDB. The selected fields from the cardimage that were decoded for comparison with Flatfile 1 records are stored in both Part 1 and in the Part 2 decode section (as for Flatfile 1A). The "scorecard" is left missing [NOTE: This is uncertain and needs to be verified]. The decode section is followed by the 80-character card image, and 440 blanks. Flatfile 1C: "Metform" data which have not previously been keyed and so are in a special 100-character "cardimage" format. These have never been loaded to the MDB. Data are stored as for 1B except that the cardimage is 100 characters long, followed in Part 2 by 420 blanks. Section 2 of this document contains the detailed field-by-field specifications from conversion from the Flatfile formats into LMR. General tables (applicable to more than one field) referred to as part of the field-by-field conversion specifications are located in section 5. Sections 3 and 4 of this document detail the fields used to construct the supplementary and error attachments. /----------------------------------------------------------------------------\ Flatfile 1: [ Part 1 ][ Part 2 ][ Part 3 ] <---156 x 4-bytes (32-bits)---><---156 x 4-bytes (32-bits)---><3 x 4-bytes> Flatfile 1A and 1B: [ Part 1 ][ Part 2 ][ Part 3 ] <---156 x 4-bytes (32-bits)---><-decode + card ( 80) + blank-><3 x 4-bytes> Flatfile 1C: [ Part 1 ][ Part 2 ][ Part 3 ] <---156 x 4-bytes (32-bits)---><-decode + card (100) + blank-><3 x 4-bytes> Decode section: The decode section consists of 26 fields, each of which is a 4-byte (32-bit) integer. Fields 1-15 are decoded cardimage values, fields 16-25 are unused, and field 26 is a matching "scorecard." For version 1B, the decoded cardimage values are set but the matching scorecard is unused. For version 1C the entire decode section is unused (details to be decided). Following is the structure of the decode section, and a description of the internal structure of the matching scorecard: 1 Logbook (or Folio Number) 2 Year 3 Month 4 Day 5 Hour 6 Latitude 7 Longitude 8 Wind Direction 9 Wind Speed 10 Pressure 11 Air Temperature 12 Sea Temperature 13 Total Cloud amount 14 Sea/Wind wave direction 15 Swell wave direction 16 to 25 Not used 26 Matching scorecard: For flatfile version 1A, the "matching scorecard" will have a bit set to one for each of the above which do NOT match in the same order as above but starting from the rightmost bit of the word. For fields that match the applicable bit will be zero, plus the highest-order 17 bits will always be zero. Cardimage sections: The format within each 80- or 100-character cardimage is determined by deck. [NOTE: The cardimage documentation is available within original Met Office "Series Manuals," which have have been converted into digital form and are available separately.] \----------------------------------------------------------------------------/ Figure 1. Layouts of the Flatfile 1, 1A and 1B, and 1C formats, and of their decode and cardimage sections. {2. Specifications} Flatfile 1 Part 1, Part 2, or Part 3 integers are referred to as P1(nn) P2(nn), P3(nn) where nn is the integer number. 1) B10 10-degree box Flatfile1 : input = latitude, P1(5); and longitude, P1(6) (a) Change latitude and longitude from integer tenths of degrees to real degrees to tenths. (b) B10 is calculated by {wrlmr6}. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 2) YR year Flatfile1 : input = year, P1(2) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 3) MO month Flatfile1 : input = month, P1(1) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 4) DY day Flatfile1 : input = day, P1(3) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 5) HR hour Flatfile1 : input = hour, P1(4) (a) Change hour from integer whole hour to real hour to hundredths. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 NOTE that hours of 24 are known to exist (and will be written out to the error attachment) but it is not known for sure whether they belong at the beginning or end of the given date (such hours have been cleaned up in the Flatfile1/1A data). Flatfile1C: same as Flatfile 1B. 6) TI time indicator Flatfile1 : input = no corresponding field (a) TI = 0 (Always to the nearest whole hour.) Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 7) LON longitude Flatfile1 : input = longitude, P1(6) (a) Change longitude from integer tenths of degrees to real degrees to tenths East of Greenwich (350.0 = 10 degrees West). Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 8) LAT latitude Flatfile1 : input = latitude, P1(5) (a) Change latitude from integer tenths of degrees to real degrees to tenths. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 9) LI lat/lon indicator Flatfile1 : input = no corresponding field (a) LI = 0 (degrees and tenths) Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 10) DCK deck Flatfile1 : input = UK series number TDF deck number or country of origin, P1(47) (a) If the input is within the range 0-100 and it forms a valid country code (C1) number (valid C1 values defined in ) or is "100," then DCK=254. (b) Or if the input forms a valid TDF-11 deck number listed in Table 1, direct mapping from integer to real. (c) Or if the input forms a valid UKMO series number listed in Table 2, direct mapping from integer to real. (d) Otherwise, DCK=255. This includes input values 41-99, or, for values outside the range 0-100, if the input value does not appear in Tables 1-2. Flatfile1A: same as Flatfile 1. Flatfile1B: Flatfile 1 (c) applies. Flatfile1C: Flatfile 1 (c) applies. 11) SID source ID Flatfile1 : input = no corresponding field (determined from run information) (a) Set SID = 90 Flatfile1A: input = no corresponding field (determined from run information) (a) Set SID = 91 Flatfile1B: input = no corresponding field (determined from run information) (a) Set SID = 92 Flatfile1C: input = no corresponding field (determined from run information) (a) Set SID = 93 12) PT platform type Flatfile1 : input = off-station OWS flag, P1(141) (a) Determined by DCK if a PT value is listed in Table 1 or 2. (b) If Table 2 indicates "2/3," then PT is set to 3 unless the input off-station OWS flag is also set, in which case PT=2. This occurs as follows: (i) The main Flatfiles have no on-station OWS data, but they do contain off-station OWS data with the input flag set. In this case, PT set to 2, but only if the deck is one listed "2/3" in Table 2. (ii) On-station OWS data are in separate Flatfiles, which are handled by setting PT=3 from run information. (c) Or, default platform type from run information (generally PT=5). Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 13) QI quality indicator Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 14) DS dup status Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 15) DC dup check Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 16) TC track check Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 17) PB pressure bias Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 18) DI wind direction indicator Flatfile1 : input = flags in P1(93), P1(94) and P1(95) (a) If wind direction is missing, DI is also set to missing. (b) Or, the above flags map to DI=1, 3 and 4, respectively. (c) Otherwise, DI=0 if none of the flags is set to true. NOTE: The settings of the flags are expected to be, but no test is made to ensure that they are, mutually exclusive. In the event that they are not, the first one set in the above order becomes the setting. Flatfile1A: same as Flatfile 1. Flatfile1B: D1=missing. Flatfile1C: [NOTE: to be determined.] 19) D wind direction Flatfile1 : input = direction, P1(7) and windspeed, P1(8) (a) If P1(7) = 99, D=362 (code for "variable") (b) Or, if P1(8) = 0, D=361 (code for calm) (c) Otherwise, direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 20) WI wind speed indicator Flatfile1 : input = flags in P1(96), m/s and P1(97), measured. (a) If wind speed is missing, WI is also set to missing. (b) Or as specified in Table 3 for each deck; truth tables attached to Table 3 specify the handling for decks for which there is a dependency on the input flags. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 21) W wind speed Flatfile1 : input = windspeed, P1(8) as knots. (a) Original units m/s, back convert using factor of 0.5147. (b) Original units knots, convert using {lmrlib} routine {fxktms}. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 22) VI visibility indicator Flatfile1 : input = visibility-measured flag, P1(143) (a) VI is missing if visibility is missing. (b) Or, VI=1 if the input flag is set "true". (c) Otherwise, VI=missing. Flatfile1A: input = same as Flatfile 1. Flatfile1B: input = same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 23) VV visibility Flatfile1 : input = visibility, P1(17) (a) Visibility is transformed from decameters to VV code 90-99 according to the following mapping: < 50m VV=90 50-200m VV=91 200-500m VV=92 500-1km VV=93 1-2km VV=94 2-4km VV=95 4-10km VV=96 10-20km VV=97 20-50km VV=98 >50km VV=99 Flatfile1A: same as Flatfile 1. Flatfile1B: input = same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 24) WW present weather Flatfile1 : input = present weather, P1(9) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 25) W1 past weather Flatfile1 : input = first past weather, P1(10) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 26) W2 second past weather Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: [NOTE: to be determined.] 27) SLP sea level pressure Flatfile1 : input = sea level pressure, P1(23). (a) Convert from integer millibars and tenths to real millibars to tenths. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 28) T1 temperature indicator Flatfile1 : input = Temperature reading accuracy and scale flags: P1(99) P1(100), P1(101), P1(102), P1(105), P1(106), P1(107). (a) The input flags are mapped as follows: Input P1 flags Resultant 99 100 101 102 105 106 107 T1 F F F F F F F 0 F F T F F T F 1 T T F F T F F 2 F F F T F F T 4 F F T T F T T 5 T T F T T F T 6 Other combinations not listed above: missing Flatfile1A: same as Flatfile 1 Flatfile1B: T1=missing. Flatfile1C: [NOTE: to be determined.] 29) AT air temperature Flatfile1 : input = dry bulb temperature, P1(18) (a) Convert from integer degrees and tenths to real degrees to tenths. Flatfile1A: same as Flatfile 1 Flatfile1B: same as Flatfile 1 Flatfile1C: same as Flatfile 1 30) WBT wet bulb temperature Flatfile1 : input = wet bulb temperature, P1(19) (a) Convert from integer degrees and tenths to real degrees to tenths. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 31) DPT dew point temperature Flatfile1 : input = dew point, P1(20) (a) Convert from integer degrees and tenths to real degrees to tenths. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 32) SST sea surface temperature Flatfile1 : input = sea surface temperature, P1(24). (a) Convert from integer degrees and tenths to real degrees to tenths. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 33) SI sea-surface-temperature method indicator Flatfile1 : input = SST "not by bucket" flag, P1(104). (a) If set, SI=9. (b) Otherwise, SI=0 (bucket method). Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 34) N total cloud amount Flatfile1 : input = total amount of cloud, P1(11) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 35) NH lower cloud amount Flatfile1 : input = amount of low cloud, P1(15) (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 36) CL low cloud type Flatfile1 : input = type of low cloud, P1(12) (a) Direct mapping from integer to real. NOTE: A value corresponding to CL=10 is not available in MDB. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 37) HI cloud height indicator Flatfile1 : input = "cloud height measured" flag, P1(144) (a) HI is missing if cloud height is missing. (b) Or, HI=1 if the input flag is set "true". (c) Otherwise, HI=missing. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 38) H cloud height Flatfile1 : input = height of low cloud, P1(16) (a) < 50m H=0 (b) 50- 99m H=1 (c) 100- 199m H=2 (d) 200- 299m H=3 (e) 300- 599m H=4 (f) 600- 999m H=5 (g) 1000-1499m H=6 (h) 1500-1999m H=7 (i) 2000-2499m H=8 (j) >=2500m H=9 Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 39) CM type of medium cloud, P1(13) Flatfile1 : input = type of medium cloud, P1(13) (a) Direct mapping from integer to real. NOTE: A value corresponding to CL=10 is not available in MDB. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 40) CH type of high cloud, P1(14) Flatfile1 : input = type of high cloud, P1(14) (a) Direct mapping from integer to real. NOTE: A value corresponding to CL=10 is not available in MDB. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 41) WD wave direction Flatfile1 : input = wave direction, P1(38) (a) An input value of 128 is code for a Wind Wave (i.e., wave direction was not separately observed but can be taken from wind direction). By agreement, set WD=missing. (b) An input value of 99 (direction indeterminate) converts to code of 38. (c) Otherwise, direct mapping from integer to real. Flatfile1A: same as Flatfile 1 Flatfile1B: same as Flatfile 1 Flatfile1C: same as Flatfile 1 42) WP wave period Flatfile1 : input = wave period, P1(39) (a) Map half-seconds to whole seconds, round to odd. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 43) WH wave height Flatfile1 : input = wave height, P1(40) (a) Convert integer half-metres to whole metres. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 44) SD swell direction Flatfile1 : input = first swell direction, P1(41) (a) An input value of 99 (direction indeterminate) converts to code of 38. (b) Otherwise, direct mapping from integer to real. Flatfile1A: same as Flatfile 1 Flatfile1B: same as Flatfile 1 Flatfile1C: same as Flatfile 1 45) SP swell period Flatfile1 : input = first swell period, P1(42) (a) Map half-seconds to whole seconds, round to odd. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 46) SH swell height Flatfile1 : input = first swell height, P1(43) (a) Convert integer half metres to whole metres. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 47) C1 country code Flatfile1 : input = UK series number TDF deck number or country of origin, P1(47). NOTE: See DCK for related processing. MDB documentation reserves P1(47) values 0-100 for "country of origin," with 41-99 "not yet assigned," and 100 defined as unknown. (a) If the input is within the range 0-40 it forms a valid country code (C1) number (valid C1 values defined in ), and thus a direct mapping is made from integer to real (C1 is left missing if the input is 041-100). (b) Or if the input forms a valid TDF-11 deck number listed in Table 1, C1=missing. (c) Or if the input forms a valid UKMO series number listed in Table 2, copy from C1 column of the table. (d) Otherwise, C1=missing and P1(47) is placed in the error attachment. Thus P1(47) is placed in the error attachment if the input is 41-100, or, for values outside the range 0-100, if the input value does not appear in Tables 1-2. NOTE: For version 1.0 of the program, P1(47) was also placed in the supplemental attachment (positions 134-136) when it was greater than 100 and not in Tables 1-2; for later versions, positions 134-136 are left blank since the error attachment holds P1(47) when it is an unrecognized value. Flatfile1A: same as Flatfile 1 (Information may be available in original cardimage.) Flatfile1B: (c) applies. Flatfile1C: (c) applies. 48) C2 second country code Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 49) SC ship course Flatfile1 : input = ship direction, P1 miscellaneous group, first element. (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 50) SS ship speed Flatfile1 : input = ship speed, P1 miscellaneous group, second element. (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 51) A barometric tendency Flatfile1 : input = barometric tendency, P1 miscellaneous group, third element. (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing. (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 52) PPP amount of SLP change Flatfile1 : input = amount of SLP change, P1 miscellaneous group, fourth element. (a) Direct mapping from integer to real. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 53) IS ice accretion Flatfile1 : input = ice accretion, P1 miscellaneous group, second element. (a) Direct mapping from integer to real Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 54) ES ice thickness Flatfile1 : input = ice thickness, P1 miscellaneous group, fourth element. (a) Direct mapping from integer to real Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 55) RS ice accretion rate Flatfile1 : input = ice accretion rate, P1 miscellaneous group, third element. (a) Direct mapping from integer to real Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 56) II ID indicator Flatfile1 : input = call sign or ship number, P3(1) or P3(2) and P3(3). (a) If there is a ship callsign, with or without a ship number, then II=missing. (b) Or, if there is only a generic callsign, then II=2. (c) Or, if there is a ship number, with or without a generic callsign, then II=9. (d) Otherwise, II=missing. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: [NOTE: to be determined.] 57-64) ID(8) ID/call sign Flatfile1 : input = call sign or ship number, P3(1) or P3(2) and P3(3). (a) If there is information in the ship callsign field, with or without a ship number, then ID=callsign. (b) Or, if there is only a generic callsign then ID=generic callsign. (c) Or, if there is a ship number, with or without a generic callsign, then ID=ship number as ASCII numerals. (d) Otherwise, ID=missing. NOTE: The up-to-eight non-blank call sign characters, or the five ASCII ship number numerals, are placed into CTRUE. Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1. Flatfile1C: same as Flatfile 1. 65) OS observation source Flatfile1 : input = no corresponding field, determined from whether country-of-origin practice is known. (a) OS is set to missing, if so indicated by Table 3. (b) Or, OS is set to 1 for UK decks indicated by Table 3. (c) Otherwise, OS is set to missing. Flatfile1A: same as Flatfile 1 Flatfile1B: same as Flatfile 1 Flatfile1C: same as Flatfile 1 66) OP observation platform Flatfile1 : input = auxiliary-ship flag, P1(146). (a) OP is set to missing, if so indicated by Table 3. (b) Or, if country-of-origin (C1) practices are known, OP is set according to Table 4 utilizing P1(146). (c) Otherwise, OP is set to missing. Flatfile1A: same as Flatfile 1 Flatfile1B: same as Flatfile 1 Flatfile1C: same as Flatfile 1 67) T2 second temperature indicator Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 68) IX station/weather indicator Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 69) WX wave period indicator Flatfile1 : input = no corresponding field but period known to be converted from code in every case. (a) Wave present, WX=1 (b) No wave present, WX=missing Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) 70) SX swell period indicator Flatfile1 : input = no corresponding field but period known to be converted from code in every case. (a) Swell present, SX=1 (b) No swell present, SX=missing Flatfile1A: same as Flatfile 1. Flatfile1B: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) Flatfile1C: same as Flatfile 1 but input is always missing (Information may be available in original cardimage.) 71) IRD IMM receipt date Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field 72) A6 allowance 6 flag Flatfile1 : input = no corresponding field Flatfile1A: input = no corresponding field Flatfile1B: input = no corresponding field Flatfile1C: input = no corresponding field {3. Contents of supplemental attachment (Attm4)} Flatfile 1 : bytes description 1-7 indicators and flags: Character 1 - Flatfile elements 93 to 100 packed Character 2 - Flatfile elements 101 to 108 packed Character 3 - Flatfile elements 109 to 116 packed Character 4 - Flatfile elements 117 to 124 packed Character 5 - Flatfile elements 125 to 132 packed Character 6 - Flatfile elements 133 to 140 packed Character 7 - Flatfile elements 141 to 148 packed Note: Characters 1-7 were derived from 32-bit elements 93-146 simply by testing for a "truth" value of 1. 8-12 ship number, P3(1) (only if non-missing, converted to ascii) 13-112 blank (in all other versions, holds cardimage). 113-117 sea ice, P1(52-56), stored as five ascii characters. The codes used for these fields changed starting in 1970 *(the flatfile description says "Pre 1970" thus we assume that 1970 data are in the new code). For data starting in 1970 refer to FM13 in WMO-No.306 (tables 0639, 3739, 0439, 0739 and 5239); for earlier code, the five elements (extracted from Met.O.509) are: 113. Kind of Ice. 0 No ice. 5 Shore lead. 1 New ice. 6 Heavy fast ice. 2 Fast ice. 7 Heavy pack ice/drift ice. 3 Pack ice / drift ice. 8 Hummocked ice. 4 Packed (compact) slush or sludge. 9 Icebergs. 114. Effect of ice on navigation. 0 Navigation unobstructed. 1 Navigation unobstructed for steamers, difficult for sail. 2 Navigation difficult for low-powered steamers, closed to sail. 3 Navigation possible only for powerful steamers. 4 Navigation possible only for steamers constructed to withstand ice pressure. 5 Navigation possible with the assistance of ice-breakers. 6 Channel open in solid ice. 7 Navigation temporarily closed. 8 Navigation closed. 9 Navigation conditions unknown (eg owing to bad weather). 115. Bearing of ice edge. 0 No ice edge can be stated. 5 Ice edge towards SW. 1 Ice edge towards NE. 6 Ice edge towards W. 2 Ice edge towards E. 7 Ice edge towards NW. 3 Ice edge towards SE. 8 Ice edge towards N. 4 Ice edge towards S. 9 Ice in several directions. 116. Distance to ice edge from reporting ship. 0 Up to 1 mile 5 8-12 miles 1 1-2 miles 6 12-16 miles 2 2-4 miles 7 16-20 miles 3 4-6 miles 8 More than 20 miles 4 6-8 miles 9 Unspecified or no observation 117. Orientation of ice edge. 0 Orientation impossible to estimate - ship OUTSIDE the ice. 1 Ice edge lying in a direction NE to SW with ice situated to NW. 2 Ice edge lying in a direction E to W with ice situated to N. 3 Ice edge lying in a direction SE to NW with ice situated to NE. 4 Ice edge lying in a direction S to N with ice situated to E. 5 Ice edge lying in a direction SW to NE with ice situated to SE. 6 Ice edge lying in a direction W to E with ice situated to S. 7 Ice edge lying in a direction NW to SE with ice situated to SW. 8 Ice edge lying in a direction N to S with ice situated to W. 9 Orientation impossible to estimate - ship INSIDE the ice. 118-133 significant cloud groups ("8" groups), P1(25 to 36), information for each of up to 4 groups stored as 4 ascii characters (NChh); refer to FM13 in WMO-No.306 for codes of cloud amount, genus and height for each layer (tables 2700, 0500 and 1677 respectively). 134-136 invalid TDF11 Deck or UKMO Series number, stored as three ascii characters, if one found in P1(47). This determination is made by comparing the Deck or Series number to those defined in Tables 1-2. 137-140 third wave report, P1(44-46); wave direction as two hexadecimal characters, period and height each as one hexadecimal character. Refer to elements 44-46 above for description. Flatfile 1A: bytes description 1-7 indicators and flags as version 1. 8-12 ship number, P3(1) (null or, if non-missing, converted to ascii). 13-92 cardimage, converted to ascii. 93-112 not used (set to blank). 113-140 as version 1. Flatfile 1B: bytes description 1-7 not used (set to blank). 8-12 ship number, P3(1) (null or, if non-missing, converted to ascii). 13-92 cardimage, converted to ascii. 93-140 not used (set to blank). Flatfile 1C: bytes description 1-7 not used (set to blank). 8-12 ship number, P3(1) (null or, if non-missing, converted to ascii). 13-112 cardimage, converted to ascii. 113-140 not used (set to blank). {4. Contents of error attachment (Attm5)} Flatfile 1 : A 32-bit integer from Part 2, corresponding to the Part 1 input field (if any) used to construct each LMR field in the above specifications, is copied to the array (CTRUE) used to fill the error attachment. If the Part 2 field contains the Flatfile missing value (maxneg) or there were no corresponding Flatfile input data, the length of the CTRUE data as specified in LTRUE is set to zero; otherwise LTRUE is set to 4 (for 4 bytes). This approach was chosen because if Part 2 data are extant, while Part 1 data are missing, this indicates that the Part 1 value was rejected as part of MDB quality control, and we therefore desire to include the Part 2 value in the error attachment. In this case the FTRUE element ordinarily used to construct the LMR field will be missing, but CTRUE/LTRUE extant. Data may also be written out to the error attachment in the event that {wrlmr6} detects a range problem in an extant FTRUE value, in which case again the Part 2 value stored in CTRUE/LTRUE will be written out to the error attachment. Note that this could lead to unexpected results if for some reason the Part 1 and 2 values were not identical, or the Part 1 value was extant but the Part 2 value was missing, but we do not expect any occurrences of these patterns which would signal unknown problems in MDB. Flatfile 1A: Each 32-bit integer used from part 1 to load a value into the array (FTRUE1) used to complete the LMR main recort is copied to the corresponding location in array CTRUE and LTRUE set as above. This may be written out to the error attachment in the event that {wrlmr6} detects a range problem in the FTRUE value. Flatfile 1B: same as Flatfile 1A Flatfile 1C: same as Flatfile 1A {5. General tables} Table 1. Mapping of series (Ser) numbers 000-200 to deck (DCK), source ID (SID), platform type (PT) and country code (C1) ("--" indicates missing). The series range 000-100 has been designated by the Met Office to cover data probably received via WMO Resolution 35, but whose exact identity has been lost (country code may be reflected in series number, as noted). The range 101-200 has been designated to include the TDF-11 data received decades ago from NOAA. For decks 110-197: Periods of record (and description information listed in parentheses) are from Minhinick and Folland (1984); except periods of record for decks 128 and 195 are from Slutz et al. (1985), and that for deck 129 is from UK documentation for the MDB (August 1987). ------------------------------------------------------------------------------- Ser DCK SID PT C1 Description and approximate period of record =============================================================================== 000- 100 254 90 5 * WMO (IMMPC or IMMT) format; foreign or unknown origin 110 110 90 5 -- US Navy Marine (US WBAN11) 1945-1951 116 116 90 5 -- US Merchant Marine 1949-1960 117 117 90 5 -- US Navy Hourlies (US WBAN) 1952-1964 118 118 90 5 -- Japanese Ships No. 1 1933-1953 119 119 90 5 -- Japanese Ships No. 2 1953-1961 128 128 90 5 -- International Marine (1963- ) 129 129 90 5 -- Retard for US area (1961-1972, rec'd mid '86) 181 281 90 5 -- US Navy MAR** 1920-1945 185 185 90 5 -- USSR Marine IGY 1957-1958 187 187 90 5 -- Japanese Whaling Fleet 1946-1956 188 188 90 5 -- Norwegian Whaling Fleet 1932-1939 189 189 90 5 -- Netherlands Marine 1939-1955 193 193 90 5 -- Netherlands Marine 1854-1938 195 195 90 5 -- US Navy Ships Logs (1941-1946) 196 196 90 5 -- Deutsche Seewarte Marine 1949-1954 197 197 90 5 -- Danish Marine (Polar) 1860-1956 other 255 90 5 -- Undocumented decks/series (presumed labelling errors) ------------------------------------------------------------------------------- * For series 000-040 within the range 000-100, the series number represents country code 0-40 derived from from IMMPC or IMMT data. In these cases series 000-040 is mapped into C1. Otherwise within the 000-100 series, C1 is missing. ** US Navy Monthly Aerological Record (MAR); i.e., deck 281 within COADS. ----------- Table 2. Mapping of series (Ser) numbers 201-239 to deck (DCK), source ID (SID), platform type (PT) and country code (C1) ("--" indicates missing). For series for which original card images are available, "90-2" in the SID col umnindicates that SID can be either 90 (a card image was not found; Flatfile 1 format), 91 (a card image was found; Flatfile 1a format), 92 (a card image was found, but matching Flatfile 1 data were not found and instead were partially reconstructed; Flatfile 1b format). For series for which original card images were not archived, SID=90. For PT, "2/3" indicates that PT is set to 3 unless the off-station OWS flag, P1(141), is set. Decks falling outside those listed specifically in this table are mapped into deck 255 as indicated in Table 1. ------------------------------------------------------------------------------- Ser Doc* DCK SID PT C1 Description and approximate period of record =============================================================================== 201 a 201 90-2 5 3 All ships (1930 code) 1850-1920 202 a 202 90-2 5 3 All ships (1921 code) 1921-1929 203 a 203 90-2 5 3 Sel.ships (1930 code) 1930-1948 204 a 204 90-2 5 3 HM ships (1930 code) 1930-1948 205 a 205 90-2 5 3 Scots Fishery Cruisers MARIDS (1930 code) 206 a 206 90-2 2/3 3 OWS (1930 code) 1947-1949 207 a 207 90-2 5 3 Sel.ships (1930 code) 1945-1948 208 b 208 90** 4 -- Light Vessels 1949-1956 209 b 209 90-2 5 -- Sel.ships (inc.foreign) 1953-1956 210 b 210 90-2 2/3 -- OWS (inc.Dutch"J") 1950-1954 211 c 211 90-2 5 -- Scots Fishery Cruisers 1956-1961 212 c 212 90** 4 -- Light Vessels 1956-1961 213 c 213 90-2 5 -- "Selected" ships 1956-1961 214 c 214 90-2 2/3 -- OWS 1956-1961 215 d 215 90-2 5 --# German Marine 1860-1938 216 g 216 93 5 3 METFORMS 1935-39? 218 - 218 90 2/3 2 US OWS 1953- 221 e 221 90-2 5 --# MARIDS and Trawlers 1961- 222 e 222 90** 4 --# Light Vessels 1961- 223 e 223 90** 5 --# "Selected" Ships 1961-1981 224 e 224 90** 2/3 --# OWS 1961-1981 225 - 225 90 5 -- Norwegian Format 1953- 226 - 226 90 2/3 3 OWS (1949 code) 1949-1952 227 - 227 90 5 3 "Selected" Ships 1949-1953 229 - 229 90 5 3 HM Ships 1961- 230 - 230 90 5 -- IMMPC format 1960-1981 233 - 233 90 5 3 "Selected" Ships 1982- 234 - 234 90 2/3 3 OWS 1982- 235 - 235 90 15 3 RIGS / PLAT / AWS 1982- 239 - 239 90 5 3 HM Ships 1982- (decks 254-255 are also used; ref. Table 1) ------------------------------------------------------------------------------- * Documentation for the original card images is available: a) Series 1-7: Surface Marine Card Form 789; 1854-1953 b) Series 8-10: Surface Marine Card Form 1915; 1953-1956 c) Series 11-14: Surface Marine Card Form 1971; 1956-1961 d) Series 15: Card format for German data ; 1860-1939 e) Series 21-24: Surface Marine Card Form 6407; 1962-1981 g) Documentation is available for the newly keyed METFORMS. ** Documentation for the original card images is available, but the card images themselves were not archived (lost or discarded). # A country code may be recoverable from the cardimage (if available in the supplemental attachment; extraction was not attempted during this conversion). ----------- Table 3. Simple mapping ignoring MDB fields (resultant indicator numeric value, or missing), or more complex handling involving interpretation of MDB fields (lower-case letters or Table 4), by deck of WI and OS/OP. ------------------------------------------------------------------------------- DCK WI* SI** OS OP Description & approx. period of record =============================================================================== 254 t missing missing Table 4 WMO format; foreign or unknown origin 255 t missing missing issing Undocumented TDF-11 decks 110 t missing missing missing US Navy Marine 1945-1951 116 t missing missing missing US Merchant Marine 1949-1960 117 t missing missing missing US Navy Hourlies 1952-1964 118 t missing missing missing Japanese Ships No. 1 1933-1953 119 t missing missing missing Japanese Ships No. 2 1953-1961 128 t missing missing missing International Marine (1963- ) 129 t missing missing missing Retard for US area (1961-1972) 181 t missing missing missing US Navy MAR 1920-1945 185 t missing missing missing USSR Marine IGY 1957-1958 187 t missing missing missing Japanese Whaling Fleet 1946-1956 188 t missing missing missing Norwegian Whaling Fleet 1932-1939 189 t missing missing missing Netherlands Marine 1939-1955 193 t missing missing missing Netherlands Marine 1854-1938 195 t missing missing missing US Navy Ships Logs (1941-1946) 196 t missing missing missing Deutsche Seewarte Marine 1949-1954 197 t missing missing missing Danish Marine 1860-1956 ------------------------------------------------------------------------------- 201 5 u missing missing All ships (1930 code) 1850-1920 202 5 u missing missing All ships (1921 code) 1921-1929 203 5 u missing missing Sel.ships (1930 code) 1930-1948 204 5 u missing missing HM ships (1930 code) 1930-1948 205 5 u missing missing Scots Fishery Cruisers MARIDS 206 5 u missing missing OWS (1930 code) 1947-1949 207 5 u missing missing Sel.ships (1930 code) 1945-1948 208 5 u missing missing Light Vessels 1949-1956 209 5 u missing missing Sel.ships (inc.foreign) 1953-1956 210 5 u missing missing OWS (inc.Dutch"J") 1950-1954 211 3 u missing missing Scots Fishery Cruisers 1956-1961 212 3 u missing missing Light Vessels 1956-1961 213 3 u missing missing "Selected" ships 1956-1961 214 3 u missing missing OWS 1956-1961 215 5 missing missing missing German Marine 1860-1938 216 5 missing missing missing METFORMS 1935-39? 218 u missing missing missing US OWS 1953- 221 3 missing missing missing MARIDS and Trawlers 1961- 222 3 u missing missing Light Vessels 1961- 223 u u 1 Table 4 "Selected" Ships 1961-1981 224 3 u missing missing OWS 1961-1981 225 u missing missing missing Norwegian Format 1953- 226 u u missing missing OWS (1949 code) 1949-1952 227 u u 1 Table 4 "Selected" Ships 1949-1953 229 u u 1 Table 4 HM Ships 1961- 230 u u 1 Table 4 IMMPC format 1960-1981 233 u u 1 Table 4 "Selected" Ships 1982- 234 u u 1 Table 4 OWS 1982- 235 u missing missing missing RIGS / PLAT / AWS 1982- 239 u u 1 Table 4 HM Ships 1982- ------------------------------------------------------------------------------- * Special handling for WI: "t" = TDF11 handling: Indicator resulting measured m/s WI -------- --------- --------- yes yes missing yes not set 7 not set yes missing not set not set 6 "u" = UK default handling: Indicator resulting measured m/s WI -------- --------- --------- yes yes 1 yes not set 4 not set yes 0 not set not set 3 ** Special handling for SI: "u" = UK default handling: Indicator resulting not observed by bucket SI ---------------------- --------- yes 9 not set 0 ---------- Table 4. List of Countries whose "source" practices are known, and resulting OP settings for likely WMO Resolution 35 data. ------------------------------------------------------------------------------- Country (C1) Logbook GTS OP OP P1(146) true P1(146) false =============================================================================== India (07) T F 3 1 Hong Kong (08) T F 3 1 Japan (17) T F 3 1 Germany (21) T F 3 1 UK (03) T F 3 1 ------------------------------------------------------------------------------- {References} Refer to for the structure of the COADS LMR6 format. Minhinick, J.H., and Folland, C.K., 1984: The Meteorological Office Historical Sea Surface Temperature Dataset (MOHSST). MET-O 13 Branch Memorandum No. 137. Slutz, R.J., S.J. Lubker, J.D. Hiscox, S.D. Woodruff, R.L. Jenne, D.H. Joseph, P.M. Steurer, and J.D. Elms, 1985: Comprehensive Ocean-Atmosphere Data Data Set; Release 1. NOAA Environmental Research Laboratories, Climate Research Program, Boulder, Colo., 268 pp. (NTIS PB86-105723). {Appendices} Appendix 1: Structure of Flatfile 1. Appendix 2: Converted cardimage data and flags (Flatfile 1A, 1B and 1C) [NOTE: Appendices currently are available separately.]