0. Introduction
This is a collection of background details for the conversion to LMR (supp. F) from various character-based formats, and for the conversion back into TD-1129(M). The common characteristics and incompatibilities of the various formats in TD-11 and the Exchange format are discussed in sec. 1, as background for the design of LMR. In sec. 2, extensive corrections that were made to correct known data problems and other details of the conversion into LMR are given. Finally, sec. 3 defines changes made in the conversion from LMR back into TD-1129(M).
1. Long Marine Report Background
Brief format layouts give the primary fields in TD-11 (Tape Deck-11); more specific information on TD-11, including the representation of data within fields, will be found in [5], [6], and [7]. Different versions of the Exchange format are described in more detail since documentation of these was not readily available.
1.1 TD-11
TD-11 formats can be grouped into three classes:
1) TD-1100These trace sequentially the evolution of the format through time in response to changing observational methods, often as a result of differing WMO code conventions, and a desire (perhaps unrealistic) to arrive at a single format suitable for all such data. The following discussions of each class include lists of data sets processed from each class.
2) TD-1127
3) TD-1129(M)
TD-1100
Data sets processed: Atlas, HSST Pacific, Old TDF-11 Supplement B-C,
Monterey Telecom., OSV, MSQ 486 and 105 Omissions, NODC Surface, OSV Z.
The original TDF-11 (Tape Data Family-11) now called TD-1100, comprises at least 18 distinct sub-formats, linked by tape deck number "11xx" to a source card deck "1xx." Exceptions are that tape deck 1181 describes card deck 281, and later additions such as card decks 555 and 891 cannot fit this pattern either. Report length is 140 characters, but a variable number of characters at the end could be blank depending on sub-format. The form of any member of TD-1100 can be expressed by
location = 1-26(26) + regular1 = 27-77(51) + irregular1 = 78-140(63).This notation shows the start position i, end position j, and length k of sections as given by "section = i-j(k)", which when concatenated ("+") depict the whole format. The format layout is given by Table I1-1.
Field Char number position Description ---------------------------------------------------------------- 1 1-3 card deck 2 4-6 10° Marsden Square 3 7-8 1° Marsden sub-square 4 9 quadrant (1-4) 5 10-12 latitude (degrees N, S) 6 13-16 longitude (degrees E, W) 7 17-20 year 8 21-22 month 9 23-24 day 10 25-26 hour (GMT) 11 27-29 wind direction and indicator (code) 12 30-33 wind speed and indicator (knots) 13 34-36 visibility and indicator (code) 14 37-38 present weather (code) 15 39 past weather (code) 16 40-44 sea level pressure (mb) 17 45-48 temperature indicator and air temperature (°C) 18 49-51 wet bulb temperature (°C) 19 52-54 dew point temperature (°C) 20 55-57 sea surface temperature (°C) 21 58-60 air-sea temperature difference (°C) 22 61 total cloud amount (oktas) 22 62 lower cloud amount 22 63 type of low cloud 22 64 cloud height indicator 22 65 cloud height 22 66 type of middle cloud 22 67 type of high cloud 23 68-69 direction of waves (code) 24 70 period of waves (code) 25 71-72 height of waves (1/2 meters) 26 73-74 direction of swell (code) 27 75 period of swell (code) 28 76-77 height of swell (1/2 meters) 29 78-79 ocean weather station number or country code 30 80 card indicator 31 81 ship type 32 82 additional data indicator 33-36 83-88 additional data 37 89 ice indicator 38 90-93 ship number 39 94-140 supplemental data ----------------------------------------------------------------
standard1 = 78-81(4) + additional = 82-88(7) + standard2 = 89-93(5) + supplemental = 94-140(47).The supplemental section is used to preserve the original units or form of fields whose conversion might be open to question, or which are unique to a sub-format. Table I1-2 shows the (supposedly) non-blank length of supplemental for each of 17 sub-formats.
Tape deck Card deck Supplemental = 94- Trailing blanks ------------------------------------------------------------ 1110 110 140(47) 0 1116 116 120(27) 20 1118 118 118(25) 22 1119 119 112(19) 28 1128 128 101(8) 39 1181 281 134(41) 6 1184 184 112(19) 28 1185 185 100(7) 40 1187 187 119(26) 21 1188 188 97(4) 43 1189 189 116(23) 24 1192 192 136(43) 4 1193 193 116(23) 24 1194 194 120(27) 20 1195 195 113(20) 27 1196 196 126(33) 14 1197 197 125(32) 15 ------------------------------------------------------------
Tape Deck-1127 has the general form
location = 1-26(26) + regular1 = 27-77(51) + regular2 = 78-140(63),where regular2 takes the place of irregular1 in TD-1100. Quality flags have been added and the format of regular2 is invariant, regardless of deck number. Also, the call sign is usually used in place of ship number. Table I1-3 gives the format layout.
Field Char number position Description ---------------------------------------------------------------- 1 1-3 card deck 2 4-6 10° Marsden Square 3 7-8 1° Marsden sub-square 4 9 quadrant (1-4) 5 10-12 latitude (degrees N, S) 6 13-16 longitude (degrees E, W) 7 17-20 year 8 21-22 month 9 23-24 day 10 25-26 hour (GMT) 11 27-29 wind direction and indicator (code) 12 30-33 wind speed and indicator (knots) 13 34-36 visibility and indicator (code) 14 37-38 present weather (code) 15 39 past weather (code) 16 40-44 sea level pressure (mb) 17 45-48 temperature indicator and air temperature (°C) 18 49-51 wet bulb temperature (°C) 19 52-54 dew point temperature (°C) 20 55-57 sea surface temperature (°C) 21 58-60 air-sea temperature difference (°C) 22 61 total cloud amount (oktas) 22 62 lower cloud amount 22 63 type of low cloud 22 64 cloud height indicator 22 65 cloud height 22 66 type of middle cloud 22 67 type of high cloud 23 68-69 direction of waves (code) 24 70 period of waves (code) 25 71-72 height of waves (1/2 meters) 26 73-74 direction of swell (code) 27 75 period of swell (code) 28 76-77 height of swell (1/2 meters) 29 78-79 country code 30 80 ship direction (code) 31 81 ship speed (code) 32 82 barometric tendency (code) 33 83-85 amount of pressure change (mb) 34 86 type of ice accretion on ship (code) 35 87-88 thickness of ice on ship (cm) 36 89 rate of ice accretion (code) 37 90-96 ship, OSV, or buoy call sign 38 97 original wind speed units indicator 39 98 original temperature units indicator 40 99 sea temperature measurement method indicator 41 100-101 wind wave period (seconds) 42 102 description of ice type (code) 42 103 effect of ice on navigation (code) 42 104 bearing of principal ice edge (code) 42 105 distance to ice edge from ship (code) 42 106 orientation of ice edge (code) 43 107-108 amount of precipitation (code) 43 109-110 time period for precip. amount (code) 44 111 significant cloud amount (code) 45 112 significant cloud type (code) 46 113-114 significant cloud height (code) 47 115 ship position - flag 48 116 wind - flag 48 117 visibility - flag 48 118 present weather - flag 48 119 past weather - flag 48 120 pressure - flag 48 121 air temperature - flag 48 122 dew point/wet bulb - flag 48 123 sea surface temperature - flag 48 124 cloud - flag 48 125 wave - flag 48 126 swell - flag 48 127 pressure change - flag 49 128-129 quality code 50 130-134 Julian date (year, day) of QC 51 135-136 blank 52 137-140 reserved for NCDC use only ----------------------------------------------------------------
This format is intended to replace both TD-1100 and TD-1127 as an all purpose character-based marine format. TD-1129 is for recent data, and its variant TD-1129M is for old data from TD-1100 (e.g., the Eltanin and South African Whaling data sets). The notation TD-1129(M) refers to either TD-1129 or TD-1129M. In general the form is
location = 1-26(26) + regular3 = 27-78(52) + irregular2 = 79-148(70).In comparison with regular1, overpunches have been eliminated and the air-sea temperature difference has been dropped from regular3, so that its contents are essentially equivalent to regular1. Table I1-4 gives the format layout for TD-1129.
Field Char number position Description ------------------------------------------------------------------------- 1 1-3 card deck 2 4-6 10° Marsden Square 3 7-8 1° Marsden sub-square 4 9 quadrant (1-4) 5 10-12 latitude (degrees N, S) 6 13-16 longitude (degrees E, W) 7 17-20 year 8 21-22 month 9 23-24 day 10 25-26 hour (GMT) 11 27 wind direction indicator 11 28-29 wind direction (code) 12 30 wind speed indicator 12 31-33 wind speed (knots) 13 34 visibility indicator 13 35-36 visibility (code) 14 37-38 present weather (code) 15 39 past weather (code) 16 40-44 sea level pressure (mb) 17 45 temperature indicator 17 46-49 air temperature (°C) 18 50-53 wet bulb temperature (°C) 19 54-57 dew point temperature (°C) 20 58-61 sea surface temperature (°C) 21 62 total cloud amount (oktas) 21 63 low or middle cloud amount 21 64 type of low cloud 21 65 cloud height indicator 21 66 lowest cloud height 21 67 type of middle cloud 21 68 type of high cloud 22 69-70 direction of waves (code) 23 71 period of waves (code) 24 72-73 height of waves (1/2 meters) 25 74-75 direction of swell (code) 26 76 period of swell (code) 27 77-78 height of swell (1/2 meters) 28 79-80 country code 29 81 ship direction (code) 30 82 ship speed (code) 31 83 barometric tendency (code) 32 84-86 amount of pressure change (mb) 33 87 type of ice accretion on ship (code) 34 88-89 thickness of ice on ship (cm) 35 90 rate of ice accretion (code) 36 91-97 ship, OSV, or buoy call sign 37 98 original wind speed units indicator 38 99 original temperature units indicator 39 100 sea temperature measurement method indicator 40 101-102 wind wave period (seconds) 41 103-104 swell wave period (seconds) 42 105 concentration of ice (new code 1982) description of ice type (code) 42 106 stage of ice development (new code 1982) effect of ice on navigation (code) 42 107 ice of land origin (new code 1982) bearing of principal ice edge from ship (code) 42 108 bearing of principal ice edge from ship (new code 1982) distance to ice edge from ship (code) 42 109 situation or trend (new code 1982) orientation of ice edge (code) 43 110-111 amount of precipitation (code) 43 112-113 time period for precip. amount (code) 44 114 significant cloud amount (code) 45 115 significant cloud type (code) 46 116-117 significant cloud height (code) 47 118 second past weather (code) 48 119-120 second swell direction (code) 49 121-122 second swell period (seconds) 50 123-124 second swell height (1/2 meters) 51 125 ship position - flag 52 126 wind - flag 52 127 visibility - flag 52 128 present weather - flag 52 129 past weather - flag 52 130 pressure - flag 52 131 air temperature - flag 52 132 wet bulb temperature - flag 52 133 dew point temperature - flag 52 134 sea surface temperature - flag 52 135 cloud - flag 52 136 wave - flag 52 137 swell - flag 52 138 pressure change - flag 53 139-140 quality code 54 141-142 QC - year 54 143-144 QC - month 55 145 indicator for wave measurement (1982 code) 56 146 source of observation on card (1982 code) 57 147 observation platform (1982 code) 58 148 source ID (A-X corresponds to 1-24 in LMR) -------------------------------------------------------------------------
For older data (TD-1129M), irregular2 contains information that is practically equivalent to that contained in irregular1 according to the following transformation.
a) Standard1 data 78-81(4) in TD-1100 move to 79-82(4) in TD-1129M.Clearly, depending on the sub-format, supplemental data may not all fit. Decks 110, 117, 281, 192, 150, 151, and 152 require special treatment:
b) Additional data 82-88(7) move to 141-147(7).
c) Standard2 data 89-93(5) move to 83-87(5).
d) Supplemental data 94-140(47) move to 88-124(37).
Supposedly, standard1 and standard2 are always blank in this deck. Omitting a presumably useless hundreds position of relative humidity in the first character (making 0 and 100% equivalent), the remaining supplemental data move instead to 79-124(46).
Similarly omitting the hundreds position of relative humidity in the first character, supplemental data move to 88-124(37).
Since these have shorter supplemental data, and a standard1 and standard2 that are also supposed to be blank, supplemental data can move to 79-124(46) without omitting the first character.
In practice, these special transformations do not work as stated because supplemental, standard1, and standard2 often contain undocumented or erroneous characters. Some of these characters were "area codes" assigned for special Atlas studies or they were dates when data were added to a data base at NCDC.
1.2 Exchange Format
The United States, Germany (F.R.G.), the Netherlands, and the United Kingdom apparently used this as the format for exchange of reports gathered in the Historical Sea Surface Temperature (HSST) Data Project. In order to minimize processing, data from the German and Netherlands areas of responsibility (Atlantic and Indian Oceans) were translated from the Exchange format directly into LMR, even though a TD-1100 transcription was available from NCDC. For the United States area of responsibility (Pacific Ocean), the TD-1100 was used because it contained additional data not available in the Exchange format.
The Exchange format obtained (Table I1-5) is considerably shorter
(46 characters) than any class of TD-11. The format differs slightly
depending on which country provided a report (no details were
available on the U.K. format), and each area of responsibility
contains data merged together from the four countries. Although source
ID (and card deck) identify the area of responsibility, no identifier
is available in the Exchange format showing which country provided a
report. For translation to LMR, a special supplemental was defined for
this format (later subject to special treatment as described for deck
152 in translation from LMR to TD-1129M). Characters from columns 33-35
and 42-46 were saved in the supplemental attachment. This includes
the wind speed and all flag information from the original format, as
shown in Table I1-5.
Field Char number position Description -------------------------------------------------------------------------------- 1 1-1 octant ---------------------------------------------- S. Hemisphere N. Hemisphere Longitude ---------------------------------------------- 5 0 0-90°W 6 1 90-180°W 7 2 180-90°E 8 3 90-0°E ---------------------------------------------- 2 2-3 10° square (tens digit of latitude and longitude) 3 4-5 month 4 6-9 year 5 10-11 latitude (units and tenths digit) 6 12-13 longitude (units and tenths digit) 7 14-15 day 8 16-17 hour (GMT) 9 18-21 sea surface temperature 10 22-25 air temperature 11 26-29 wet bulb temperature in tenths of a degree Celsius, or replaced by 999 if missing, with the sign as the first position. In the U.S. and Nether- lands formats the sign is blank if the value is positive. In the German format the sign is "+", "-", or "E" (the latter used only for wet bulb with ice). 12 30-32 wind direction in whole degrees from north (converted from other units if necessary) with 000 for calm, 990 for variable, or 999 for missing. 13 33-35 wind speed in tenths of a meter per second (converted from other units if necessary) with 999 for missing. 14 36-40 sea level pressure in tenths of a millibar, with 99999 for missing. 15 41 total cloud amount in oktas, with 9 for obscured, or blank for missing. In the U.S. and Netherlands formats, missing occurs with f-sus2 (field 20) of 2, 3, 6, or 7. 16 42 f-sea U.S. and Netherlands formats only, flag for measurement precision of sea surface temperature and the state of the wet bulb. Codes 0 to 9: Codes 0 to 4 (Netherlands for frozen wet bulb); codes 5 to 9 for unfrozen wet bulb, even when showing temperature below freezing point (or Netherlands wet bulb temperature missing): -------------------------------- Code Code Precision -------------------------------- 0 5 0.1°F 1 6 0.1°C 2 7 0.5°F 3 8 0.5°C 4 9 1°F or 1°C -------------------------------- 17 43 f-air U.S. and Netherlands formats only, flag for measurement precision (as given by f-sea) of air and wet bulb tempera- tures. Codes 0 to 9: codes 0 to 4 (Netherlands for wet bulb temperature missing); codes 5 to 8 for temperatures measured by an aspirated or whirling psychrometer; code 9 for original units or precision of temperatures unknown. 18 44 f-wind Flag for wind observation. Codes 0 to 9: codes 0 to 4 indicate wind speed measured; codes 5 to 9 indicate wind speed estimated or converted from Beaufort force, or method of observation unknown (in the German format, only codes 1, 6, and 7 are documented, with 6 and 7 differing by definitely indicating a conversion from Beaufort force): -------------------------------- Code Code Point compass -------------------------------- 0 5 360 1 6 36 2 7 32 3 8 16 4 9 8 -------------------------------- 19 45 f-sus1 U.S. and Netherlands formats only, flag for suspect values of sea surface and air temperatures, and wind. Codes 0 to 7: ----------------------------------------------------------- Code Condition ----------------------------------------------------------- 0 none of the following conditions 1 suspect sea surface 2 suspect air temperature 4 suspect wind 3,5-7 more than one value suspect, codes added together ----------------------------------------------------------- 20 46 f-sus2 U.S. and Netherlands formats only, flag for suspect values of pressure, cloud amount, or additional report. Codes 0 to 7: ----------------------------------------------------------- Code Condition ----------------------------------------------------------- 0 none of the following conditions 1 suspect pressure 2 cloud amount not reported 4 additional report at same time in the same 1° square though not identical. 3,5-7 more than one value suspect, codes added together. --------------------------------------------------------------------------------
The three TD-11 classes take the following forms:
TD-1100: location = 1-26(26) + regular1 = 27-77(51) + irregular1 = 78-140(63)Note that location remains unchanged. Actually positions 1-45 are invariant. Moreover, the contents of regular1 and regular3 are essentially equivalent as noted previously. Thus, only the final section of each class contains variable information, and the first two sections in each can all be entered into a uniform location and regular section in LMR. Adding a control section and an irregular section at the end completes the format as described fully in supp. F.
TD-1127: location = 1-26(26) + regular1 = 27-77(51) + regular2 = 78-140(63)
TD-1129: location = 1-26(26) + regular3 = 27-78(52) + irregular2 = 79-148(70)
2. Corrections and Conversion into LMR
A number of known data problems were corrected at the conversion into LMR, and prior to sorting the data as required by duplicate elimination. These and other conversion details given here impact the LMR, and in some cases also apply to the TD-1129(M). See supp. K for a description of earlier changes made in the translation from miscellaneous formats (e.g., Japanese, Australian) into TD-11, performed at NCDC.
2.1 Character Translation Tables
Some possible overpunch-numeric combinations can result in confusing character conventions. Therefore, the following conventions were always used. For the most part, these are consistent with the most commonly used conversions.
12 overpunches and numbers 1-9 map to letters A-I.These letters are well defined in all character sets. In the supplemental attachment, ebcdic is used to represent the letters A-Z, and special characters translate into the ship character set as shown in Table I2-1.
11 overpunches and numbers 1-9 map to letters J-R.
------------------------------------------------------------ | ebcdic | ascii | CDC dpc |Ship 026 punch code| Hex Char| Hex Char | Octal Char | Hex --------------|------------|-------------|-------------|---- 12-0 | C0/4C {/< | 7B/3C {/< | 72 < | C0 11-0 | D0/4F }/| | 7D/21 }/! | 66 ! | D0 12 | 4E + | 2B + | 45 + | CB 11 | 60 - | 2D - | 46 - | DA 0-8-7 | 50 & | 26 & | 67 & | CA 0-1 | 61 / | 2F / | 50 / | E1 12-8-4 | 5C * | 2A * | 47 * | EA ------------------------------------------------------------
For deck 194, if watch number was 6, 1 was added to the day (and month/year if applicable), and watch number was changed to 9.
2.3 Hour
Any time hour was 24, 1 was added to the day (and month/year if applicable), and hour was changed to 00. An hour of 99 was considered missing.
2.4 Pre-July 1963 Wave Fields Applicable to both wave and swell data* before July 1963 (exclusive):
a) If 51 ≤ direction ≤ 86, then direction = direction - 50 and height = height + 10.
b) If direction equals 99 and height < 10, then height = height + 10.
________________________
* Prior to the code change of 1 July 1963, only the higher of the
(wind) wave and swell was reported. Standard practice at NCDC was to
put this into the wave (not swell) fields.
________________________
2.5 Cloud, Wave, and Swell Fields
Sometimes / was keyed in place of - in fields where - was a legal value, and / or - were keyed in place of space (S) when the cloud or wave fields were missing. Specifically:
a) Cloud Fields. (& was also keyed in place of -.) These two steps were used to determine if the LMR cloud fields C, NH, CL, HI, H, CM, CH were all missing.
i) Any / or & changed into -.Otherwise - was changed into S in fields where - was not legal (C, NH, HI).
ii) If all seven positions were then S or - in any combination, then all seven fields were considered missing and were changed into SSSSSSS.
b) Wave and swell fields were all missing if they fit one of these 5-character patterns:
i) SSSSSand were all changed into pattern i). In addition, waves only (not swells) were all missing if they fit one of these 5-character patterns (which include i)-iii) as special cases):
ii) -----
iii) /////
iv) DDSSSwhere D is any character. That is, if the last three characters were SSS, ---, or /// this field was changed into pattern i).
v) DD---
vi) DD///
2.6 Indicators Referring to Missing Data
Non-blank indicators referring to blank (missing) data were made blank:
a) Wind direction indicator if direction was blank.This rule does not apply to indicators that refer to erroneous data.
b) Wind speed indicator if speed was blank.
c) Visibility indicator if visibility was blank.
d) Temperature indicator if all of the temperature fields were blank.
e) Cloud height indicator if cloud height was blank.
2.7 Time/Space Location Errors
Reports with errors or inconsistencies in time or geographical location were written out to a reject file for later work. For reports in which the Marsden Square disagrees with quadrant, latitude, and longitude (or the corresponding inconsistencies in Exchange format data), both the 10° box and 1° MSQ should be recomputed and the report relocated accordingly, when time permits. This was done for a few of the smaller data sets.
2.8 Card Deck Assignments
The following new card decks were assigned during this project:
*155 HSST Indian (Boulder conversion)Only those decks with an asterisk (*) required action at this conversion stage, the others had been assigned during NCDC's conversions. NCDC assigned 154 to its conversion of both the HSST Indian and Atlantic basins.
*156 HSST Atlantic (Boulder conversion)
897 Eltanin
*898 Japanese (change from 926)
899 S. African
900 Australian
926 IMMPC
2.9 Monterey Telecom. Pre-processing
Owing to the questionable quality of this data set, checks were made for the following conditions:
a) Positions 70-77 (period and height of sea; direction, period, and height of swell) equal to -0031000.Any report with one or more of a)-c) true was written to a reject file. This was expected to eliminate most hard duplicates (supp. K) internal to the Monterey Telecom.
b) Present weather missing (blank) when past weather was any nonblank character.
c) Calm wind direction when speed was greater than or equal to 7 knots.
2.10 Existing Ship Type
Only TD-1100 inputs had a field for ship type, to which these changes were made:
a) For any deck, a ship type of 2 with a negative overpunch was converted to 3 in order to help eliminate overpunches from the format. Subsequently, ship type was set to 2 if not 2 or 3 for source IDs 8, 9, and 20 (OSV data).
b) For source IDs 2 and 7 (HSST Pacific and Monterey Telecom.), ship type was set to missing.
c) For deck 891, a ship type of 6 was intended to indicate a research ship (or SD, meaning station data) but was inadvertently assigned to every report in this deck, including bathythermographs (XBT and MBT), during the translation into TD-11. Position 103 was expected to contain the type. Thus,if type = 1 (MBT) then ship type = 7;If type was not one of these values, tests were made for the presence of the weather elements sea surface temperature, air temperature, pressure, and wind (speed and direction). If only sea surface temperature was extant, the ship type was changed to 7; otherwise ship type was left 6 to indicate a research ship.
if type = 2 (XBT) then ship type = 7;
if type = 3 (SD) then ship type = 6.
2.11 Derived Ship Type
Inputs other than in TD-1100 did not have a field for ship type. Data in the Exchange format had no form of ship identification, so ship type became missing. For data in TD-1129M, ship type was set to 6 for Eltanin data, or else it was set to missing. For data in TD-1127 and TD-1129, ship type was set to missing with two exceptions for buoy data: a) '70s Decade or '70s Mislocated Data (source IDs 18 and 23) had ship type set to 5 if deck was 143 or 876-886; b) for source ID 24 ship type was automatically set to 5.
2.12 Past Weather Containing Overpunch
A negative overpunch with a numeric past weather was stripped off and the numeric retained in decks 151, 192, and 899.
2.13 Wind Speed Conversion from Knots to Meters Per Second
Decks 128, 150, 151, 152, 185, and 926 have been identified as cases in which some or all of the original wind speeds were translated from meters per second into whole knots to fit in TD-11. The international convention (1 m s-1 = 1.9438445 knot) was used to convert all decks back to meters per second, regardless of the fact that the U.S. convention (1 m s-1 = 1.94254 knot) was probably used for the reverse conversion in the six decks; this was done because of a lack of complete documentation -- the problem should be fixed later.
2.14 19th Century IMMPC
These were all changed to the corresponding year of the 20th century because manual inspection showed 19th century reports always to be adjacent, with a sharp break, to reports in the 20th century.
2.15 Japanese Wind Direction
These special characters were changed when encountered in the high-order position of the wind direction in the Japanese data:
! changed into 0,In addition, when one of the three (lower-case only) letters was encountered, 100 was subtracted from the wind speed. (Original wind speeds less than 100 were considered erroneous.)
w changed into 1,
u changed into 2,
v changed into 3.
2.16 South African Minus Sign
Any field in the "regular" section that contained all minus signs (-), and for which "all minus signs" was not a legal value, was made blank.
2.17 Bucket Indicator
Only in TD-1129 was there a bucket indicator in the "regular" section, and only in the Australian set was there a value, 9, for a missing indicator. A blank in this field was interpreted as missing, except in the Australian set, where it was interpreted as intake.
2.18 Australian '70s
This set had WMO-defined quadrant numbers. These were translated into the quadrant system used by NCDC according to the following:
WMO NCDC ---------- 3 4 5 3 7 1 1 2 ----------
2.19 Wet Bulb With Ice in the German Exchange Format
At conversion time the sign character (E), which specified wet bulb with ice, was unknown, so that all wet bulb temperatures containing a character other than a blank, plus, or minus were considered erroneous.
2.20 Temperature Indicator
This indicator has a different meaning for source TD-11 data versus source Exchange data. In TD-11, the only legal values correspond to TI = 0, 1, or 2 (0.1°C, 1°C, or 0.5°C). Data converted from 0.1°F, 1°F, or 0.5°F were set at NCDC with TI = 0 to indicate that the tenths position of temperature, after conversion, might be any digit (e.g., not constrained to 0 or 5).
In contrast, the Exchange format has a flag with possible values for original measurement precision of 0.1°F, 0.1°C, 0.5°F, 0.5°C, and 1°F or 1°C. No allowance was made for mixed precision among the different variables or the state of the wet bulb as given by the Netherlands version of the Exchange format. When mixed precision was indicated, or precision of 1°F or 1°C, TI was set to missing.
3. Conversion from LMR into TD-1129(M)
For some recent data (TD-1129 or TD-1129M) this step reversed the conversion into LMR, except that corrections and modifications made at that stage were retained. However, most of the data required rearrangement of fields or other modifications to achieve a more uniform format. These transformations are covered in the background on TD-1129(M) in sec. 1.1. Additional details are given here.
3.1 Bucket Indicator
A missing indicator, and the values for unknown and implied bucket (BI codes 0 and 2) were all translated into 0 in TD-1129.
3.2 Uncertain Duplicates
Only reports with a dup status strictly less than 3 were converted into TD-1129(M), which eliminated all uncertain duplicates as defined in supp. K.
3.3 Undocumented Supplemental Data
Undocumented fields, such as the Atlas "area code" from original positions 137-140, were not blanked out, and as many such characters as would fit were included.
3.4 Erroneous Fields and their Indicators
A non-blank indicator associated with an erroneous field was blanked out, as was the field. However, quality control flags referring to erroneous data were retained.
3.5 Leading Zeros
Numeric values were prefixed by leading zeros where necessary to fill the entire field up.
3.6 Positive Temperatures
These have an explicit plus in the sign position.
3.7 Exchange Source Wind Directions
Because the value in degrees for decks 155 and 156 (or source IDs 3 and 4) may not coincide with any of the compass midpoints chosen for a given direction indicator, as discussed in supp. F, a deviation ± 2° around the values in Table F2-1 was allowed.
3.8 Source ID
Source ID was coded as a single character (A-X) corresponding to the numeric values in use in LMR (1-24), and placed in position 148 of TD-1129(M).
3.9 Special Transfers for 1970-1979 Data Exclusive of the '70s Decade
In order to make the '70s strictly TD-1129 (not TD-1129M),
special modifications were required for some source TD-1100 data.
Decks 128 and 891, exclusive of the '70s Decade (i.e., not source ID
18), were modified. Tables I3-1 and I3-2 show the respective transfers
made for these two decks of data from TD-1100 positions 78-140. Any
data not explicitly transferred from positions 78-140 were deleted,
resulting in the loss of some supplemental data from TD-1129.
Field TD-1100 TD-1129 ----------------------------------------------------------------- ocean weather station number or country code* 78-79 79-80 when column 82=1 type of ice accretion on ship 83 87 thickness of ice on ship 84-85 88-89 rate of ice accretion 86 90 when column 82=6 ship direction 83 81 ship speed 84 82 barometric tendency 85 83 amount of pressure change 86-88 84-86 when column 82=8 significant cloud amount 83 114 significant cloud type 84 115 significant cloud height 85-86 116-117 ship number 90-93 91-94 original temperature units indicator 98 99 sea temperature measurement method indicator 99 100 wind wave period 100-101 101-102 ----------------------------------------------------------------- * Not transferred if TD-1100 position 81 (ship type) was 2 or 3. _____________________
Field TD-1100 TD-1129 -------------------------------- ship number 90-95 91-96 --------------------------------
Because the '70s Decade data set had been previously quality controlled at NCDC, two sets of QC flags are available in LMR. In order to reconcile the two sets of flags, which are based on differing procedures, the more serious value from each pair of flags (see supp. J) was output. This will help catch those suspect or erroneous fields that NCDC failed to flag, and at the same time retain those flags received during the track check performed at NCDC.
Overlaying of flags was done only for the '70s Decade (source ID 18), not the '70s Mislocated Data (source ID 23). Except for flags R (correct) and S (missing), the flag with the higher alphabetic ranking A through Q was chosen. Flag R was always discarded in favor of any one of A through Q, and S should appear only with missing fields. In the event a new flag stated a field was missing, but the old did not, the new flag was chosen. In order to identify the quality control procedure(s) that produced the resulting flags, three different QC dates were output: a) old date if all old flags, or old and new were the same; b) June '84 if old and new mixture; or c) May '83 if all new.