=============================================================================== International Comprehensive Ocean-Atmosphere Data Set (ICOADS): Release 2.4 Digitization Format: US Maury Collection 22 September 2007 ================================================================= Document Revision Information (previous version: 9 September 2002): Updates (heading only) for Release 2.4. ------------------------------------------------------------------------------- {1. Introduction} This is documentation for the format that was used for digitization of the US Maury Collection, which is an amplification and clarification of the format documentation presented in NCDC (1998). Blank was used to represent missing data in this format. The format included two types of records: header (Table 1) and data (Table 2). The two types of records can be joined by means of the voyage header (positions 1-7 in both record types). Both record types are 173 characters in length. Header records contain blank in column eight, whereas in data records that column contains the first digit of year. To assist data conversion and interpretation, inventory results for individual fields have been noted, including Table 3, which summarizes inventory results for the three sets of wind direction and force fields. Table 1. Header record format. There was one header record for each voyage. Generally, a new header was created when a ship began a return voyage or started a second voyage to another destination (a given ship may have made many voyages). Fields whose description contains "Blank" (not numbered) were intended to contain values that did not materialize during the digitization of the data. ------------------------------------------------------------------------------- # Pos. Total # Field Description Explanation range of pos. name =============================================================================== 1 1-7 7 cvoyh Voyage number Sub-divided as follows: ---------------------------------------------------------------------------- 1-2 2 Reel number The microfilm reel number. 3-6 4 Frame number The number found at the top of the microfilm frame (e.g., 0578) where the voyage begins. The frame number was kept until the voyage is complete, even if the voyage covered several sequential frames. 7 1 Voyage The seven characters in positions sequence 1-7 constitute a unique number identifying each voyage. The 7th character (1-9) was used in case more than one voyage begins on a single microfilm frame. For example, if a voyage on reel 45 starts on frame 0578 and took two and a half frames for all the observations, the first 7 positions would be the same for each data record (observation) on those two and a half frames; in this case it would be 4505781. If there were two additional unique voyages on the second half of the third frame (frame 0580), the numbers would be 4505801 and 4505802 respectively. If the voyage starting on the second half of the third frame continued through the entire fourth frame (frame 0581), the number would continue as 4505802, and a new voyage beginning on the fifth frame would be 4505821. ---------------------------------------------------------------------------- -- 8 1 Blank -- 2 9-32 24 cnship Name of ship The name of the ship as it appeared in the log. Some editing performed. 3 33-47 15 ctship Type of ship The type of ship as it appeared on the inventory frame at beginning of the reel. Examples: bark, ship, schooner. Some editing performed. 4 48-49 2 cft Form type Documents the log form types. Only two form types, 01 and 02, were established during the keying of the US Maury collection. [NOTE: These are: 01 = daily reports 02 = multiple reports per day In the keyed data, leading zero was always present. A total of 10,230 were cft=01, and 2,106 were cft=02.] 5 50-73 24 comm Commander The vessel's commander as it appeared on the form. Some editing performed. 6 74-97 24 cfr From city The city from which the vessel began its voyage, as it appeared on the form. Some editing performed. 7 98-121 24 cto To city The voyage destination as it appeared on the form. Some editing performed. -- 122-173 53 Blank -- ------------------------------------------------------------------------------- Table 2. There is at least one data record (i.e., marine report or "observation") for each voyage. ------------------------------------------------------------------------------- # Pos. Total # Field Description Explanation range of pos. name =============================================================================== 1 1-7 7 cvoyd Voyage number A unique number connecting the header information with the data. These are the same 7 characters as are assigned to the first seven positions in the header record (see Table 1 for more information). 2 8-11 4 cyr Year Four-digit year. 3 12-13 2 cmo Month 01 - 12. 4 14-15 2 cdy Day 01 - 31. 5 16-17 2 chr Hour (of Converted to 24-hour clock (00-23) ship's where required, otherwise as it position) appears on the form. The hour given with the latitude and longitude field; e.g., if the form heading indicates noon position, hour was then coded as 12. If no time was indicated, leave blank. [NOTE: The instructions for keying hour seem ambiguous, i.e., what does "where required" mean? Could this have been the source of some problems in the hour field? Local time should be added to the above description.] 6 18-22 5 clat Latitude Positions 18-19 = degrees positions 20-21 = minutes (if missing leave blank) position 22 = hemisphere (N, S, or left blank if missing) Note: If latitude was missing the observation was still keyed. In most cases the latitude can be interpolated from adjacent locations. If tenths of degrees were given, they were converted to minutes and placed in positions 20-21. 7 23-28 6 clon Longitude Positions 23-25 = degrees positions 26-27 = minutes position 28 = hemisphere (E, W or left blank if missing) Note: Same as for latitude. [NOTE: Were minutes of longitude also left blank if originally blank?] 8 29-35 7 curd Current (same as positions 104-110) direction 9 36 1 cursi Current speed Units in which speed was measured: indicator Description Code --------------------------- ---- Knots 1 Miles per hour 2 Miles traveled in 24 hours 3 Miles traveled in 21 hours 4 Meters per second 5 Kilometers per hour 6 Descriptive terms 7 Units not known 8 Distance known, time unknown 9 10 37-40 4 curs Current speed If speed was in fractions, code as follows: Speed Code ------ ---- 1/4 0025 1/3 0033 1/2 0050 3/4 0075 1 0100 1 1/2 0150 15 1500 15 1/2 1550 If current speed was described [in narrative terms], code as follows: Descriptive Term Code ---------------- ---- Brisk 40 Declining 41 Faint 42 Fine 43 Fresh 44 Gale 45 Good 46 Hard 47 Heavy 48 Less 49 Light 50 Moderate 51 Nice 52 Pleasant 53 Squall 54 Steady 55 Stiff 56 Strong 57 Unsteady 58 11 41-42 2 curm Minutes Minutes of latitude or longitude of "current drift" over a given time period (as indicated in positions 43-44); e.g. 11' is coded as 11, 2' is coded as 02. 12 43-44 2 curtp Time period Time period (in hours) over which the amount of current drift in minutes was calculated, e.g.: 1 hour code as 01 21 hours code as 21 24 hours code as 24 Note: Most are reported for a 24-hour period, e.g., 11 minutes of drift calculated over a 24-hour period. 13 45 1 cmvi Magnetic 1 = degree (positions 46-50) variation 2 = points (positions 46-50) indicator 14 46-50 5 cmv Magnetic If indicator = 1: variation Positions 46-47 = degrees Positions 48-49 = minutes Positions 50 = E or W If indicator = 2: Positions 46-47 = points Position 48 = tenths of points Position 49 = blank Position 50 = E or W Example: "1 1/2 Pts E" coded as 15 in positions 47-48, E in position 50, positions 46 and 49 left blank. Note: If degrees, minutes, points or East/West declination are missing, only the affected positions are blank. 15 51-52 2 chb Barometer Time of barometer reading (hour observation 00-23), if available. Blank if time missing (as was generally the case). [NOTE: In the keyed data, this field was missing except chb=08 (126 cases).] 16 53-56 4 cb Barometric If pressure was given in inches: pressure (first Positions 53-54 = inches or only entry) Positions 55-56 = hundredths of inch Examples: Inches Code ------ ---- 29.9 299 29 29 29 9/10 299 29 9/100 2909 If pressure was given in millimeters: Positions 53-55 = mm Position 56 = tenths of mm Examples: Millimeters Code ----------- ---- 768.4 7684 768 768 768.0 7680 17 57 1 ct1 Temperature 1 = All temperatures are Fahrenheit indicator 2 = All temperatures are Celsius [NOTE: In the keyed data, ct1 values were limited to 1, 2, and blank. The following additional indicator values were defined, but never keyed:] 3 = All temperatures are Reaumer 4 = Dry bulb temperature is Fahrenheit Sea temperature is Celsius 5 = Dry bulb temperature is Celsius Sea temperature is Fahrenheit 6 = Attached thermometer is Fahrenheit Dry bulb temperature is Fahrenheit Sea temperature is Celsius 18 58-61 4 cbt1 Attached Code such as follows: thermometer If Fahrenheit, e.g.: (first or 79 keyed as 079 (pos. 61 blank) only entry) -5 keyed as -05 " -12 keyed as -12 " 101 keyed as 101 " If Celsius, e.g.: 30 keyed as 030 " 10 keyed as 010 " -2 keyed as -2 (pos. 58,61 blank) -15 keyed as -15 30.1 keyed as 0301 -10.3 keyed as -103 If Reaumur (same as Celsius) Note: Position 61 is blank for whole degrees, and quarter degrees are rounded to the nearest tenths of a degree. [NOTE: Based on the examples, leading zeros are optional ("0301" versus " -2"), plus sign was omitted, and the minus sign could occupy any position. A digit in positions 59-61, respectively, indicated 10s, units, and 10ths. No example was given of a digit in position 58, but this presumably would indicate 100s. Values less than -99 cannot be stored; in the keyed data, however, no negative values whatsoever appeared.] 19 62-65 4 cbt2 Attached (same as positions 58-61) thermometer (second entry) 20 66-69 4 cbt3 Attached (same as positions 58-61) thermometer (third entry) [NOTE: In the keyed data, fields cbt2-3 were always blank. Since there was only one field (cb) for barometric pressure, these fields were unnecessary.] 21 70-71 2 cha1 Hour of first Hour in terms of a 24-hour clock air temp. (local time). Examples: entry 9 AM keyed as 09 9 PM keyed as 21 Note: Left blank if no time was indicated. [NOTE: In the keyed data, cha1 as expected was: almost exclusively missing in form type 2 data, and when extant almost exclusively 09 (245,541 cases; 08 in 105 cases, and 18 in 54 cases).] 22 72-75 4 ca1 Air temp. (same as positions 58-61) (first entry) 23 76-79 4 cs1 Water temp. (same as positions 58-61) at surface (first entry) 24 80-83 4 cd1 Water temp. (same as positions 58-61) at depth (first entry) [NOTE: These is only one field for water temperature at depth, but the explanation for this field states: "first entry." Does this mean that if more than one entry was available that only the first entry was included in this position? E.g., if only a second entry was available in the original data, what was done? Clarification should be added if available.] 25 84-85 2 cha2 Hour of second (same as positions 70-71) air temp. entry 26 86-89 4 ca2 Air temp. (same as positions 58-61) (second entry) 27 90-93 4 cs2 Water temp. (same as positions 58-61) at surface (second entry) [NOTE: The second group of temperature fields (cha2,ca2, cs2) was generally missing (a total of 158 extant values in each field).] 28 94-95 2 cha3 Hour of third (same as positions 70-71) air temp. entry 29 96-99 4 ca3 Air temp. (same as positions 58-61) (third entry) 30 100-103 4 cs3 Water temp. (same as positions 58-61) at surface (third entry) [NOTE: The third group of temperature fields (cha2,ca2, cs2) was generally missing (a total of 52 extant values in each field).] 31 104-110 7 cwd1 Wind direction Left justify and blank fill. (first part --If a standard direction was given, or only entry) it was keyed as N, NNE, NE, ENE, E, ESE, SE, SSE, S, SSW, SW, WSW, W, WNW, NW, NNW. --If Calm, Cl, Variable, Var., Vrb or Baffling was indicated it was keyed as C (calm), V (variable), or B (baffling). --Combinations of alpha and numeric were keyed as written. Examples: N45E, S67E, N15W. --If one direction was "by" another, an "X" was used to connect the directions. Examples: "WNW by N" keyed as WNWXN "NNE by NE" keyed as NNEXNE --If one direction was "to" another, a "t" was used to connect the directions. Examples: "NNW to N" keyed as NNWTN "SE to SSE" keyed as SETSSE. --Directions not in English were translated to English and coded accordingly. Example: French code "ONO" was keyed as WNW. --If a direction was not codeable, a dash (-) appears in position 104. 32 111-113 3 cwf1 Wind force Sub-divided as follows: ---------------------------------------------------------------------------- 111-112 2 Wind force Beaufort Wind Scale (0-12) unless the (first part observation was in a descriptive form. or only entry) If this occurred, the wind force was coded [20-33 if the following specific terms were present:] Beaufort wind Code --------------- ------ Calm 20 Light air 21 Light breeze 22 Gentle breeze 23 Moderate breeze 24 Fresh breeze 25 Strong breeze 26 Moderate gale 27 Fresh gale 28 Strong gale 29 Whole gale 30 Storm 31 Hurricane 32 Baffling 33 Note: Descriptive terms corresponding to 20-32 are Beaufort's 1805 terminology (see Curtis, 1897). For all the Beaufort codes (0-12 and 20-33) listed above, position 133 should always be blank. 113 1 Add. wind Or, additional descriptive terms for force info. wind force could be encoded using the (when following 1- or 2-word combinations available) (i.e., coded as a 2-digit code for one word in positions 111-112, followed optionally by a 1-digit code for a second word in position 113). POSITIONS 111-112 POSITION 113 ----------------- ------------- TERM | CODE TERM | CODE ----------------- ------------- Brisk 40 Breeze 1 Declining 41 Gale 2 Faint 42 Trade 3 Fine 43 Wind 4 Fresh 44 Gale 45 Good 46 Hard 47 Heavy 48 Less 49 Light 50 Moderate 51 Nice 52 Pleasant 53 Squall 54 Steady 55 Stiff 56 Strong 57 Unsteady 58 Violent 59 Gentle 60 Fair 61 Variable 62 Smart 63 Small 64 Increasing 65 Changeable 66 Tremendous 67 Furious 68 Prosperous 72 High 73 Examples: Brisk Gale = 402 in pos. 111-113 Strong = 57 in pos. 111-112, pos. 113 blank Moderate Gale = 512 in pos. 111-113 Pleasant Breeze = 531 in pos. 111-113 Gale = 45 in pos. 111-112, pos. 113 blank ---------------------------------------------------------------------------- 33 114-120 7 cwd2 Wind direction (same as positions 104-110) (middle part) 34 121-123 3 cwf2 Wind force (same as positions 111-113) (middle part) 35 124-130 7 cwd3 Wind direction (same as positions 104-110) (latter part) 36 131-133 3 cwf3 Wind force (same as positions 111-113) (latter part) [NOTE: See Table 3 for a summary of inventory results of the wind direction and force fields for the three parts of the day, separated by form type. These results indicate, as expected, that the fields for the middle and latter parts of the day were almost always missing for form type 2 data.] 37 134-135 2 ccf1 Cloud form Abbreviate cloud types as follows: (first or only) Cloud Type Code ---------- ---- Cirrus CI Cirrocumulus CC Cirrostratus CS Altocumulus AC Altostratus AS Stratus ST Stratocumulus SC Nimbostratus NS Cumulus CU Cumulonimbus CB 38 136-142 7 ccd2 Direction of (same as positions 104-110) clouds (first or only entry) 39 143-144 2 ccf2 Cloud Form (same as positions 134-135) (second entry) 40 145-151 7 ccd2 Direction of (same as positions 104-110) clouds (second entry) 41 152-153 2 ccf3 Cloud Form (same as positions 134-135) (third entry) 42 154-160 7 ccd3 Direction of (same as positions 104-110) clouds (third entry) 43 161-162 2 csc Proportion 0 through 10. Example: of sky clear "9/10", "9.", or ".9" are all keyed as 09. 44 163-164 2 chx Hours of Whole hours with any fraction rounded weather to the nearest hour. Examples: "1/2" keyed as 1 in position 163 "1.5" keyed as 2 in position 163 "10 hours" keyed as 10 in positions 163-164. Note: Hours of weather are generally associated with a present weather indicator value of "2". Blank if not reported. 45 165 1 cix Present 1 = Beaufort Weather Code weather 2 = Four Choice Code indicator 3 = WMO Code 4677 46 166-171 6 cx Present "Present weather" generally describes weather the state of the atmosphere. --If there is a 1 in position 165 (weather indicator): Beaufort Weather Code ----------------------------------- Code Present Weather ----- ---------------------------- B Blue sky C Cloudy sky (detached clouds) D Drizzle F Fog G Gloomy H Hail L Lightning M Mist O Overcast skies P Passing showers Q Squall R Rain (continuous) S Snow T Thunder U Ugly threatening sky V Exceptional visibility W Dew Z Haze The above codes are left justified, and can be combined. Examples: BC = Partly cloudy; pos. 166-167 FDGO = Fog and drizzle, gloomy with overcast skies; pos. 166-169 SPO = Overcast skies with passing snow showers; pos. 166-168 FDGOUM = pos. 166-171 --If there is a 2 in position 165 (weather indicator), the following may be found singly in position 166 or in combination in positions 166-169: Four Choice Code ------------------------- Code Present Weather ---- ------------------ A Fog B Rain C Snow D Hail --If there is a 3 in position 165, present weather was more closely represented by WMO Code 4677 (ww; 00-99) (refer to for details on Code 4677). As many as three different ww codes can be reported in positions 166-171; if one or two codes are reported, they are left- justified with trailing positions left blank. Examples: rain = 60 (pos. 166-167) drizzle = 50 " fog = 40 " rain and fog = 6040 (pos. 166-169) thunder, rain, snow = 176070 (pos. 166-171) *OR* 95 (pos. 166-167) 47 172-173 2 cmvq Magnetic The letters "MV" are placed in pos. variation 172-173 to indicate that the value in quality pos. 46-50 failed the minimum quality control control at the time the observation indicator was digitized. This flag was set if the magnetic variation entered in the log was off by more than ten degrees from Hydrographic Office Chart No. 1706. ------------------------------------------------------------------------------- Table 3. Inventory results for wind direction and force for the three parts of the day, separated into form type 1 (daily) and 2 (multiple reports/day). The totals reflect the total number of form type 1 or 2 data records (i.e, associated with a header record of that type). The nonblank percentage is with respect to the corresponding total, indicating the amount of extant data. ------------------------------------------------------------------------------- direction force =============================================================================== form type: 1 2 1 2 --------------- --------------- total: 855724 546138 855724 546138 --------------------------------------------------- first part of day: cwd1: cwf1: form type: 1 2 1 2 --------------- --------------- patterns: 1685 1166 251 220 blank: 19354 52703 56643 62243 nonblank: 836370 493435 799081 483895 %nonblank: 98% 90% 93% 89% --------------------------------------------------- middle part of day: cwd2: cwf2: form type: 1 2 1 2 --------------- --------------- patterns: 1298 49 263 20 blank: 204231 546043 219991 546040 nonblank: 651493 95 635733 98 %nonblank: 76% <1% 74% <1% --------------------------------------------------- latter part of day: cwd3: cwf3: form type: 1 2 1 2 --------------- --------------- patterns: 1459 48 262 19 blank: 236966 546047 244766 546041 nonblank: 618758 91 610958 97 %nonblank: 72% <1% 71% <1% ------------------------------------------------------------------------------- {References} NCDC (National Climatic Data Center), 1998: The Maury Collection: Global Ship Observations, 1792-1910 (CD-ROM, Version 1.0, February 1998). NCDC, Asheville, NC. Curtis, R.H., 1897: An attempt to determine the velocity equivalents of wind- forces estimated by Beaufort's scale. Quart. J. Roy. Meteor. Soc., 23, 24-61.