Impacts of changes in coding and observational procedures

Changes in coding and observational procedures require that data adjustments be made to assure data continuity. Marine weather observations over time have been recorded on a number of different form types as communications technology, the science of meteorology, and subsequent coding practices evolved.

The earliest observations from the East India logbooks, for example, pre-date establishment of the Beaufort Wind Scale and even the invention of the barometer in 1643. This means much of the information from this early period would be incompatible with current codes and methods of measurement. Similarly, the earliest reports in the Maury Collection contain wind direction but no wind speed. Nevertheless, these data sets represent a unique resource of weather information having great value.

Unforseen data continuity problems have been encountered in our endeavor to collect and digitize U.S. merchant (1912-46) and Arctic ice island data. Some problems surfacing in the retrospective data are the result of established observing practices in effect at the time, while others result from observer error due to carelessness or lack of procedural knowledge/training. Date and time are recurring problems throughout the original records because observers often miscoded the date and time when converting from local ship watch to local time to Greenwich time and date. Also, the dry bulb and wet bulb temperatures occasionally seem reversed on the observational form. In most of these cases the data are corrected during the QC process.

Changes in instructions to U.S.-recruited observers generally resulted in the issuance of a new edition of the standard Weather Bureau Instructions to the Marine Meteorological Observer, which later became known as Circular M (for "marine handbook"). Table 2 gives examples of such changes encompassing the period of the 1912-46 merchant data. We believe the instructions prior to the first edition in 1906 were typically attached to the logbooks. The instructions remained fairly consistent from 1906 until the introduction of major international code changes in 1949. Even the international code changes of 1929--when great progress was made in standardizing methods of reporting weather observations, especially by radio--did not substantially alter U.S. coding practices.

Unfortunately, observers frequently did not adhere to the observing instructions. One example of a problem associated with the 1912-46 merchant data is that wind directions were sometimes not coded in accordance with the U.S. instructions (32-point scale), occasionally appearing as 3 digits (e.g. 240). It is interesting to note that in the instructions in the 1938 and 1941 editions, the Weather Bureau broke somewhat with tradition by allowing the winds to be reported on the marine weather log form (1210A) in two acceptable codes:

"The direction of the wind may be entered in the appropriate column on Form 1210A either directly in terms of compass points or in code, according to the scale 01-32, in which 08=E, 16=S, etc. However, inasmuch as the wind direction must be coded in figures whenever an observation is transmitted by radio, it is customary for most observers to enter the code number and this procedure is preferred by the Weather Bureau. Therefore, observers who have been accustomed to recording the wind direction directly in terms of compass points are urged to make a practice of using the numeric scale instead."

This practice required that different form types be developed before keying the data, but did not intrinsically lead to any data biases in the wind directions. However, this does possibly lead to biases in other observed elements such as cloud amounts by inadvertently establishing a precedent of substituting the radio code for the established code to be used on the observational form.

For example, the "total cloud amount" entered on the logbook forms appears to have come from two different codes, one developed for the log form entry in tenths (0-10) and the other a single digit code (0-9) adopted for radio transmission. Apparently, the observers, for convenience, often used the radio code instead of the form code to make their entry on the log form. Since there is generally no way of distinguishing which code was used, this innocent practice introduced an observational bias (e.g., overcast skies coded as 10 in the form code and as 8 in the radio code) that can only be corrected statistically after making certain assumptions regarding overall cloud distributions (Table 3).

Careful checks will be required for conversion of digitized sea level pressure values into a common COADS format because of the many changes in instrumental or reporting procedures. In a number of cases all the necessary information needed to make these checks is not available on the original form. Before the time of radio transmissions, observers using mercurial barometers were instructed not to make the corrections for temperature and gravity, but simply enter the value as read; the Weather Bureau made the necessary corrections upon receiving the forms.

Some of the early aneroid barometers issued did have an attached thermometer for convenience in reading the ambient air temperature (dry bulb). It is difficult to imagine, however, that the mariner would have mounted the valuable barometer in the open air exposed to all the weather elements and not in the protection of the cabin. There is also a large number of attached thermometer entries indicated to have come from an aneroid barometer that have the same value as the dry bulb temperature. This can be corrected by careful QC.

Many of the U.S. merchant marine logbook forms located in the archives for the period 1912-46 contain only one observation per day at 0000 UTC. This is in contrast to both the mid- to late 1800s when U.S. observations were reported every two hours and to contemporary international ship observations, which are generally reported every six hours (0000, 0600, 1200, 1800 UTC).

Starting sometime before 1906, the Weather Bureau required that radio reports be sent twice or even four times a day but requested only the 0000 UTC observation be sent by mail. The logic behind the practice of reporting only the 0000 UTC observation on the forms is explained in the 1906 edition. The Weather Bureau felt that with the advent of weather forecasting as a science, mariners could best determine which route to take based on conditions actually encountered (referencing the daily synoptic charts) rather than on average conditions.

The 0000 UTC observational practice can bias the digital database, however. For example, at certain longitudes all reports are observed near the average diurnal cycle for maximum heating, while at other longitudes reports are observed near maximum cooling, with reports from intervening longitudes falling somewhere in between. These biases can be statistically corrected.

One partial solution to correcting the bias of once-daily logbook observations would be to supplement them with radio messages. However, few of these radio messages (often garbled) are available in their original format today. Reasons for this include:

• lack of storage space,
• lack of economical or viable technology at the time for archiving the information on film or digital media, and
• deterioration over time of the teletype paper.

A large number of these messages were plotted on the Northern Hemisphere Charts and are available dating back to 1899. These observations would be very time consuming and expensive to digitize, and a number of the elements would have to be estimated because of the coarseness of the plotting code. For example, wind speeds were plotted only to the nearest 5 knots, wind direction would have to be based on direction of the plotted wind shaft, and ship position estimated from the location of the plotted station model.