Comparisons of Real-time Surface Marine Data from the NOAA/NWS National Centers for Environmental Prediction (NCEP) and the US Navy Fleet Numerical Meteorology and Oceanography Center (FNMOC) for March 1997

1. Background

As a near-real-time extension of the International Comprehensive Ocean-Atmosphere Data Set (ICOADS), the NOAA/ESRL Physical Sciences Division receives monthly files of surface marine data gathered from the Global Telecommunications System (GTS) from NCEP, and previously received monthly data from FNMOC. The NCEP data are provided courtesy of Diane Stokes at NCEP, and the FNMOC data were provided courtesy of Roy Mendelssohn and Lynn deWitt at the NOAA/NMFS Pacific Environmental Fisheries Laboratory (PFEL). These are basic observational data (individual marine reports from ships, buoys, etc.).

A major change in processing became effective in January 1997 at FNMOC, and in March 1997 at NCEP, when the operational centers transitioned to the BUFR format. Data from NCEP were previously available in Office Note 124 (ON124) format, and data from FNMOC were previously available in "Surface Ship" format. NCEP now provides a simple ASCII format (containing selected weather elements) as derived from the full BUFR data, and formerly provided the simple ASCII format as derived from ON124. Both the NCEP data sources have been subjected to "dup-merge" processing at NCEP that may combine partially complete reports into a single report, or modify or eliminate data elements through quality controls. The level of QC applied by FNMOC, if any, is unknown.

2. March 1997 Comparison Plots

For March 1997, overlapping ASCII files were available from NCEP (derived from BUFR), NCEP (derived from ON124), and FNMOC (derived from BUFR). The following figures provide examples of data continuity problems and differences apparently introduced into the data by storage in BUFR format; section 3 is a discussion of the differences. It should be noted that each file has been subjected to a simple (exact) duplicate elimination procedure (internal to each data source).

Figures 1-2:

Figure 1: Percentage of Sea surface temperature (SST) observations per (signed) tenths position. NOTE: Positive and negative are treated separately (each adding up to 100%). (Missing data are not included.)

Figure 2: Similarly to Figure 1, for air temperature (AT).

Figures 3-4:

Figure 3: Similarly to Figure 1, for dew point temperature (DPT).

Figure 4: Similarly to Figure 1, for dew point depression (DPD).

Figures 5-6:

Figure 5: Percentage of wind observations per different combinations of speed and direction (999 indicates missing). Blank for speed indicates that the value to the left is repeated, i.e., following is a full list of the combinations being compared (999 direction with 999 speed, etc.):

direction	999	0	>0	999	0	>0	999	0	>0
speed	999	999	999	0	0	0	>0	>0	>0

Figure 6: Percentage of cloud observations per octa (missing data are not included).

Figure 7:

This is a set of figures showing the total number of observations per data element (stratified similarly to the previous figures), instead of percentages, for the NCEP BUFR and ON124 data (FNMOC observation counts are not shown). The organization of the panels is as follows:

SST negative tenths		SST positive tenths
AT negative tenths		AT positive tenths
DPD negative tenths		DPD positive tenths
DPT negative tenths		DPT positive tenths
wind speed and direction combinations
cloud octas

Figure 8:

Figure showing the total number of observations per hour.

3. Discussion

a) SST and AT: NCEP's use of a conversion factor of 273.15 for conversion to Kelvin, and rounding to tenths Kelvin precision for storage in BUFR (the maximum precision available at that time in NCEP's version of BUFR), has apparently lead to original positive Celsius temperatures with tenths positions of ".3" and ".8" being transformed into the same numeric representation as those ending in ".2" and ".7" (a negative 0.1 bias). This problem is clearly evident in AT, but less so in SST.

The question has not been resolved why SST and AT have different behavior, and whether this difference could possibly have been introduced by the conversion from BUFR to ASCII. FNMOC has used a different conversion constant (273.2), thus it agrees well with the ON124 benchmark data. (There were also a very small number of NCEP BUFR AT values ending in ".3" and ".8", also presently unexplained.)

Similarly, negative temperatures (of which there are generally fewer, and very few for SST; ref. Figure 7) have ".2" and ".7" transformed into ".3" and ".8" also indicating a negative 0.1 bias.

b) DPT and DPD: The ASCII format contains DPD, constructed by subtracting AT - DPT. DPT has been reconstructed by DPT = AT - DPD, thus it may be impacted by Kelvin conversion problems in both AT and DPT. The same negative 0.1 bias problems that appear in AT are clearly evident in both positive and negative DPT.

NOTE: Storage of all these temperature values in BUFR was extended to hundredths Kelvin at NCEP to resolve these problems starting approximately 17 February 1999.

c) Wind data: We have concentrated our examination on the differences between the datasets represented in the smaller bars (other than both direction and speed missing, or both extant), which illustrate very different conventions used in the BUFR data at FNMOC versus NCEP to store calm and variable winds. Similarly to cloud data (discussed below), BUFR does not fully support the originally reported distinction of a "variable" wind direction. If, for example, an observer reports a non-standard data combination such as variable direction with missing speed, there is no convention in BUFR to archive that data combination. Variable winds were intended by WMO to be represented as speed >0 and direction 0 (FNMOC uses that convention; NCEP does not).

NOTE: The wind speed indicator, showing the units in which and the method by which wind speed was originally recorded (WMO code 1855), is always missing in NCEP's BUFR data until approximately 21 October 1997. For data decoded into BUFR starting at that time, the BUFR format at NCEP was expanded to include the wind speed indicator information, if available in the original GTS report.

d) Cloudiness data: There appears to be reasonably good agreement between the various sources. However, it should be noted that BUFR does not support the distinction between code figures "9" (sky obscured by fog, snow, or other meteorological phenomena), "/" (cloud cover indiscernible for reasons other than code figure 9, or observation is not made), and missing. [NOTE: Details need to be confirmed at NCEP.]

e) Hour: More FNMOC data appear at the main synoptic hours (00, 06, 12, 18), to some extent (at least in comparison to ON124) than at the intermediate synoptic hours (03, 09, 15, 21). These additional reports may be due to Navy shipping receipts not available at NCEP. Conversely, both NCEP sources appear to provide some additional (possibly buoy or Coastal-Marine Automated Network) data at off-synoptic hours.

NOTE: These tests were completed in 1997, and this page reflects results available at that time, with a few minor updates. Since 1997, changes were made in NCEP's storage of data in BUFR to resolve some problems (e.g., the temperature and wind speed indicator problems noted above). Further tests would be required to determine whether the above problems in wind and cloudiness data have been resolved. In addition, tests would be advisable to ensure that the full resolution and all defined data configurations of all marine data elements available in the GTS SHIP (FM13) and BUOY (FM18) codes are adequately preserved in BUFR.

[Documentation and Software][Links to additional]