Release 1a (1980-97): Suspicious patterns of cloud fields "00 0    9 0 0" in US Air Force Global Weather Central (GWC; deck 888) data; concentrated in summary statistics for November 1982 through February 1984

Comparisons of deck 888 reports against duplicates from other sources have revealed frequent occurrences during approximately November 1982 through February 1984 where deck 888 reports contain the pattern "00 0    9 0 0" (i.e., N=0, NH=0, CL=0, HI=missing, H=9, CM=0, and CH=0), whereas total cloud (N) values greater than zero appear in other sources (e.g., Table 1).

Another aspect of the problem is that GWC data were heavily selected in Release 1a duplicate elimination (dupelim) processing, in preference to other decks, during this approximate period. We believe that this generally occurred when non-significant weather data values not actually reported over GTS were assigned to deck 888 by the Air Force. In these cases, other data sources lacked weather data, and thus dupelim frequently chose deck 888 over other decks because it had more data elements. This relates to problems associated with the introduction of the station/weather indicator in 1982 (IX; see Table 1).

Figures 1-4 illustrate the evolution and impact of the problem on global statistics of total cloudiness (compared to previously available Release 1a data for 1980-95, which did not include GWC data past 1981). For unknown reasons, unusual concentrations of differences appear in areas such as the Great Lakes (Figures 2-3). Figures 1 and 4 are months bracketing the core of the problem.

Figures 5-8 illustrate the distributions of total cloud values available in the overall output LMRF (all decks), and in the GWC data that were input to dupelim. Figures 7-8 zoom in on the core problem period 1982-84. Figures 9-12 provide maps of the frequency of the suspicious pattern in the input GWC data.

The GWC problems apparently don't go all the way back to the beginning of 1982, but this could be masked by the fact that we have a large gap in the available GWC data (more clearly shown in Figure 8). Also, GWC may have changed procedures at various times, which would require in-depth studies to isolate such changes.

Table 1. Examples of deck 888 reports containing the cloud field pattern "00 0    9 0 0", versus duplicate reports from other selected decks (254=UK MDB, 555=US Navy FNMOC, 892=NCEP, 926=IMMT, 927=US-keyed). Only selected LMR fields are shown.* Total cloud (N) appears under the column marked twice in headings that appear above it with an "x". The station/weather indicator (IX) values in deck 927 appear suspicious: IX=1 should indicate that present weather (WW) was "included," which was the case in only one of these example groupings. In contrast, GWC appears to have inserted arbitrary WW and past weather (W1) values (WW=2, W1=0) in many cases; we assume this was done when IX=2 (weather data omitted; no significant weather to report) was received.** A side effect of this insertion of arbitrary weather codes was that dupelim selected GWC in preference to other decks containing apparently legitimate total cloud values. The truncation of call sign to four characters should be noted in GWC data, which is another inherent deficiency in this source.

--------------------------------------------------------------------------
 Selected location section fields:   Wea:   Cloud fields:    II, ID, etc.
============================================x=============================
   B                 L    L   D S                             IIIIIIII
   1   Y M D   HT    O    AL  C I    WWW     N CH   C C      IDDDDDDDDOOTI
   0   R O Y   RI    N    TI  K D    W12    NH LI H M H      I12345678SP2X
   -   - - -   --    -    --  - -    ---    x- -- - - -      -------------
 170198211 11200028190 4400055583           8                 CG2260
 170198211 11200228190 4400488879    20     00 0  9 0 0       CG22
 170198211 11200028190 4400089229           8   010          0CG2260
 170198211 11200028190 4400092647           8                0CG2260  19 1

 169198211 2   0027940 4460025490   20      8                 CG2240  11
 169198211 2   0027940 4460055583   206     8                 CG2240
 169198211 2   0227940 4460488879   206     00 0  9 0 0       CG22
 169198211 2   0027940 4460089229   206     8   010          0CG2240
 169198211 2   0027940 4460092647   2065    8                0CG2240  19 1

 169198211141800027790 4190055583           7                 WA8463
 169198211141800227790 4190488879    20     00 0  9 0 0       WA84
 169198211141800027790 4190089229           7   010          0WA8463
 169198211141800027790 4190092722           7                0WA8463      1

 169198211171200027270 4530055583           0                 WA8463
 169198211171200227270 4530488879    20     00 0  9 0 0       WA84
 169198211171200027270 4530089229           0   010          0WA8463
 169198211171200027270 4530092722           0                0WA8463      1

 169198211211200027550 4570055583           1                 WA8463
 169198211211200227550 4570488879    20     00 0  9 0 0       WA84
 169198211211200027550 4570089229           1   010          0WA8463
 169198211211200027550 4570092722           1                0WA8463      1

 169198211221800027350 4550055583           2                 WA8463
 169198211221800227350 4550488879    20     00 0  9 0 0       WA84
 169198211221800027350 4550089229           2   010          0WA8463
 169198211221800027350 4550092722           2                0WA8463      1

 169198211271800027760 4330055583           3                 WA8463
 169198211271800227760 4330488879    20     00 0  9 0 0       WA84
 169198211271800027760 4330089229           3   010          0WA8463
 169198211271800027760 4330092722           3                0WA8463
--------------------------------------------------------------------------

----------

Figure captions and links to figures:
Note: The pattern in the figures is represented as "00 0 9 0 0" rather than "00 0    9 0 0" (i.e., omitting the missing HI value).

Figure 1. Map of differences of mean total cloud for 2-deg lat x 2-deg lon boxes: old (1980-95) minus new (1980-97) Release 1a enhanced statistics (the new values were aggregated to 2-deg from 1-deg boxes). This was the month preceding the core of the problem.

Figure 2. As for Figure 1, except November 1982.

Figure 3. As for Figure 1, except November 1983.

Figure 4. As for Figure 1, except March 1984. This was the month succeeding the core of the problem.

Figure 5. Time-series (bar chart) of LMRF (all decks output) showing the frequency of different extant total cloud (N) values. Release 1b data were used for 1973-79, and new Release 1a data for 1980-97 (available GWC data extend back to 1973). The category "0 other" refers to N=0 appearing with subsidiary cloud elements other than those forming the pattern "00 0    9 0 0" (thus the total of N=0 is arrived at by adding the two categories "0 other" and "00 0    9 0 0").

Figure 6. As for Figure 5, but raw GWC data input to dupelim. Gaps in the input GWC data are noticeable in 1974 and 1982, and the category "0 other" is replaced by "00 0    9 0 0" starting in 1982 (coincident with the problems).

Figure 7. Time-series (line plot) of LMRF (all decks output) showing the frequency of different total cloud values (including blank=missing) for a period surrounding the core of the problem (1982-84). Reports with missing N increase towards the end of the period due to increasing numbers of automated platforms (e.g., buoys) that do not report cloudiness.

Figure 8. As for Figure 7, but raw GWC data input to dupelim. The figure clearly delineates the GWC data gap ~June-September 1982, and the apparent peak of the problem in mid-1983.

Figure 9. Map (10-deg x 10-deg) of raw GWC data input to dupelim showing occurrences of the cloud field pattern "00 0    9 0 0" for October 1982. This was the month preceding the core of the problem.

Figure 10. As for Figure 9, but November 1982.

Figure 11. As for Figure 9, but November 1983.

Figure 12. As for Figure 9, but March 1984. This was the month succeeding the core of the problem.

[Documentation and Software][Links to additional]