ICOADS Web information page (Tuesday, 07-Jan-2014 23:59:15 UTC):
Report-rejection (Hour-missing; Flag ZQ) Problem in Initial Release 1c Data
Figure 1a | Figure 1b | Figure 2a | Figure 2b | Figure 2c | Figure 2d | Figure 2e | Figure 3a | Figure 3b | Figure 3c | Figure 3d | Figure 3e
Release 1c monthly summary statistics (1800-1949) that became available in March 2001 were impacted by a report-rejection problem that significantly reduced numbers of observations and spatial coverage in early decades (especially prior to 1860). This also impacted users of observational data in Long Marine Reports Fixed-length (LMRF) format, or data subsets derived from LMRF, which employed software (trimqc0.f) to select LMRF. The problem and software were corrected, and data reissued, during August 2001 (netCDF files were corrected around September 2001). Summary of products and impacts from the ZQ problem: * LMRF global and subset binary data files (NOT IMPACTED) (caveat: if your software calls subroutine trimqc0.f, the data were IMPACTED; the corrected subroutine is available at icoads.noaa.gov/software/) * ASCII subsets created at NCAR from the LMRF observations (YES IMPACTED) * Binary and ASCII subsets created at NCAR from the MSG statistics (YES IMPACTED) * netCDF files served from PSD (YES IMPACTED). The problem stemmed from usage of the report-status (ZQ) flag to reject reports flagged erroneous by the NCDC-QC. Our intention in checking ZQ was to eliminate reports falling over a 1° latitude x 1° longitude landmask. However, ZQ was also set if hour (or day) was missing. Moreover, the Maury Collection, covering 1784-1863 and providing the bulk of the available Release 1c data prior to about 1854, included many daily reports, in which hour was set to missing. Thus much of the Maury Collection was eliminated through use of the ZQ flag. To correct the problem, we have adjusted the report-selection rules for calculation of monthly summary statistics (and the corresponding trimqc0.f software) to omit the check for the ZQ flag, applicable only to pre-1950 data. Reports are still checked for falling over a 2° latitude x 2° longitude landmask (using the LZ flag). More permissive landmask checks may actually be more appropriate for early historical data due to the more approximate nature of reported ship and port positions. As a consequence, however, it should be noted that allowable spatial coverage rules (for statistics and observations subsetted from LMRF) are different before and after 1950. Figures 1a-1b show total numbers of Long Marine Reports (LMR) output from Release 1c duplicate-elimination processing, per year. Most duplicates were removed from LMR by this processing, but "uncertain" duplicates were retained with flags to indicate their status. As illustrated in Figures 1a-1b, selected uncertain duplicates, plus reports falling over a 2° landmask, were removed to create LMRF from LMR. The remaining data constitute LMRF. Figures 1a-1b further illustrate LMRF rejected from the statistics through use of the ZQ flag (in red), prior to the correction (plus non-ship reports rejected only from the standard statistics). Many other data sources occasionally contain missing hour (or day), but systematically missing hours generally appear to be limited to the Maury Collection. Thus minor data differences may arise through 1949 as a result of the ZQ correction, but the differences are concentrated prior to 1860. Figures 2a-2e illustrate decadal spatial coverage through the 1860-69 decade before and after the ZQ correction, for major observed variables. In the corrected data, wind speed (Figure 2d) obtains the largest and earliest increases in coverage, because many Maury reports were limited to wind observations. Conversely, total cloudiness (Figure 2e) shows the least improvement because the Maury Collection contained little or no usable cloudiness data. Figures 3a-3e show the numbers of observations and 2° boxes per month (1820-1859) included in the enhanced statistics, for each of the same major observed variables as shown in Figs. 2a-2e, before and after the ZQ correction. Again, cloudiness was not impacted by the problem.
U.S. National Oceanic and Atmospheric Administration hosts the icoads website privacy disclaimer
Document maintained by firstname.lastname@example.org
Updated: Jan 7, 2014 23:59:15 UTC