ICOADS Web information page (Tuesday, 17-May-2011 22:02:20 UTC):

Release 2.5 1970-2007 Outstanding Data Questions

Note: This page represents work in progress. More detailed discussion will be added at a later date (tables defining deck and source ID are available here).


Figure 1a. Mislocated S Hemisphere duplicates discovered by Jack Woollen. For September 1983-January 1984 any report from deck 926 in the Southern Hemisphere containing an ID with first two characters "TF" was deleted.


Figure 1b. Pacific ship track moved 100 degrees east. For December 1983 any report from deck 732 containing an ID "TF" was deleted.

Table 1. Input numbers of platform type (PT) moored buoy (PT=6), drifting buoy (PT=7), and Coastal-Marine Automated Network (C-MAN; PT=13) reports for 1970-79, stratified by deck and source ID (DCK SID). One noteworthy data feature is grey shaded.


Table 2. As for Table 1, except for 1980-97.


Table 3a. As for Table 1, except for 1998-2004.


Table 3b. As for Table 1, except for 2005-2007.



Figure 2. Transient duplicate match information for 2000-04 (numbers of reports). Colors indicate match dup status (DUPS, ref. Fig. 3). On the abscissa, the lower numbers label the number of weather elements in common, and the upper numbers embedded within those categories label the number different. Noteworthy feature: the red bars e.g. indicate DUPS=6 (time/space only match). (Note: This is referred to as "transient" information because the final output DUPS can be changed through subsequent matches with all possible duplicates.)


Figure 3. Transient duplicate match information (ref. Fig. 2) for 2000-04 (percentage of reports) plotted for selected deck-to-deck matches. Noteworthly feature: the large percentage of DUPS=11 (time/space/ID only matches) for 792 vs. 874 (NCEP GTS ship vs. SEAS) matches (see also Example 6 below).

Example 1. Transient matches between decks 792 vs. 926 (NCEP FM 13 ship vs. IMMT). Noteworthy differences: ID in the first two cases; SST sign in the last two cases; cloud elements. (Note: these are random examples of deck-to-deck differences; whether any of them represent systematic patterns would require further detailed investigation. Field abbreviations, listed vertically, are defined here.)
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=3 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200111 1   0 6080   3603NYT     340170******101522 43  56**** -20****7************************************00  ****106792103 5101*0 101************4
200111 1   0 6080   3601183FYT  34017098 22*101522 43  56**** -20****7**3**** 710*************************00NO**25106926112 5 11** 00400*0 20****14
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=7 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200111 1   0 6530   730LFN       30 5797808210065****  48****   0  906634AA*******************************00  ****106792103 5101*0 004**** 1*****14
200111 1   0 6530   730121LF3N   30 5797808*100652 34  48****   0  98663400*******************************00NO**25106926112 5 10** 00400*0 70****14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=3 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
199811 1   0 7390  2250UCTZ     361200******102350 76************  501**8*********************************32  **** 72792100 511000 101*0** 1*******
199811 1   0 7390  2250UCTZ     140 6098 22*10235****  20********  5099***********************************32RU**23 72926112 5 10** 00100** 70****1*
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=7 DIFFER=5
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 2 1   0 6020 35400UCUF     32015097 222100004 10  20********  817745AA*******************************61  ****105792103 5111*0 101**** 1*****14
2003 2 1   0 6020 35400UCUF     30024097258*10023****  20********  88779500*******************************24RU**13105926112 5 11** 00100** 70****1*
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
                                                                                                                              DD        WD
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 5 1   0 7480 35990DBLK      1013097****101412  6 -41**** -84  -9***5*********************************00  **** 69792103 5 11*0 101***********14
2003 5 1   0 7480 35990DBLK      1013097****101412  6 -41  55 -84   9***5*********************************00DE**14 69926112 5121** 0010030 70****44
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
                                                                                                                              DD        WD
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2004 8 1   0 8260 35270DBLK     20011093**** 99903  1  -8****  -9 -18***1*********************************61  **** 33792103 5121*0 101***********14
2004 8 1   0 8260 35270DBLK     20011093**** 99903  1  -8  -8  -9  18***1*********************************61DE**14 33926112 5 11** 0010010 70****44
Example 2. Transient matches between decks 792 vs. 927 (NCEP FM 13 ship vs. US-keyed). Noteworthy differences: ID differences, cloud elements.
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=5 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2001 1 1   0 4400 18720DWCBPD   25011398**** 99467 23  70  40   0  6076457A** 5 525 6 6*******************63  ****160792103 5100*0 104**0* 1*****34
2001 1 1   0 4400 18720WCBP     25011398**** 99467 23  70  40   0  6076457A** 5 525 6 6**********AAAAA****63US****160927113 5 1000 104*00* 1*****34
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=5 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 1 1   0-1640 19710DELYJ9D   10 7798 211*****5 10 300 260 250****332420** 6 3 4 6 3*******************54  ****377792103 5101*0 104**0********14
2003 1 1   0-1640 19710ELYJ9     10 7798 211100905 10 300 260 250****332420** 6** 4 6 3**********AAAAA****54US****377927113 513100 104*00********14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=2 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200210 1 300 4440 27720VCBZ     190129*************** 240************4**9**** 3 2*************************83  ****169792103 5111*0 104************1
200210 1 300 4440 27720VCBZ     19012898 212********* 240************4*AAAA** 3 2****************AAAAA****83CA****169927113 5 1100 104*0*********14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=7 DIFFER=6
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2001 3 1 300 4290 27760CG2960   210 2697 122100887 10  -2  -2  -2  10666500*******************************00  ****169792103 5 11*0 104**2* 0*****14
2001 3 1 300 4290 27760CG2960   280 7298 322101557 10 -36 -45  -7   0886400** 0 0****************AAAAA****00CA****169927113 511100 104*00* 0*****14
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
1998 7 41200 1390 290308PAO      90 6298****101144  0 286******** 283522342** 7 3 8 7 411 4 2*************13  ****279792100 513000 104*0** 1*****24
1998 7 41200 1390 290308PAO      90 62981149101144  0 286 275 270  15541362** 9 2 1 9 2** 6 1****AAAAA****43CA****279927113 5 1100 104*30* 1*****24
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=6 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200412 1   0 1140 28440C6T2063   5012998 110100801  5 275 235 210****2*0909**** 6 5** 5*******************14  ****278792103 5 11*0 103**0********14
200412 1   0 1140 28440C6T2063   5012898 110100801  5 275 235 210****200909****** 5**************AAAAA****14US****278927113 512100 103*00********14
Example 3. Transient matches between decks 926 vs. 927 (IMMT vs. US-keyed). Noteworthy difference: ID.
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=4 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200210 1   0 3710 29340WCZ553    80 9898****102214  0 225 175 140****44*6*0** 4 236 6 2*******************24US**42207926112 5 12** 0040200**0*****4
200210 1   0 3710 29340WCZ5535   80 9898****102214  0 225 175 140****44A6AA** 4**36 6 2**********AAAAA****24US****207927113 510200 104*00********34
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=7 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 5 1   0 4560 27480WZP816    40 4698505*10125****  44********   38******* 1 1*************************23US**46169926112 5 12** 00402** 10****14
2003 5 1   0 4560 27480WZP8164   40 4698505210125****  44********   38*AAAA** 1******************AAAAA****23US****169927113 510200 104*0** 1*****14
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
1994 627   0 1750 11660ELLO     100 60** 21210105**** 295************6**9***********************************25 323261926 72 512100 101*0****0******
1994 627   0 1750 11660ELLO     100 6798 21210105**** 295************6*AAAA**********************AAAAA******CA****261927113 5 1100 104*0*********24
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=7 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2002 8 1   0 4220 15590OWZU2    17012998 12*102101 10 180 160 150 1578085*0** 3 1 8 4 3*******************15US**41157926112 5122** 0040000 10****14
2002 8 1   0 4220 15590OWZU2    17012898 122102101 10 180 160 150 1578085AA** 3 1 8 4 3**********AAAAA****15US****157927113 5 1200 104*00* 1*****14
Example 4. Transient matches between decks 700 vs. 926 (VOSClim BUFR vs. IMMT). Noteworthy differences: Wind speed and visibility differences; omission of wave/swell elements from BUFR (or not decoded into IMMA).
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=6 DIFFER=2
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 424   0-2730 11360V2FM     140 4197152*10168**** 240 232 228****852230*******************************33  ****405700110 513100 103**0**********
2003 424   0-2730 11360V2FM     140 4096152*101683  6 240 232 228****852230** 5 122 9 4*******************33AU**21405926112 5111** 0000001 00****14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=6 DIFFER=2
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 6 1   0 1590 12500MZHC8    21012998 12*10079**** 281 256 245****711406*******************************34  ****262700110 513100 103**0**********
2003 6 1   0 1590 12500MZHC8    21013097 12*10079**** 281 256 245****711406** 5 314 9 120 6 2*************34AU**21262926112 5111** 0000001 10****14
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=7 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200412 1   0 3200 18130MXBC6    18011098 22*10130**** 224 200 186 202630774*******************************65  ****196700110 512200 100**0**********
200412 1   0 3200 18130MXBC6    18010098 22*101308 20 224 200 186 202630774** 3 329 9 6*******************65GB**41196926112 5 12** 0000000 10****14
 MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=7 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200412 1   0-2550 10540C6HS4    120 8297582*10170**** 182 172 165 220877220*******************************33  ****404700110 513000 103**0* 3*******
200412 1   0-2550 10540C6HS4    120 8296582*101703 10 182 172 165 220877220** 5 312 6 420 5 3*************33AU**21404926112 5120** 0030001 10****14
Example 5. Transient matches between decks 793 vs. 926 (NCEP FM 13 buoy vs. IMMT). Noteworthy differences: PT. Deck 793 PT=missing because these numbers are not legal 5-digit moored buoy numbers. Deck 926 was assigned PT=5 without regard to the IMMT Observation platform field (more work is needed in this area as IMMT can contain data from non-ship platform types).
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=6 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2000 8 1 600 5690   18062136    180 72961022101377 10 145 142 140************ 2 1***************************  ****142793103 6101*0 304**0********14
2000 8 1 600 5690   18061236    180 7296102*101377 10 145 142 140****7763**** 2 1*************************00GB**41142926112 5 11** 0040000**0****14
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=6 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2001 2 1   0-3840  101030102    290103955022100907 20 170 168 170*******************************************  ****467793103**101*0 003**0********14
2001 2 1   0-3840  1010ELBX4    29010395502*100907 20 170**** 160****8874**** 5 4*************************23IL**13467926112 5 11** 00300*0 10****1*
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=6 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2002 3 1 600 1280  507003       110 4198 200101201 10 280 220 193************ 4 1***************************  ****255793103**101*0 004**0********14
2002 3 1 600 1280  5070DICB     110 4198 20*10120**** 280 220 194****422340** 4 1*************************64DE**11255926112 5 11** 0040001**0****14
 MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE      COMMON=6 DIFFER=0
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2003 4 1   0 4590 274607620     110 2697 22*10148****  17********************** 0***************************  ****169793103**101*0 004***********14
2003 4 1   0 4590 27460WCV762   110 2697 22*10148****  17************8********* 0*************************00US**46169926112 5 11** 00402****0****14
Example 6. Transient matches between decks 792 vs. 874 (NCEP FM 13 ship vs. SEAS). Noteworthy differences: While these appear to be legitimate matches, many weather elements are different (e.g., large wind speed differences) or missing in SEAS. The SEAS data may need to be more carefully validated (see also Fig. 3).
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=1 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2002 9 11200 3850 14450OULL2    150 8295****101757  3 250******** 258***A*********************************55  ****192792103 5 11*0 104**** 1*****34
2002 9 11200 3850 14450OULL2    150285*********************** 175*****22A55**********************AAAAA****58  ****192874111 511100 304***********14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=1 DIFFER=1
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200210 8 600 6150 19120WBO3345  330 82**8172101292 19  25************8*AAAA** 3 435***********************82  **** 89792103 5 11*0 104***********14
200210 8 600 6150 19120WBO3345  330128***********8172******** 129 304822A82**********************AAAAA******  **** 89874111 511100 304**** 1*****14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=7 DIFFER=6
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
2002 5 3   0-3550 17880P3RD8    25015496 111101350  5 180 150 130 195442500** 6 625 6 7*******************54  ****447792103 5 11*0 104**0* 1*****14
2002 5 3   0-3550 17880P3RD8    250925964200100050111 130**** 135 606422554****** 6**** 7*****150AAAAA****95  ****447874111 511100 304*3** 1*****14
 MATCH=11 CERTAIN: TIME/SPACE/ID                                COMMON=7 DIFFER=6
                                                                                                                              DD        WD         
                L     LIIIIIIIII                S   P       W   D   S                   S S S    IIIII  R           B  D  S   UU        BP         
   Y M D   H    A     ODDDDDDDDD       V WWW    L   P   A   B   P   S NC CC W W W S S S D P HI ERCCCCC  RTDV C COO  1  C  I P PPTL IDWVITT SHTPWSII
   R O Y   R    T     N123456789  D  W V W12    PA  P   T   T   T   TNHLHMH D P H D P H 2 2 2S SS12345  RRSS 1 2SP  0  K  D T SCII IIIITII IICBXXXR
200311 51800 4450 30390WCZ5528  320 6798 222103050  5 128 100  80  5066146A** 4 235 5 4*******************53  ****172792103 5 11*0 103**0* 1*****14
200311 51800 4450 30390WCZ5528  32065898616*100050222  75**** 305 402622453****** 5**** 4*****100AAAAA****50  ****172874111 511100 303*0** 1*****14

Figure 4. Deck 144 (TAO/TRITON and PIRATA Buoys) raw output as flagged by DUPS (numbers of reports). Noteworthy feature: Many data from the different SIDs appear to be unique (DUPS=0). SIDs 86 and 87 were deleted during final duplicate elimination preconditioning.


Figure 5. As for Figure 4, except for Deck 714 (Canadian Integrated Science Data Mgmt. Buoys). Noteworthy feature: SID 61 was often the best duplicate (DUPS=1) instead of SID 63. SIDs 61 and 62 were deleted during final duplicate elimination preconditioning.


Figure 6. As for Figure 4, except for Deck 780 (NODC/OCL World Ocean Database). Noteworthy feature: SIDs 88 and 89 were almost entirely worst duplicates although several DUPS=6 (time/space only matches). SIDs 88 and 89 were deleted during final duplicate elimination preconditioning.


Figure 7. As for Figure 4, except for Deck 883 (US National Data Buoy Center). Noteworthy feature: Excellent matching and selection as best duplicate (DUPS=1) of SID 122 vs. SIDs 50 and 121 except for missing SID 122 data in 1984, 1990-91. SID 122 was deleted during final duplicate elimination preconditioning.


Figure 8. As for Figure 4, except for Deck 667 (IATTC). Noteworthy feature: Although previously thought to be unique SID 71 sometimes duplicated itself. Any deck 667 report exactly identical to any other deck 667 report was deleted (see also Table 4).

Table 4. Deck 667 (IATTC). Noteworthy feature: 5,640,769 matches and only 28,442 duplicates. The uncommon disparity between the number of transient matches and total duplicates in 1980, 1984, and 1988 was indicative of multiple copies of single reports, while most of 1989 was duplicated (see also Fig. 8).




Figure 9. Deck 714 (ISDM drifters) showing reports with II=0 (ID present, but unknown type), PT=missing (see also Table 5).

Table 5. Deck 714 (ISDM drifters) listing: Left: the numbers with II=0 (see Fig. 9) or II=3 (WMO 5-digit buoy number) per year and SID; Right: illegal or unrecognizable buoy IDs associated with PT=missing (and II=0).



Figure 10. Deck 714 (ISDM drifters) comparing the QC flag settings (percentage of input reports) for position (1=good, 3=doubtful) between the earlier data versions (SIDs=61-62) and the latest data version (SID=63, used for Release 2.5). This plot is included for reference to show that an earlier QC problem was resolved (excessive flagging of position 1993-97, which lead to the flag data being "included," as opposed to "excluded" during the latest processing).

Table 6. Final Release 2.5 platform type (PT) versus sea surface temperature measurement method (SI) (blank for PT or SI signifies missing). A few suspicious, or noteworthy, patterns are highlighted. For the oceanographic platform types (PT= 10, 11, 12, 17, 18, 19, 20, 21), it should be noted that SI values such as bucket or intake not expected from the profile instrument types appear when the profile temperature value is unavailable or unusable, and a reference SST value is used instead.



Figure 11. Time evolution of the sea surface temperature measurement method indicator (SI) (see Table 8) per NCEP decks 792 (FM 13 ship), 793 (FM 13 buoy), 794 (FM 18 buoy), and 795 (C-MAN).

Table 7. NOAA/NCEP and Release 2.5 processing of the sea surface temperature measurement method indicator (SI) from original GTS messages (FM 13), into NCEP's BUFR format, and of FM 13 or NCEP BUFR data into IMMA format, for 1998-2007. Colon (":") signifies a translation between the different coding systems (ref. Table 8 below). Thus before 24 July 2002 NCEP incorrectly used the FM 13 SST indicator, instead of the BUFR equivalent. For ICOADS however, we were aware of the problem and corrected for it.
SI
  before 24 July 2002 starting on 24 July 2002
BUFR FM 13: FM 13 (incorrect)FM 13: BUFR
IMMA FM 13: IMMA BUFR: IMMA


Table 8. For reference, sea surface temperature measurement method indicator (SI) values currently available in four different observations formats (BUFR, IMMA0, IMMT-3, and FM 13), plus in the WMO Pub. 47 metadata format.
BUFR IMMA0 IMMT-3 FM 13 WMO Pub. 47
0 Ship intake 0 bucket 0 Bucket thermometer 0 Positive or 0 IntakeBTT Bait tanks thermometer
1 Bucket 1 condenser inlet (intake) 1 Condenser inlet 1 Negative IntakeBU Bucket thermometer
2 Hull contact sensor 2 trailing thermistor 2 Trailing thermistor 2 Positive or 0 BucketC Thermometer in condenser intake on steam ships, or inlet engine cooling system on motor ships
3 Reversing Thermometer 3 hull contact sensor 3 Hull contact sensor 3 Negative BucketHC Hull contact sensor
4 STD/CTD sensor 4 through hull sensor 4 "Through hull" sensor 4 Positive or 0 Hull contact sensorHT "Through hull" sensor
5 Mechanical BT 5 radiation thermometer 5 Radiation thermometer 5 Negative Hull contact sensorOT Other
6 Expendable BT 6 bait tanks thermometer 6 Bait tanks thermometer 6 Positive or 0 OtherRAD Radiation thermometer
7 Digital BT 7 others 7 Others 7 Negative OtherTT Trailing thermistor
8 Thermistor chain 8 unknown (applicable only to decks 705-707)
9 Infra-red scanner 9 unknown or non-bucket
10 Micro-wave scanner10 implied bucket (an HSST SID or any match)
11 Infrared radiometer11 reversing thermometer or mechanical sensor
12 In line thermosalinograph12 electronic sensor
13 Towed body
14 Other
15 Missing value


Table 9. A small number (16) of Russian Marine Met. Data Set (MORMET; deck 732) examples were provided by Steve McIntyre (30 August 2010) from January 1973 near the equator in the west Pacific, where negative SST and AT are obviously unphysical, and where he concluded that the problems appeared likely to have arisen from mislocated reports (incorrect latitude/longitude). The data examined by Mr. McIntyre were from the previous Release 2.4. In this table instead we list the "Intermediate" output from the latest R2.5, i.e. containing flagged duplicates with a new data source included for R2.5: Russian Research Vessel (R/V) Digitization (deck 735). (Field abbreviations, listed vertically, are defined here.) Only four of the 16 MORMET reports survived duplicate elimination (see field DUPS, where e.g. 6 = worse duplicate: time/space match with ID mismatch) and based on comparison with the R/V data they do not appear to be spatially mislocated. Moreover the spurious SST and AT observations are all flagged by the trimming flags (SF and AF) as extreme (6 = less than 4.5 sigma lower limit). Additionally, wind speeds in MORMET also appear to be halved versus the R/V data (possible m/s to knots units translation problems), whereas locations and most other listed meteorological data elements (i.e. outside of temperatures and winds) between the two sources appear to match reasonably well. Figure 12 illustrates the locations of the MORMET (deck 732) data for January 1973, with colors indicating the SST trimming flag (SF) values. Figure 13 contrasts the data locations and SF values for all other decks. Figure 14 illustrates the MORMET locations, with colors indicating the wind trimming flag (UF and VF) values (note: the higher value of UF or VF is plotted, as that is how the data are trimmed). Figure 15 illustrates the R/V locations, with colors indicating the SST trimming flag (SF) values.
                         A                                               W    D                                      DD      
                L     L  T     N                               S   P     B   WP   D     S                     D  S   UUSS  AS
   Y M D   H    A     O ITTLDV I I        I CD   W   V V WW    L   PI   AT   BT   P S   S NCH CC W W W S S S  C  I P PPQQSANN
   R O Y   R    T     N MCIISS D I        D 1I  DI  WI V W1    PA  PT   TI   TI   T I   TNHLIHMH D P H D P H  K  D T SCZAFFCC
1973 1 52100 -600 16500 0300                 0220  6009715 10093    0 -21   35  -40   -197790500   6 5 411 8732 57 5 62156699
1973 1 52100 -600 16500 0200    10UMAY       0200111009715 10093    0 282       248 0 2857790500   6 5 411 8735 64 5 12  1111
1973 1 6   0 -660 16500 0200    10UMAY       02501 7009715 10094    0 272       244 0 2867690560   4 3 411 8735 64 5 12  1111
1973 1 6   0 -660 16500 0300                 0250  4009715810094    0 -27   38  -42   -197590560   4 3 411 8732 57 5 62156699
1973 1 6 900 -810 16450 0300                 0250  20098 2 10089    0 -24   41  -47   -166320501     1     9732 57 5 62156699
1973 1 6 900 -810 16450 0200    10UMAY       02501 30098 2 10089    0 276       236 0 2926320501     135   9735 64 5 12  1111
1973 1 61500 -950 16400 0300                 0220  30098 2 10083    0 -20   48  -58   -155210601     2     8732 57 5 62146699
1973 1 61500 -950 16400 0200    10UMAY       02001 60098 2 10083    0 284       215 0 2935210601     235   8735 64 5 12  1111
1973 1131200 -730 15630 0300                 0361   0098 2010110    0 -14             -1499        0 0     0732 57 5 0 146699
1973 1172100 -910 16500 0300                 0360  10098 1 10105    0 -20   32  -36   -123320600   0 0   6 2732 57 5 62146699
1973 1172100 -910 16500 0200    10UMAY       03601 10098 1 10105    0 284       255 0 2983320600   0 013 6 2735 64 5 12  1111
1973 118 300 -800 16500 0300                 0200  10099 3 10081    0  -9   29  -36    -86610600     0   9 4732 57 5 62156699
1973 118 300 -800 16500 0200    10UMAY       01801 10099 3 10081    0 304       255 0 3066610600     0 5 9 4735 64 5 12  1111
1973 118 900 -680 16500 0300                 0130  1009715 10094    0 -21   31  -34    -76430501     0 4   4732 57 5 62156699
1973 118 900 -680 16500 0200    10UMAY       01201 2009715 10094    0 282       259 0 3076430501     0 4   4735 64 5 12  1111
1973 1181500 -560 16500 0300                 0350  20098 2 10081    0 -19   33  -38   -105520500     1 4   4732 57 5 62156699
1973 1181500 -560 16500 0200    10UMAY       03501 40098 2 10081    0 286       252 0 3025520500     1 4   4735 64 5 12  1111
1973 125   0 -800 15580 0300                 0 40  20097 2110070    0 -17              -67  03 9   3 1 0   0732 57 5 0 146699
1973 1251500 -590 16480 0300                 0330  30098 1 10059    0 -21   36  -42   -132110640     2     3732 57 5 0 156699
1973 1252100 -700 16470 0300                 0360  3009715 10066    0 -19   29  -33   -147630504   3 2   8 4732 57 5 62156699
1973 1252100 -700 16470 0200    10UMAY       03201 6009715 10066    0 286       261 0 2957630504   3 236 8 4735 64 5 12  1111
1973 126 300 -820 16430 0300                 0330  30098 2 10045    0 -15   28  -32   -107420602   3 2   8 3732 57 5 62146699
1973 126 300 -820 16430 0200    10UMAY       03001 50098 2 10045    0 293       263 0 3027420602   3 236 8 3735 64 5 12  1111
1973 126 900 -940 16400 0300                 0 60  2009725 10061    0 -23   31  -33   -117530504     1     3732 57 5 62146699
1973 126 900 -940 16400 0200    10UMAY       0 601 4009725 10067    0 278       260 0 3017530504     136   3735 64 5 12  1111
1973 1261500 -950 16400 0300                 0 90  4009715 10041    0 -23   28  -29   -126530504     3     4732 57 5 62146699
1973 1261500 -950 16400 0200    10UMAY       0 901 8009715 10041    0 279       267 0 2996530504     336   4735 64 5 12  1111
1973 127 600 -900 15570 0300                 0170  9009721610026    0 -21             -11887052    4 2 0   0732 57 5 0 146699



Figure 12. Locations of MORMET data for Jan. 1973, with colors indicating the SST trimming flag values: gray SF=1; green SF=2,3; blue SF=4,5; red SF=6,7; cyan SF=C. Following is an explanation of the trimming flags shown (see sec. 6 here for further information):
1 = within 2.8 sigma limits
2 = less than 2.8 sigma lower limit
3 = greater than 2.8 sigma upper limit
4 = less than 3.5 sigma lower limit
5 = greater than 3.5 sigma upper limit
6 = less than 4.5 sigma lower limit
7 = greater than 4.5 sigma upper limit
C = limits missing (ocean/coastal box)
A box (covering -9.5 to -5.6 latitude, and 155.7 to 165 longitude) has been inserted to mark the approximate region of the original 16 MORMET reports. Note suspicious data boxes, possibly indicating large blocks of mislocated data.


Figure 13. Locations of all non-MORMET decks for Jan. 1973, with colors indicating the SST trimming flag values: gray SF=1; green SF=2,3; blue SF=4,5; red SF=6,7; cyan SF=C (as for Fig. 12).


Figure 14. Locations of MORMET data for Jan. 1973, with colors indicating the wind trimming flag values (note: the higher value of UF or VF is plotted, as that is how the data are trimmed): gray UF/VF=1; green UF/VF=2,3; blue UF/VF=4,5; red UF/VF=6,7; cyan UF/VF=C (as for Fig. 12).


Figure 15. Locations of the R/V data for Jan. 1973, with colors indicating the SST trimming flag values: gray SF=1; green SF=2,3; blue SF=4,5; red SF=6,7; cyan SF=C (as for Fig. 12).


[Documentation and Software][Links to additional]


U.S. National Oceanic and Atmospheric Administration hosts the icoads website privacy disclaimer
Document maintained by icoads@noaa.gov
Updated: May 17, 2011 22:02:20 UTC
http://icoads.noaa.gov/pre2.5-2.html