Hacking Climate Change With WeatherLink

by Cap'n Zilog  (capnzilog@moshimail.com)

WeatherLink is a cloud service maintained by Davis Instruments for the benefit of Davis' network of publicly reporting weather monitoring stations.

If you already own a weather station like the Davis Vantage Pro2, you can add your station to the network with the addition of a WeatherLinkIP data logging network dongle which is basically a set-and-forget device.

After a few minutes spent establishing your account and tying it to the device's ID, you can view the basic details of your weather station online along with thousands of others at WeatherLink's global reporting map, www.weatherlink.com/map.php.  WeatherLink can also be used to share your reports with other networks such as the Citizen Weather Observer Program or GLOBE Science Network.

The problem is that Davis makes it very diffi cult to actually get your data out of the cloud once it's in.  The software provided by the manufacturer for this purpose is an antiquated Windows 3-era program simply called WeatherLink that looks like it hasn't been updated in years.

In addition, the source of WeatherLink's cloud data is hidden, the binary data records themselves are kept private, and the software makes it difficult to perform a complete data dump from the 'Net more than once.  The last item may merely be a misguided attempt to reduce the burden on Davis' servers by encouraging incremental data retrieval, but it has the effect of blocking user access to their own raw records for performing long-term ad-hoc weather analysis.

It should be noted that WeatherLink software users also have the capability to download recent datasets contained within the memory of their WeatherLinkIPs, but this information is mainly of use for near-term reporting purposes.

The true value of the WeatherLink Network is the time-stamped data cache uploaded by every WeatherLink device connected to the Davis server.  These records can go back over two years - with a maximum user archive size currently fixed at 10,240 records.  This is sufficient to hold 853 days of detailed weather history (about 500k of information) at the slowest two hour reporting rate.

Other interval rates such as 1, 5, 10, 15, 30, and 60 minutes are available, but you can't change the logging rate once your account is set up without losing your archived data.

For long-term weather tracking, the 60 minute rate works best, as it's the fastest reporting interval to cover a year's span of time without exceeding the archive limit.  Exceeding this means dropping the earliest records, so in order to maintain a detailed weather history of your location from year to year and track your area's climate change, you should download your archives annually.

Fortunately, the protocol for downloading your data is a simple task for any web browser.  Sniffing packets while performing a web data grab with a fresh copy of Davis' WeatherLink 5.9.3 software revealed two types of database queries available via the HTTP GET protocol, formatted as follows.

First, the Query URL, which results in a server response like so:

http://weatherlink.com/webdl.php?timestamp=0&user=[username]&pass=[password]&action=headers
"Model=16 Records=525 MaxRecords=10240 ArchiveInt=60 ConsoleVer=Sep 29 2009 VantageTX=0"

This header information is mainly for the benefit of providing support for WeatherLink but it confirms a few useful things:

  1. The nature of the weather station reporting.
  2. The number of records available.
  3. The station's logging interval (hourly, in this case).
  4. Confirming the Davis maximum record limit.

The username and passwords are yours to supply; the history of every WeatherLink station that's ever reported to WeatherLink is downloadable even if its hardware is down or offline.

Next up is the Dump URL, which allows you to download the data itself and (browser willing) save it as a binary file:

http://weatherlink.com/webdl.php?timestamp=0&user=[username]&pass=[password]&action=data

Note that since Davis has changed the IP address of weatherlink.com once or twice since I've been following this, as long as you stick to the correct server name you should be fine.  However, don't confuse this URL with www.weatherlink.com, which maps to a different IP and may not work.

Davis's raw weather data records follow the Rev. "B" archive format, which is public and available at www.davisnet.com/support/weather/download/VantageSerialProtocolDocs_v230.pdf.  These 52-byte records contain every field reportable by Davis Weatherlink stations to the server and then some; if your station does not have solar or soil temperature reporting for example, then the unused fields will be left blank (0xFF).

What follows is the breakdown of a typical Davis Vantage Pro2 Plus weather station record with no extra sensors attached:

Example Davis WeatherLink Rev. "B" Archive Record (52 bytes)

88 15 C8 00 04 02 0E 02 03 02 00 00 00 00 D3 75
00 00 F7 04 C1 02 2B 44 01 05 0A 0A 00 01 00 00
00 2C FF FF FF FF FF FF FF FF 00 FF FF FF FF FF
FF FF FF FF

Rev. "B" Archive Record (Little-Endian, LSB First)

88 15        ;! Archive write date: 0x1588 = 0001010:1100:01000
             ; (Year = 10 + 2000, Month = 12, day = 8)
C8 00        ;! Archive write time [Hour = int(x/100), Minute = (x % 100) = 02:00 hrs]
04 02        ;! Outside temp. (0x0204 = 516 = 51.6' degF)
0E 02        ;! High outside temp. over archive period (0x020E = 526 = 52.6 degF)
03 02        ;! Low outside temp. over archive period (0x0203 = 515 = 51.5 degF)
00 00        ;! Rainfall clicks (.01' bucket tips over archive period)
00 00        ;! Highest rain rate (in bucket tips per hour)
D3 75        ;! Barometer (0x75D3 = 30163 Hg/1000)
00 00        ;! Solar radiation (W/m2)
F7 04        ;! Number of wind speed data packets received (0x04F7 = 1271)
C1 02        ;! Inside temperature (0x02C1 = 705 = 70.5 degF)
2B           ;! Inside humidity at end of archive period (0x2B = 43%)
44           ;! Outside humidity at end of archive period (0x44 = 68%)
01           ;! Average wind speed (mph)
05           ;! Highest wind speed over archive interval (mph)
0A           ;! Direction of high wind speed = SW
             ;  0 = North 
	         ;  1 = North-Northeast
	         ;  2 = Northeast
	         ;  3 = East-Northeast 
	         ;  4 = East
	         ;  5 = East-Southeast
	         ;  6 = Southeast
	         ;  7 = South-Southeast
	         ;  8 = South 
	         ;  9 = South-Southwest
	         ;  A = Southwest
	         ;  B = West-Southwest
	         ;  C = West 
	         ;  D = West-Northwest
	         ;  E = Northwest
             ;  F = North-Northwest
0A           ;! Prevailing wind direction = SW
00           ;! Average ultraviolet index / 10
01           ;! ET in/1000
00 00        ;! Highest solar radation over archive period (W/m2)
00           ;! High ultraviolet index over archive period (W/m2) [divide this by 10]
2C           ; Forecast rule at end of archive period
FF FF        ; Leaf temperature (F + 90)
FF FF        ; Leaf wetnesses (0-15)
FF FF FF FF  ; Soil temperatures (F + 90)
00           ; Download Record Type (0x00 = RevB)
FF FF        ; 2 Extra humidity values
FF FF FF     ; 3 Extra temperatures (F + 90)
FF FF FF FF  ; 4 Soil moistures (cb)

The comments with exclamation marks represent fields that also appear in the human-readable "Download.txt" files generated by Davis' WeatherLink software.  These contain several items computed by WeatherLink for display purposes which are not present in the raw Davis archive records themselves, including:

"Arc Int" is, of course, the Archive Interval retrieved from performing the Davis header query described earlier.

I'm going to close this article with a data structure for parsing your own WeatherLink Network archives.  It's a short step from this to writing a program that can, for example, generate tab-delimited Excel files which you can use to plot your weather history in any manner desired.

The sky's the limit!

typedef struct {
  unsigned short bfDfDay:5, bfDfMonth:4, bfDfYear:7;
  unsigned short uwWriteTime, uwOutTemp, uwHiOutTemp, uwLowOutTemp,
    uwRainClicks, uwHiRainRate, uwBarometer, uwSolarRad,
    uwNWindPackets, uwInTemp;
  unsigned char ucInHumidity, ucOutHumidity, ucAvgWind, ucHiWind,
    ucHiWindDir, ucWindDir, ucUvi, ucEt;
  unsigned short uwHiSolarRad;
  unsigned char ucHighUV, ucForecastRule;
  unsigned short uwLeafTemp, uwLeafWets;
  unsigned char uwSoilTemp[4], uwRecordType, uwXHumidity[2],
    uwXTemp[3], uwSoilMoist[4];
} DAVISREVBRECORD;
Return to $2600 Index