Adding Push Notifications to the Tempest WeatherBot

This is the "final" installment of my summer 2022 blog series about the development of my Slack Tempest WeatherBot. The last two posts walked through expanding interactivity with the bot and also adding external NWS API data to the feature set. This post is the "cherry on top" as it outlines something I'd been wanting and considering since the very beginning: "push" notifications for high-impact events such as lightning or new alerts.

Lightning Data is Fascinating

One of the things I've loved most about the Tempest station is that it detects lightning up to about 25-30 miles away. Even without radar, if you look at data or charts of this information you can really see the movement of a thunderstorm...and it's also really fascinating to see the number of detections, since not everything is a cloud-to-ground event.

Additionally, I've discovered that major power infrastructure events, specifically power restorations at the transmission or substation level, are often detected as lightning events with Tempest. You can almost pinpoint the time and location of a restoration!

I haven't written about lightning in the rest of this series because it's a tricky one. The bot summaries and app home details show lightning information when it's available, but lightning is always an after-the-fact thing. You either have data for it...or you don't.

Issue #1 in the repo for the WeatherBot was simply "Implement Lightning Data" ... and finally this past April I closed it out!

Creating a Push Notification

In Slack it's not at all difficult to create a "push" notification: use a webhook to a specific channel with a crafted message...or use the API to do the same (and get a few more nice features). All of those things are pre-existent with the bot code...we just need to deal with monitoring for new events.

Holding Out for a Hero

Throughout my development of the WeatherBot, I have exclusively used the REST API for Tempest...but there's also support for WebSockets. Technically there's also super-localized direct access via UDP but I have never considered it as my host isn't on my local network.

This was my opportunity to play around a bit with WebSockets, which felt like an aged/antiquated tech to me...but at the same time using WebSockets provides a more "real-time" level of data resolution (could be near-instantaneous) than using REST which only provides 1-minute minimum interval.

I started out by finding and installing a super-lightweight PHP WebSocket client from Textalk which gave me the basics to work with a WebSocket. This gave me the ability to monitor for lightning events which are well-documented in the Tempest API WebSocket reference.

Ultimately a WebSocket connection is made (request listen_start and process the ack) and we monitor for an evt_strike to pop up and then do things accordingly by crafting a basic message with the details (like distance and time).

Similarly, if we use a regular REST observation, we just make a call every minute and monitor for new strikes in the resulting data. If something is found, then a basic message with the details is created and sent off just like the WebSocket version.

BUT...There Are Other Considerations

The WeatherFlow folks request that only one WebSocket be open at a given time which makes complete sense. But if that's the case, how might we reasonably set this up to monitor the status of the socket and/or react accordingly (i.e. restart the process)? While this all sits on a stable server on the Internet...that server does reboot periodically...and what about situations where we had some weird network hiccup?

For WebSockets, I settled on creating a mechanism that will 'pause' and respond to new data every 15 seconds or so. This seemed appropriate, but the value can be modified in the configuration files. Further, I decided to use a 1-hour cadence per WebSocket connection. This is intended to gracefully close an existing, live socket in advance of the 1 hour window, and then via the job invoked from cron will open a new WebSocket. To that end, the system would naturally recover after no more than an hour...and under most circumstances that seemed appropriate.

What I'm not writing about here is the boatload of randomized testing I was doing in this quest of WebSockets. Getting the intervals just right and coming to the conclusions above took a lot of trial and error to get the math/calculations correct and affording some flexibility in latency. It's not a perfect solution, but I found myself coming back to the adage of "exactly what problem am I trying to solve?"

With REST, the process is similar in that an active job is triggered hourly...but it waits for a defined interval (~1 minute) between making new requests. By building it this way I could support (and switch between) both mechanisms (REST or WebSockets) without changing other bot functionality (e.g. the cron job that makes it all happen).

Holy Noise, Batman!

Fortunately I had built this out on a day there were storms in the forecast. I didn't have to wait long to test with real, live data.

But HOLY SHIT for noise!

Some metering was going to be necessary, because every detection shipped a notice to the special alerts channel in Slack. I knew this would be the case, but it was something else to see it in action! So I added parameters to "when" it would be acceptable to send a new notice:

• No lightning detected in the last ten minutes (or it has been at least ten minutes since the last notice was sent);
• Lightning detected since the last notice, but must be a defined distance (I chose 8km/5mi as this distance) closer to force a new notice; or
• ANY strike closer than the defined distance (~8km/5mi) as long as it's closer than the last super-close strike.

I didn't have a good chance to test the third scenario out of the gate as the storms (when monitoring has been live and active) had few "close" strikes. However, a few weeks later we had a couple intense storms...and those provided very useful in tweaking the third scenario (specifically to only notify when something's even closer).

As a base, though, WebSockets could notify once every 15 seconds if necessary; REST notices would be no more than once every minute. I have stuck with REST for my regular day-to-day bot usage. An example of some rapid-fire notices is below. I've noted four increasingly-close-range strikes (matching the third scenario) with red arrows in the image:

What About Push Notices for Alerts?

I'd already built the key parts for querying alerts, so creating a process to push notices for new (or "higher severity") alerts was pretty straightforward. Using much of the logic above, I just created a job script to do the same thing for alert data, using many of the same defaults already developed and in place.

All Day, Every Day?

This was a bit of a balance question. I chose to set up the cron job for monitoring/push processes to only run during the majority of the normal "day," like 7 a.m. through 8-9 p.m....but that's based on my own desires. I'm not usually around a device after 7-8 p.m....or before 7 a.m., so it doesn't make a lot of sense to have all these things running in the "off hours." This is all configurable in the bot config files and cron.

It's Quite the Project!

I've spent time over the last 18 months or so (on and off) poking around at improvements and new features with the bot. Undoubtedly there will be more to come, but it's been a super fun way to work with real data (from my own backyard) and do something cool, albeit niche, with it. It's been a hell of a learning process, too...with the various APIs and the goofy tricks with the Slack Block Kit.

As I noted in the opening post, the WeatherBot website and source/example code is available for anyone to peruse. If you have a Slack workspace and a Tempest station, give it a shot and feel free to submit a PR for fixes or new stuff!