How Big Data Enables Real-Time Weather Forecasts

A new generation of sensors and apps are making weather prediction more accurate.

By News Center Team on July 20, 2015

It wasn’t very long ago that the weather forecast was a joke in the manner of airline food: how, after so many years, was it possibly so bad? We’d look, every morning, to the morning news, where a professional meteorologist would show us a big map and tell us whether it would rain that day, and in the upcoming week or so. They were frequently wrong. But a new generation of sensors and technologies are starting to deliver real-time, hyperlocal weather forecasts — sometimes down to the minute.

Weather forecasts are nothing more than guesses based on data, which isn’t inherently bad. The idea is, you take every bit of data you can find: snapshots from satellites, plus temperature, humidity, and wind speed measurements from various locations, and you run it through what’s called a model, or a projection. The model says, well, if the temperature is X degrees here and Y degrees here, and the wind from area A is colliding with wind from area C, and the humidity in area C is however much percent, then the weather in Anytown, USA is likely to be a certain way by a certain time. It’s much more complicated than that, but that’s the basic idea.

These are guesses — well-informed guesses, but guesses. They’ll always be guesses, but the thing is, they can be better ones. And the way to get better guesses is to get more data. That’s where modern technologies come in, because if there’s anything we’ve got enough of these days, it’s data.

That data comes from more places than ever before. Most weather apps rely on the National Weather Service, which operates over 140 Doppler radar stations throughout the United States. Doppler is a form of radar that bounces off of humidity in the air to give in-depth data, and the National Weather Service operates more of these than any other. But the National Weather Service isn’t resting on its laurels: in 2014, the NWS announced the new High-Resolution Rapid Refresh model, which is orders of magnitude more accurate than the previous model. The HRRR gives data every 15 minutes instead of every hour, and the radar itself is more powerful than ever before. “Using the HRRR, forecasters have an aerial image in which each pixel represents a neighborhood instead of a city,” writes the NWS in a statement.

But not all weather predictors rely entirely on the NWS, no matter how good it is. “Our WeatherBug apps use real-time weather data from our own weather and lightning sensor network,” says Rachel Hunt of WeatherBug, an extremely popular mobile weather app. WeatherBug has set up sensors, newly inexpensive, at thousands of locations across the country — they may not be as powerful as the NWS’s, but there are a lot more of them. (And WeatherBug, like any other app that wants it, has access to the NWS’s data as well.) “Our data streams in at the rate of more than 6 billion transactions a day on average, and the data feeds our forecasting algorithms,” says Hunt.

Other apps go even more local. WeatherSignal uses the sensors that are right there in your phone to create sort of portable, teensy weather data tracking hotspots. From your typical Samsung Galaxy smartphone, for example, “we can collect temperature, pressure, light intensity, magnetic flux and humidity,” writes the WeatherSignal team.

Another app, Dark Sky, takes a slightly different approach. A mobile app that began its life on Kickstarter, Dark Sky doesn’t try to present you with all the data it can find. Instead it takes the information provided by the NWS, along with dozens of other sources from across the world, runs it through a fantastically complicated system that separates the noise within the NWS data from the good stuff, and tries to answer a simple question: what will the weather be like within the next hour? According to Dark Sky founder Adam Grossman, predicting weather actually gets easier as you shorten the time frame. What Dark Sky actually does is track storm movement, and connects it with your current location by continually tracking your location in the background, whether the app is open on your phone or not. “While the weather becomes chaotic and unpredictable at large timescales (hours to days), its behavior becomes increasingly linear at smaller and smaller timescales,” writes Grossman in a blog post explaining the app.

These apps are taking advantage of all the amazing new resources we have at our disposal: cheap sensors, always-on connections, constant location tracking and ever more processing power to cope with all that data. And they can, with increasing accuracy, tell you pretty much exactly when it’s going to rain.

Tags: [weather data](http://www.verizonwireless.com/news/index.tags_weather data.html), [weather apps](http://www.verizonwireless.com/news/index.tags_weather apps.html), [weather information](http://www.verizonwireless.com/news/index.tags_weather information.html)