It’s a conundrum that every Irish bride will be familiar with.
You’ve found that special someone, resigned yourself to dealing with the in-laws from hell and mentally prepared yourself for a weekend of waving willy straws in Dingle at the hen.
All that remains? Picking a date when it’s not going to rain for the big day itself.
Alas, it’s here that even the most organised Bridezilla will find themselves stumped, because it’s nigh on impossible, even in 2019, to accurately predict the long-range weather forecast.
Victoria Beckham. Pic: GC Images
We live in a world where our phones seem to know we’re pregnant before we do, so why then can we still not determine a long-term weather forecast?
The answer, according to Dr Clifford Gilmore, a mathematical science researcher from UCC is simple, chaos.
Writing for RTÉ’s Brainstorm, he’s broken down the process of predicting the weather so as to help us better understand why long-term weather prediction has us stumped.
- According to Dr Clifford, meteorologists continually collect various weather markers, such as temperature and air pressure, and enter the data into mathematical models of the Earth’s atmosphere.
- Next, simulations are run on supercomputers to produce weather forecasts.
- But no matter what, even the most meticulous work can only ever be an approximation of reality.
Pic: PA Wire
- With the weather, even the tiniest departure from the original approximations means that over time the weather forecast will begin to differ from the real weather.
- This phenomenon goes by the catchy name of ‘sensitive dependence on initial conditions,’ or the more user-friendly ‘butterfly effect.’
Here’s Why Predicting the Weather Is Still So Difficult
A three-day weather forecast today is as good as the one-day forecast 10 years ago. ANGELA WEISS/AFP/Getty Images
It turned out the northeast snow storm today came a few hours later than we were warned.
Weather forecast technology has come a long way. A three-day forecast today is as good as the one-day forecast 10 years ago, thanks to the massive computing power of supercomputers that can consolidate trillions of data points on atmospheric conditions into simple simulations.
And yet, pinpointing where and when a snow storm will hit is still extremely challenging.
One straightforward reason is simply the number of factors at play.
“One of the great challenges to predict today’s storm in the northeast is the type of precipitation—will it be rain or snow or a little bit of both? These fine-scale details can be very difficult to track from one hour to the next because there are so many variables that can influence these,” said Greg Carbin, chief of forecast operation at the National Weather Service, the federal agency that provides weather forecasts to major TV networks and other media we get weather information from.
- The distance between a light drizzle and a blizzard can be as close as 30 miles, meaning there can be a shower in State Island and barely any rain in the Bronx at the same time.
- A more technical reason, as a 2016 Economist article pointed out, is that conflicting prediction models can produce vastly disparate results.
- For example, before Hurricane Sandy hit the East Coast in 2012, most American weather models predicted the storm would bypass the mainland and head towards the Atlantic Ocean, while European models correctly identified the storm track.
Weather forecasting starts with raw data describing atmospheric conditions collected by a number of sources, ranging from satellites to on-the-ground weather stations. This information, in the form of trillions of data points, is then processed through models that generate the most probable simulations of the weather at a future time.
As a general rule,the more data computers can crunch (and the faster they can do so), the more accurate the forecast results will be.
“A good weather forecast requires two parts: an accurate initial state of the atmosphere and a good model with sufficient resolution.
But, in reality, an accurate three-dimensional initial state of the atmosphere is exceptionally challenging.
That creates uncertainties that get amplified as the atmospheric simulation evolves in time,” Xi Chen, a researcher in atmospheric and oceanic sciences at Princeton University, told Observer.
Chen’s lab produced a model called FV3, which can utilize tens of thousands of processors to work simultaneously on atmospheric simulations. The model was adopted by the National Weather Service in 2016 as part of an upgrade following the misforecast of Hurricane Sandy. The new model is currently under implementation.
The existing model of the National Weather Service divides the Earth into a grid of 13 km-by-13 km blocks to observe and make predictions.
“However, many crucial weather phenomena, such as precipitation, are largely determined by cloud processes, which could be of much smaller scales,” Chen said.
“Therefore, scientists rely on a technique called ‘physical parameterization’ to estimate such processes, which inevitably introduces uncertainties.
Our job is to minimize the uncertainties by both improved theories and hopefully more available computing resources.”
“Improvements in forecast accuracy have been pretty dramatic in past decades. The global models have gotten pretty good at indicating potential significant weather five to seven days out. For example, the snow storm we are dealing with today was predicted a week ago, even though details still need to be worked out,” Carbin told Observer.
“It’s telling the future, after all,” Chen added.
Why the weather forecast will always be a bit wrong
The science of weather forecasting falls to public scrutiny every single day. When the forecast is correct, we rarely comment, but we are often quick to complain when the forecast is wrong. Are we ever likely to achieve a perfect forecast that is accurate to the hour?
There are many steps involved in preparing a weather forecast. It begins its life as a global “snapshot” of the atmosphere at a given time, mapped onto a three-dimensional grid of points that span the entire globe and stretch from the surface to the stratosphere (and sometimes higher).
Using a supercomputer and a sophisticated model that describes the behaviour of the atmosphere with physics equations, this snapshot is then stepped forward in time, producing many terabytes of raw forecast data. It then falls to human forecasters to interpret the data and turn it into a meaningful forecast that is broadcast to the public.
The whether in the weather
Forecasting the weather is a huge challenge. For a start, we are attempting to predict something that is inherently unpredictable. The atmosphere is a chaotic system – a small change in the state of the atmosphere in one location can have remarkable consequences over time elsewhere, which was analogised by one scientist as the so-called butterfly effect.
Any error that develops in a forecast will rapidly grow and cause further errors on a larger scale. And since we have to make many assumptions when modelling the atmosphere, it becomes clear how easily forecast errors can develop. For a perfect forecast, we would need to remove every single error.
The Great Storm of October 1987: when forecasters got it wrong.
Forecast skill has been improving. Modern forecasts are certainly much more reliable than they were before the supercomputer era. The UK’s earliest published forecasts date back to 1861, when Royal Navy officer and keen meteorologist Robert Fitzroy began publishing forecasts in The Times.
Why is weather so hard to predict accurately?
This is a supplement to the story Cyclones in the Forecast.
It is a common mystery that elicits complaints and questions — why does the weather forecaster in the media sometimes get it wrong? Maybe “chance of showers” turned into a sunny day or a predicted thunderstorm rolled in with much stronger winds than predicted and unexpected hail.
Incorrect forecasts tend to be remembered better than correct ones, but weather forecasting is actually a lot more accurate than it gets credit for — and continually improving. Thanks to research developments leading to steadily increasing forecast skill, 72-hour forecasts today have the same level of accuracy that 36-hour forecasts had in 1990. But, some error inevitably remains.
“Weather is complex,” said Adam Clark, Iowa State alumnus ('04 BS, ’06 MS, ’09 Ph.D. atmospheric science) and research meteorologist at the National Severe Storms Laboratory. “Perfect weather predictions would require accurately observing numerous variables over every square inch of the atmosphere.”
The National Weather Service cannot cover every square inch — but they cover as much as they can through observations from weather stations, ocean buoys, satellites and balloons. They also log these observations over time, incorporating billions of past observations into their prediction models.
“These observations still have errors because the instruments themselves are not perfectly accurate,” Clark said. “Plus, the prediction models themselves have errors because the equations that explain atmospheric motion do not have exact solutions.”
The compounding of tiny errors can grow exponentially. This is commonly known as “chaos theory” or the “butterfly effect,” the idea that something very small in one part of the world, such as the flap of a butterfly’s wings, can have a large impact elsewhere in the world.
- Research, said Clark, focuses on three key areas to continue to improve predictions:
- (1) obtaining better observations and developing new ways to observe more of the Earth
(2) developing better technologies to put different observational datasets together for input to a weather prediction model
- (3) improving the weather prediction models themselves
NetApp BrandVoice: Why The Weather Is Still So Hard To Predict
Hurricane Isaac capped an extreme summer that brought record heat, powerful storms and tornadoes to much of the eastern United States. It also underscored the inexact nature of weather forecasting.
Less than a week before Isaac clobbered Louisiana, some models had it headed toward Florida. Organizers of the Republican Convention took the prudent step of delaying the start of the event, only to have the storm bypass them.
Weather forecast (Photo credit: kiwinz)
With climate change heavily upon us—and more than half the country suffering the worst drought in almost half a century—weather forecasting and climate prediction have taken on new urgency.
The ability to see weather coming long before it arrives would help businesses ranging from farmers to ski resorts, and it would give communities and residents time to gird for dramatic events that can damage and destroy homes and property, and even kill.
Well, don't hold your breath, because advances in weather forecasting move at the pace of a stalled hurricane. True, the science has come a long way since the first organized attempts in the 1920s, when, working by hand, English mathematician Lewis Fry Richardson needed just six weeks to come up with a six-hour forecast (talk about a wasted effort).
Forecasting really took off with the advent of computers, and today's four-day forecast is as accurate as a one-day forecast was 30 years ago. To meteorologists, that's a big deal. To everyone else, not so much.
Why can't they predict temperature, wind, rain and such for an entire month? The answer lies in the complexity of the weather. Short-term forecasting depends on temperature, clouds, precipitation, wind and pressure.
For long-term climate prediction, add ground and sea temperatures, ocean currents, and sea ice (or lack of it), as well as atmospheric pollution.
Even accurately predicting tomorrow's weather takes a major effort.
The National Weather Service collects billions of observations every day from geostationary and polar orbiting satellites, along with data from weather balloons and ground stations.
The information is fed into a supercomputer running software developed by the scientific community. The supercomputer spits out a forecasting model four times a day, and forecasters analyze the models to make their weather predictions.
Jim Yoe, chief administrator of the government's Joint Center for Satellite Data Assimilation, calls it a matching process. “You have to develop science for the modeling, and you need computing power to run that,” he says. “But you also need the observations, and a way to match reality to the data assimilation.”
OK, so why can't that method carry out further than a few days? Because the farther ahead you go, the more you must deal with chaos.
A butterfly flapping its wings in Tokyo may not cause it to rain in Central Park, but that's the essence of chaos theory: Small changes in the weather in one place can affect other areas as well.
Moreover, the numerical equations used to simulate the atmosphere are also subject to chaos, with small errors doubling in just five days.
The National Weather Service is trying its best. Currently it is switching over to a new supercomputer and backup, each of which will contain more than 10,000 processing cores, run at more than 200 teraflops and have 2.
6 petabytes of storage. But not even the fastest supercomputer alive is going to push the envelope much. Yoe figures that the theoretical limit on accurate forecasting with today's technology is on the order of two weeks.
That means that seasonal data models are basically crap shoots, as anyone knows who has sat through torrential downpours during what was supposed to be a mild rainy season.
“Seasonal models are nowhere near as good as a daily forecast, and it's not clear how good a seasonal model can become,” says Mike Halpert, deputy director of the NWS's Climate Prediction Center.