Sunday, May 20, 2018

Reforming the National Weather Service, Part 1: Changing the Role of Human Forecasters

In a number of ways, the U.S. National Weather Service (NWS) needs to reform its practices.

And let's be very clear, I don't mean that NWS forecasts are getting less skillful. They are getting better.  But it is clear that the NWS has fallen behind the state-of-the-art and is not producing as skillful or useful products as it could or should.

This blog will consider the role of human forecasters in the NWS and how the current approach to prediction is a throwback to past era.  As a result forecasters don't have time to take on important tasks that could greatly enhance the quality of the forecasts and how society uses them.

Let me begin by by showing you a comparison of forecast accuracy of the NWS versus major online weather sites, such as,,, and others.  This information is from the web site, which rates the 1-3 day accuracy of temperature and precipitation forecasts at cities around the nation.  I have confirmed many of these results here in Seattle, so I believe they are reliable.  I picked three locations (Denver, Seattle, and Washington DC) to get some geographic diversity.  Note that the NWS Digital Forecast is the product of NWS human forecasters.

The National Weather Service forecasters are not in the top four at any location for either the last month or year, and forecast groups such as The Weather Channel, Meteogroup, Accuweather, Foreca, and the Weatherunderground are in the lead.  And these groups have essentially taken the human out of the loop for nearly all forecasts.

Or we can take a look at some of the NWS own statistics. Here are the mean absolute errors for surface temperature for forecast sites around the U.S., comparing U.S. forecasters (NDFD), the old U.S. objective forecasting system called MOS (Model Output Statistics) that does statistical post-processing to model output, and a new NWS statistical post-processing system called National Blend of Models (NBM).  Lower is better.  The blend (no humans involved) is as good or better than the forecaster product for forecasts going out 168 h

12-h precipitation?   The objective blend is better  (lower is better on this graph).

 I could show you a hundred more graphs like this, but the bottom line is clear:  human forecasters, on average, can not beat the best objective systems that take forecast model output and then statistically improve the model predictions.

Statistical post-processing of model output can greatly improve the skill of the model predictions.  For example, if a model is systematically too warm or cold at a location at some hour based on historical performance, that bias can be corrected. (reminder:  a computer forecast model solves the complex equations the describe atmospheric physics)

National Weather Service forecasters spend much of their time creating a
graphical rendition of the weather for their areas of responsibility.  Specifically, they use an interactive forecast preparation system (IFPS) to construct a 7-day graphical representation of the weather that will be distributed on grids of 5-km grid spacing or better. To create these fields, a forecaster starts with model grids at coarser resolution, uses “model interpretation” and “smart” tools to combine and downscale model output to a high-resolution IFPS grid, and then makes subjective alterations using a graphical forecast editor.

Such gridded fields are then collected into a national digital forecast database (NDFD) that is available for distribution and use. The gridded forecasts are finally converted to a variety of text products using automatic text formatters.

So when you read a text NWS forecast ("rain today with a chance of showers tomorrow"), that text was written by a computer program,  not a human, using the graphic rendition of the forecast produced by the forecasters.

But there is a problem with IFPS.   Forecasters spend a large amount of time editing the grids on their editors and generally their work doesn't produce a superior product.  Twenty years ago, this all made more sense, when the models were relatively low resolution, were not as good, and human forecasters could put in local details.  Now the models have all the details and can forecast them fairly skillfully.

And this editing system is inherently deterministic, meaning it is based on perfecting a single forecast. But the future of forecasting is inherently probabilistic, based on many model forecasts (called an ensemble forecast).  There is no way NWS forecasters can edit all of them.    I wrote a peer-reviewed article in 2003 outlining the potential problems of the graphical editing approach...unfortunately, many of these deficiencies have come to light.

Another issue are the "seams" between forecast offices.  Often there are differences in the forecasts between neighboring offices, resulting in large changes at the boundaries of responsibilities...something illustrated below by the sky coverage forecast for later today:

Spending a lot of time on grid editing leaves far less forecaster time for more productive tasks, such as interacting with forecast users, improving very short-term forecasting (called nowcasting), highlighting problematic observations, and much more.

But what about the private sector?

Private sector firms like Accuweather, Foreca, and the Weather Channel all use post-processing systems that descended from the DICast system developed at the Research Application Lab (RAL) of  the National Center for Atmospheric Research (NCAR).    DICast (see schematic below) takes MANY different forecast models and an array of observations and combines them in an optimal way to produce the best forecast. Based on based performance, model biases are reduced and the best models given heavier weights.

This kind of system allows the private sector firms to use any input for their forecasting systems, including the gold-standard European Center model and their own high-resolution prediction systems, and is more sophisticated than the MOS system still used by the National Weather Service (although they are working on the National Blend).

The private sector firms do have some human forecasters, who oversee the objective systems and make adjustments when necessary.  The Weather Channel folks call this "Human over the Loop". 

The private is using essentially the same weather forecasting models as the NWS, but they are providing more skillful forecasts on average.  Why?  Better post-processing like DICast.

Machine Learning

DiCast and its descendants might be termed "machine learning" or AI, but they are relatively primitive compared to some of the machine learning architectures currently available.  When these more sophisticated approaches are applied, taking advantage of the increased quality and quantity of observations and better forecast models, the ability of humans to contribute directly to the forecast process will be over.   Perhaps ten years from now.

So What Should the National Weather Service Do?

First, the NWS needs to catch up with private sector in the area of post-processing of model output.  For too long the NWS has relied on the 1960s technology of MOS, and the new National Blend of Models (NBM) still requires work.  More advanced machine learning approaches are on the horizon, but the NWS needs to put the resources into developing a state-of-the-art post-processing capability.

Second, the current NWS paradigm of having a human be a core element of every forecast by laboriously editing forecast grids needs to change.  The IFPS system should be retired and a new concept of the role of forecasters is required.  One in which models and sophisticated post-processing take over most of the daily forecasting tasks, with human forecasters supervising the forecasts and altering them when necessary.

Humans are needed as much as ever, but their roles will change.  Some examples:

1.    Forecasters will spend much more time nowcasting, providing a new generation of products/warnings about what is happening now and in the near future.
2.     With forecasts getting more complex, detailed, and probabilistic, NWS forecasters will work with local agencies and groups to understand and use the new, more detailed guidance.
3.    Forecasters will become partners with model and machine learning developers, pointing our problems with the automated systems and working to address them.
4.    Forecasters will intervene and alter forecasts during the rare occasions when objective systems are failing.
5.   Forecasters will have time to do local research, something they were able to do before the "grid revolution" took hold.
6.  Importantly, forecasters will have more time for dealing with extreme and impactful weather situations, enhancing the objective guidance when possible and working with communities to deal with the impacts.  44 people died in Wine Country in October 2017 from a highly predictably weather event.  I believe better communication can prevent this.

I know there are fears that the NWS union will push back on any change, but I hope they will see modernization of their role as being beneficial, allowing the work of forecasters to be much more satisfying and productive.

Friday, May 18, 2018

Flooding in Eastern Washington, Driest May on Record for Western Washington

If you like dramatic contrasts, Washington State offers a world-class example--and I am not talking about politics.

Over eastern Washington a number of rivers are at flood stage, some reaching record levels.

Thunderstorms and heavy rain have been pummeling the Columbia Basin and northeast WA. The National Weather Service has issued a flood warning for several counties including Okanogan, Bonner, Chelan, Ferry and Pend Oreille, and the Kettle River, which flows through Ferry and Stevens counties, had a record high on May 10 of 22.54 feet on May 10, busting the previous record of 21.1 feet in 1948.

In contrast, over the west, many locations have had little precipitation in May, and there is chance Seattle will have its driest May on record.

Not black and white.  But dry and very wet.   But why?  And is spring flooding in eastern WA unusual?

Let's start by checking out the flooding situation, with information from the NOAA/NWS River Forecast Center in Portland (below).  Red dots are floods, blue dots indicated moderate floods, and purple are major floods.  And several other locations are at bank-full.

What is going on?   The combination of a healthy mountain snowpack and warming temperatures, resulting in rapid melting of snow, and the filling of regional rivers.

Here is the snowpack numbers for April 15th, from the very nice SNOTEL web site. Washington, northern Idaho, and Montana had snow-water amounts well about some locations above 150% of what is typical.

And snowpack in British Columbia, from which the Columbia River drains was also very high (see table)

So going into spring with a bountiful spring snowpack was step 1. 

Then a combination of warming temperatures and increasing spring solar radiation caused rapid snowmelt.  The departure of average temperatures from normal (below) shows our temperature anomaly (difference from normal) for the past two weeks, with much of eastern WA and northern Idaho being 6-8F above normal for that period.

So the normal rapid snowmelt of spring was supercharged by warmer than normal temperatures. 

But like a late-night commercial, I should note that there's more.    The weather pattern has produced wetter than normal conditions east of the Cascade crest (and drier than normal to the west)--see map below of the precipitation departure from normal for  5/3-5/16.

During the past few days, showers and thundershowers have moistened eastern Washington (see radar image for 3 PM yesterday), with the last 24-h totals being impressive.  A number of locations in northeast WA have gotten over 1 inch during the last day.

So why wet east of the Cascades and dry west?  Because there has been a persistent ridge of high pressure along the coast, with troughs (low pressure) moving into California and northward into eastern WA and Idaho (see upper level--500 hPa--map for 5 PM Thursday to illustrate.)  The ridge has kept western WA dry, while the transient troughs have initiated thunderstorms east of the Cascade crest.

It is important to note that spring-time flooding in eastern Washington and along the Columbia is the normal state of affairs in our region.  Before the Columbia and Snake River dams were in place, major and often catastrophic flooding was commonplace. 

One of the most famous events was the great snowmelt flood of May 1948.  Just as in this year, regional mountain snowpack was above normal, unusually warm temperatures hit during mid-month, and thunderstorms dampened eastern Washington. Richland flooded and the surge of water moved down the Columbia, inundating and destroying the city of Vanport (near Portland), which was never rebuilt.

As global warming reduces the regional snowpack later during this century, the threats of such springtime flooding should gradually decline over time.

Wednesday, May 16, 2018

Moisture Hole Reaches California

Today's water vapor imagery shows a dramatic "moisture hole" crossing central California.

Here are two amazing shots for 1 AM and  6 AM PDT this morning.
It almost looks scary.

These images show the temperatures of water vapor in the atmosphere.  White areas indicate lots of water vapor in the upper troposphere (roughly 15,000 to 30,000 ft), while dark areas indicate little.  Thus, dark colors show dry conditions in the middle to upper troposphere--the moisture hole.

This "moisture hole" is associated with a pronounced upper level low, as shown by the WRF model run last night (the 500 hPa, about 18,000 ft, heights are shown, with winds and temperature).  Moisture swings around the low, but values are less in its core.  Air that swings around the low rises more than air inside the low center.

This year we have gotten an unusual number of spring upper level lows heading into California...and there is another---even stronger--predicted for next week.

This pattern has brought clouds, precipitation and thunderstorms to northern CA, southern Oregon, and northern Nevada.  The NWS radars show lots of showers, some heavy moving westward over Oregon

And the lightning strikes for the 24-h period ending 1 AM this morning were impressive. Quite a number.

Precipitation totals over the West Coast for the 24-h period ending 8 AM had some significant totals over northern CA, which is obviously good for their water situation.  CA reservoirs are in very good shape and the late spring moisture helps keep the ground moist.

With all the action going south, Puget Sound was dry again, with only .08 inches in the Sea-Tac rain gauge so far this month.  Will we beat the all-time record (.12 inches)?   The next model run will probably provide the answer.

Monday, May 14, 2018

Some Record Minimum Temperatures This Morning and Dry May

Walking outside early this morning it was evident that something was up.   It felt really warm.

In fact, the minimum temperatures at locations around the region were warm with upper 50s at several locations, with some stations only dropping to the mid-60s.    Sea-Tac Airport had their record high low temperature for the date (58F).  In fact, it was the highest low for any date during the first half of May.

What was the reason for this morning warmth?    Well, we started out with near record warm air aloft for the date because of a ridge of high pressure overhead.  To show this, here are the climatological temperatures in the lower atmosphere (925 hPa pressure, about 3000 ft) at Quillayute, on the WA coast, with today's observation at 5 AM shown by the silver circle.  The red line shows the record for each date.  The temps aloft were near record levels.

But warm temperatures aloft are not enough since infrared cooling to space from the surface will cause low-level temperatures to fall, working against any record.

But there was another factor that worked against this nighttime cooling:  lots of water vapor in the lower atmosphere.   A good measure of water vapor content is the dew point temperature, with higher dew points indicating more water vapor.  So  here is the climatology of surface dew point at Quillayute, with the circle indicating today's values--quite high...nearly at record levels.

Water vapor is a very active absorber and emitter in the infrared and helps to keep the temperatures up at night.  That is why very dry deserts often get chilly at night even when the temperatures are extremely warm during the day.  So with lots of water vapor, the normal nighttime cooling was lessened.

One factor that helped with keeping the moisture high was the lack of offshore-directed, easterly winds.  Nothing dries out the air like strong downslope, easterly flow.  Only when downslope, easterly flow is very, very strong can it produce record high temperatures.

One more thing....this has been an extraordinarily dry May so far.   Seattle has only received .08 inches as of today...and we are halfway through the month.    The rest of the month looks fairly dry, with the forecast through May 28th never showing more than a 20% probability of rain for any day.  The driest May on record at Sea-Tac only received .12 inches.   So perhaps we have a chance to beat the record.

Ironically, the morning high resolution WRF model simulation showed the greatest rain action over northern CA and southern Oregon, which is unusual this time of the year.  Good for topping off the reservoirs in northern CA.

Saturday, May 12, 2018

Northwest Sudden Drying: The Rapid Transition to Dry Soils in May

People think of the Pacific Northwest as being a wet place, with wet, sodden soils.

Perhaps that is true during the winter and early spring. 

But something startling happens in May in much of western Washington: a rapid drying of the soils.   

A transition so rapid that some folks begin irrigating their gardens by late May.

Can this really be true?   Let's check it out.

Here is the soil moisture (8 inches down) at Langley, on southern Whidbey island, for the past month.  A major drop in soil moisture percentage from roughly 26% to around 15%.

Graph courtesy of the WSU Agweather website

Perhaps a better perspective can be gain by looking at the soil moisture over an entire year at this location.  For example, 2017.

Moisture values are high (20-26%) during the winter, but in May and early June the bottom drops out, with values dropping below 8%.  Slow drying follows in July and August, with a huge increase in October.

How about 2015, a year that was particularly warm and dry?  Again, a big drying in May

Looking at other years and other western WA stations shows that the May superdrying is a normal occurrence, something that accords with my experience as a gardener.

But why such rapid drying in May?  To understand, one must know about an important meteorological term:  evapotranspiration.  

Evapotranspiration or (ET for short) is the sum of evaporation and plant transpiration from the Earth's land and ocean surface to the atmosphere.  Over land, it is mainly water evaporating from the earth's surface and water loss by plants.  

So ET is the loss, while precipitation is the gain.  If ET exceeds the precipitation coming in, surface soil moisture declines.

Evapotranspiration is enhanced by warming of the surface (e.g., by solar radiation or warm air passing over it), drying of the air above the ground, and stronger winds (which help mix moisture away from the surface).

Think about what is happening in May.  

Precipitation is declining rapidly.  Here are average monthly precipitation totals at Seattle-Tacoma Airport.  May (roughly 1.75 inches) is a third of the November and December totals and less than half of the March rainfall.   So much less coming in.

Solar radiation, which warms the surface--promoting evaporation, increases rapidly in May, as shown by the solar radiation for the year at Langley, WA:

Surface air temperature increases in May, relative humidity decreases, and soil temperature (which is very important for surface evaporation ) increases substantially during that month (see below)

In short, a major decline of incoming moisture (from precipitation) and a big increase in evaporation from strengthening solar radiation at the surface and warming temperatures, results in a rapid decline of solar moisture in May.  

And we should not forget the large increase of evapotranspiration from plants, which are leafed out and growing rapidly in may, sucking out moisture from the soil.

Today, I was preparing the soil in my garden and could not believe how dry it was.  The May drying has gotten a particularly good start, with drier than normal and warmer than normal conditions (see graphs below)
Blue lines is normal precipitation, purple is this year. 

Red is this year, purple is normal high, cyan is normal low.

So be prepared to water your garden.  And this year, there is another threat to our plants.  A very cute, furry threat that has been increasing in numbers rapidly:

A sign of global warming?  I will leave that for a future blog.

Thursday, May 10, 2018

Will Hawaii's Vog Head Our Way?

Bellamy Pailthorp, my colleague at KNKX, and I have gotten several emails from folks worried about the effluent from Hawaii's Kilauea volcano and whether it might be heading our way.

So let's talk about it
. A major eruption is currently occurring on the Big Island of Hawaii associated with the Kilauea volcano. This eruption, really an enhancement of an ongoing eruptive event starting 1983, is not explosive and putting only minor amounts of ash into the atmosphere. But it is emitting large amounts of sulfur dioxide (SO2), hydrogen sulfide (H2S), water vapor, and other gases.

The amount of SO2 emitted by Kilauea is immense, with some days the emissions reaching 2000 tons. SO2 can combine with water to produce sulfuric acid and sulfate particles. SO2, sulfuric acid particles, and other volcano effluents can combine to for a grayish haze called VOG, which can not only reduce visibility, but can have detrimental effects on lung function.

And with this eruption that have been plenty of small to moderate earthquakes (see below).

Up until this this point, the northeasterly trade winds have been blowing most of the VOG out to sea, leaving air quality pretty decent on the big island (see below).

But changes in the trade winds may bring the volcanic effluent soon into more populated areas. A friend of mine, Professor Steve Businger, of the University of Hawaii, runs a VOG prediction effort, in which they use numerical models to predict the movement of volcano effluent around the Hawaiian islands.

Let me show you their model-based probabilities of SO2 concentrations exceeding .01 parts per million (ppm) for the next day.  The map from this morning (around 10 AM PDT) shows most heading out to sea.

But as we view the forecasts for the upcoming days, more of the SO2 is swinging around the big island and heading into the heavily populated Kona area. Some even swings up to Oahu.

One thing is clear...none of the SO2 and VOG are headed our way, so we can relax.

Hawaii is a beautiful place, but it has its environmental challenges--like very heavy rain (e.g., on Kauai), VOG, earthquakes, tsunamis, and more.