IBM improves solar energy forecasting technology

The stigma about the solar energy forecasting is over. That old argument about solar energy being unreliable is getting weaker by the minute, and here comes IBM to knock the pins right out from under it. The company has come up with a Big Data approach to predicting the weather, and the result is a solar energy forecasting system that is up to 30% more accurate than the next-best conventional system. That’s huge news for utilities and other electrical system operators, because it helps them ensure a reliable supply of power while integrating more solar energy.

Solar energy forecasting by a very thoughtful machine

IBM Research revealed that solar and wind forecasts it is producing using machine learning and other cognitive computing technologies are proving to be as much as 30 percent more accurate than ones created using conventional approaches.  Part of a research program funded by the U.S. Department of Energy’s SunShot Initiative, the breakthrough results suggest new ways to optimize solar resources as they are increasingly integrated into the nation’s energy systems.

Entering the third year, IBM researchers worked with academic, government and industry collaborators to develop a Self-learning weather Model and renewable forecasting Technology, known as SMT. The SMT system uses machine learning, Big Data and analytics to continuously analyze, learn from and improve solar forecasts derived from a large number of weather models. In contrast, most current forecasting techniques rely on individual weather models that offer a more narrow view of the variables that affect the availability of renewable energy.

IBM’s approach provides a general platform for renewable energy forecasting, including wind and hydro. It advances the state-of-the-art by using deep machine learning techniques to blend domain data, information from sensor networks and local weather stations, cloud motion physics derived from sky cameras and satellite observations, and multiple weather prediction models. The SMT system represents the first time such a broad range of forecasting methods have been integrated onto a single, scalable platform.

“By continuously training itself using historical records from thousands of weather stations and real time measurements, IBM’s system combines predictions from a number of weather models with geographic information and other data to produce the most accurate forecasts — from minutes to weeks ahead,» explained Dr. Siyuan Lu, Physical Analytics Researcher at IBM.

“By improving the accuracy of forecasting, utilities can operate more efficiently and profitably. That can increase the use of renewable energy sources as a more accepted energy generation option,” said Dr. Bri-Mathias Hodge, who oversees the Transmission and Grid Integration Group at the National Renewable Energy Laboratory (NREL), a collaborator in the project.

In 2013, solar was the second-largest source of new electricity generating capacity in the U.S., exceeded only by natural gas.  A USA SunShot Vision Study suggests that solar power could provide as much as 14% of U.S. electricity demand by 2030 and 27% by 2050.

Currently, there are two main customers for renewable energy forecasting technologies: utility companies and independent system operators (ISOs). However, the inherent difficulty in producing accurate solar and wind forecasts has required electric utilities to hold higher amounts of energy reserves as compared to conventional energy sources. With solar power installations rapidly growing, future solar penetration levels will soon require increased attention to the value of more accurate solar forecasting.

“Solar photovoltaic resources have expanded dramatically in New England in the last five years, going from just 44 megawatts to 1,000 megawatts,” said Jonathan Black, lead engineer on ISO New England’s solar PV forecasting efforts and a collaborator in the project. “Currently, most of the solar installations in New England are ‘behind the meter’ on the distribution system, so their output isn’t ‘visible’ in real time to the ISO’s system operators, but it reduces the amount of electricity demand they observe. The growing aggregate output from all these resources across our region will increasingly change the daily demand curve, so the ISO will need accurate solar forecasts to help grid operators continue to balance power generation and consumer demand.”

SMT is a kitchen sink approach that pulls together — for the first time — different kinds of forecasting systems:

It advances the state-of-the-art by using deep machine learning techniques to blend domain data, information from sensor networks and local weather stations, cloud motion physics derived from sky cameras and satellite observations, and multiple weather prediction models.

The “deep thinking” comes in as the system continuously cycles through a combination of real-time measurements and historical records, drawing on thousands of weather stations.

Don’t run out shopping for your very own SMT just yet, though. The system, which is a collaborative effort between IBM and the National Renewable Energy Laboratory (NREL) among other partners, is still in a preliminary phase.

For that matter, if you check out IBM’s video, you’ll see that the collaboration is aiming for a 50% gain in accuracy, as well as providing a platform for all wind and hydro prediction so might as well wait until version 2.0 happens:

What’s the big deal about solar forecasting in the US?

Conventional solar energy forecasting systems may be adequate for now, but that’s going to change. According to IBM, around 27% of the nation’s electricity demand could come from solar energy by 2050. A good chunk of that will be distributed solar systems, and that’s part of the problem.

For those of you new to the topic, distributed solar refers to relatively small arrays located on rooftops and other rather small properties. Until the smart grid becomes truly brilliant, it is difficult for utilities and other system operators to accurately gauge the output and demand related to these “behind the meter” arrays.

When distributed solar systems are few and far between, there is little effect on overall demand, but distributed solar is rapidly penetrating the market. Grid operators will in effect be flying blind unless more accurate forecasting models are available.

In the context of natural gas impacts (including transportation and storage issues as well as fracking), the overall benefit of more accurate solar forecasting is clear. You get a threefer: a more reliable supply of power, more efficient use of solar energy resources, and less reliance on fossil power plants.

For New England, for example, the ripple effect also includes a reduction, if not the elimination, of the need to build disruptive new gas pipelines and storage facilities.

As for the Energy Department’s involvement, the SMT project comes under the agency’s SunShot program, which aims to bring the cost of solar energy down to parity with fossil fuels.

SunShot is pushing hard for an increase in distributed solar energy, and SMT is part of a package of initiatives that provide solutions to the issues raised by these “behind the meter” systems. Energy storage, smart grid, and microgrid solutions are also on the table.