Power markets forecasting is a process of developing short- and long-term forecasts for major market zones and trading hubs in order to generate accurate forecasts for demand and forecast clearing prices in each market zone.
The 1990s saw the liberalization of power markets and the starting point, in Europe, of an Internal European Energy market that achieved security of supply, competitive prices, and enhanced services to customers. And within this market, a large variety of companies organize the production, trading, marketing, transmission, and supply of electricity. These create the electricity markets that are, in turn, powered by trading platforms for exchange members to access the market and submit orders for buying and selling power. The orders can be used to reflect supply and demand for a specific market area. Markets have also seen electricity moved across borders through Europe to increase the availability of electricity in areas where there had previously been shortages or supply inefficiencies.
In order for any prediction or forecasting in power markets to occur, there is a need for consumption information and data to feed the different models and software. There is plenty of data on key outcomes of the power sector, including access rates, generation capacity, and consumption. But data on the structural characteristics of power markets that generate outcomes is rare, and often only found in national and regional sources. These structural characteristics and how the characteristics evolve over time are important for policy design and investment opportunities, especially because understanding these characteristics can provide incentives for private investors to participate in power markets and to understand how regulations shape the development of those power markets.
For example, the World Bank has a power markets database that offers data on the structural characteristics and development stage of power markets in fifty developing and emerging markets around the world. The database draws from IFC's AIMM Framework, which evaluates markets in terms of competitiveness, integration, sustainability, and inclusiveness.
Electricity price forecasting is the basis of power markets forecasting, with a variety of methods and ideas used to generate forecasts based on demand and generation. Electricity price forecasting predicts the spot and forward prices in wholesale electricity markets, and these electricity price forecasts have become fundamental to energy companies' decision-making processes. The modeling for electricity price forecasting can be classified into six broad groups.
These are models that simulate the operation of a system of heterogenous agents interacting with each other and build the price process by matching the simulated demand and supply in the market. These models tend to focus on qualitative rather than quantitative results in order to provide insight into whether prices are above marginal costs and what influences are working on the market. The models can include cost-based models (also known as production-cost models), equilibrium or game theoretic approaches, and agent-based models. These models can be problematic when quantitative conclusions have to be drawn and when electricity prices have to be predicted with a level of precision.
These are models that try to capture the basic and economic relationships in the production and trading of electricity. The functional associations between fundamental drivers are postulated, and fundamental inputs are modeled and predicted. The fundamental drivers can include system load, weather conditions, and system parameters. The models are often generated through statistical, reduced-form, or computational intelligence techniques. However, these models tend to face two challenges: the first is a data availability and the incorporation of stochastic fluctuations of the fundamental drivers. The second challenge is, fundamental assumptions are made in the models about the relationships in the marketplace, and these assumptions can cause sensitivity in the models, especially when these assumptions are violated.
These are models that work to characterize the statistical properties of electricity prices over time, with the objective of derivatives evaluation and risk management. The main intention of these models is not to provide accurate hourly price forecasts, but to replicate the main characteristics of daily electricity prices at future time points, price dynamics, and correlations between commodity prices. If the price process used is not appropriate for capturing the main properties of electricity prices, the results from the model are likely to be unreliable. And if the model is too complex, the computational burden will prevent its use online in trading departments. These models can include:
- Spot price models, which provide a representation of the dynamics of spot prices. The drawback to these models includes the identification of the risk premium linking spot and forward prices.
- Forward price models, which allow for the pricing of derivatives but have a lack of data used for calibration to derive the properties of spot prices from the analysis of forward curves.
These are methods for forecasting the current price by using a mathematical combination of the previous prices and previous or current values of exogenous factors—typically consumption and production figures or weather variables. Two important categories of these models are additive and multiplicative models. These models differ in whether the predicted price is the sum of a number of components or the product of a number of factors. The multiplicative model, which is the less popular of the two models, can also be transformed into an additive model for log-prices. Statistical models are popular because some physical interpretation can be included in the models' components, which allows engineers and system operators to understand their behavior. Statistical models can include:
- Similar-day models
- Exponential smoothing models
- Regression models
- Time series models
- Heteroskedastic time series models
These include artificial intelligence, machine learning, non-parametric, and non-linear statistical techniques, which combine elements of learning, evolution, and non-linear statistical techniques capable of adapting to complex dynamic systems and are, in turn, sometimes regarded as intelligent. The strength of these models is their ability to handle complexity and non-linearity. The flexibility of the models can be a strength, as they can model features of electricity prices better than statistical models; it can also be a weakness, as the ability to adapt to nonlinearity and spiky behavior does not always result in better points or probabilistic forecasts. The main computational intelligence models include:
- Artificial neural networks
- Fuzzy systems
- Support vector machines
As the name suggests, hybrid models are a combination of multiple modeling and forecasting approaches and techniques. The classification is non-trivial, as most approaches to electricity price forecasting are done with a hybrid approach, such as the AleaModel from AleaSoft, which combines neural networks with a SARIMA model structure.
For the electricity market forecasting, there is a need for understanding the contribution capable from renewable energy sources, which increases the need for proper forecasting and accurate day-ahead forecasts. These models include forecasting accuracy through univariate and multivariate time series methods combined to account for the amount of energy produced, or capable of being produced, based on weather conditions (such as wind speed, cloud cover, or rainfall). As renewable energy makes a larger impact on power markets, the capability of forecasting renewable energy increases in importance.
As mentioned, wind power forecasting is focused on estimations of the expected production of one or more wind turbines, also known as wind farms, with units of kW or MW depending on the capacity of the wind farm; wind power forecasting can also be expressed in terms of energy by integrating power production over a time interval. Part of forecasting is dependent on the understanding of the relationship between wind speed and its resulting wind power. The type of terrain and regional weather patterns of the region determine the character of the wind speed, which can impact the power produced by a wind farm. Understanding this relationship on a regional scale can increase forecasting potential. Better forecasting for all renewable types, including wind power, can increase the use of renewable energy use in daily grid operations and help improve dispatch efficiency and system reliability. The correlation between wind output and predication can be relatively high, depending on the model used, with an average uncorrected error of 8.8 percent in Germany over a two-year period.
The forecasting of wind power generation may be considered at different time scales, usually from milliseconds up to a few minutes, depending on the intended applications. These are known as very short-term forecasts. Short-term approaches forecast for 48 to 72 hours and longer time scales, often up to 5 to 7 days ahead, are known as medium term forecasts while long-term forecasts work to forecast for up to or more than a month ahead. The most common approaches used in forecasting are physical, statistical, and AI-based approaches within these timescales.
Time-scales for wind power forecasting
Flow diagram of numerical weather prediction (or physical prediction approach) models.
Physical approaches to wind power forecasting use past wind power data and numerical weather prediction (NWP) in order to forecast. The main measures for developing an NWP are the choice of geographical site of a wind farm, spatial resolution, temporal resolution, and prediction horizon. These are common for all wind prediction models, but vary with the location of the wind farm.
These physical approaches and the NWP prediction are used not only for electric utilities but also for various other services, and are generally considered satisfactory for long-term forecasts. These approaches are sensitive to initial conditions, and while they can give look-ahead times for many horizons, many of these models only yield a single anticipated value for each forecast timescale and their use in stochastic optimization and risk assessment is limited.
Statistical models do not aim to describe the physical process in wind power conversion, but rather estimate a statistical relationship between relevant input data and wind power generation. This involves direct transformation of input variable into wind generation through statistical blocks, which are capable of combing inputs such as NWPs of the speed, direction, and temperature of the wind at various model levels and with on-line measurements, such as wind power, speed, or direction. The statistical block can also include several statistical linear and non-linear models, some of which are called black-box models and include most artificial intelligence based models.
Types of statistical models
As the name suggests, hybrid models are a combination of statistical and physical models, with the objective of benefiting from the advantage of each model for an optimal forecasting performance. The performance and accuracy of hybrid models depends on the availability of good NWP forecasts, the complexity of the terrain, and the availability of real-time weather and power plant data. Three combinations have been utilized to predict wind power:
- A combination of physical and statistical approaches
- A combination of models for the short-term and medium-term
- The combination of alternative statistical models
Solar power forecasting shares many of the features of wind power forecasting. In general, many of the forecasting methods fall into physical methods, in which weather data (such as temperature, pressure, terrain, and obstacles) are input into numeral weather prediction NWP models to create terrain-specific weather conditions that are, in turn, converted into energy production; and statistical models, which use historic and real-time generation data to statistically correct results derived from NWP models. However, unlike in wind power forecasting, solar forecasts also employ sky imagers (usually digital cameras which produce sky images) and satellite imaging (usually data from a network of geostationary satellites) to track and predict cloud formations at different timescales. Forecasts for solar power generation are difficult to produce and are most accurate when near real-time meter data and detailed static data are available.
One of the more important requirements for solar power forecasting is the forecast time scale demanded by the electricity value chain participants. Most industry-requested operational forecasts are intra-hour, hour ahead, day ahead, medium-term, or long-term.
Time-scales for solar power forecasting
Forecasts using numerical weather prediction (NWP) models, such as the Global Environmental Multiscale Model (GEM), Global Forecast System (GFS), North American Mesoscale Model (NAM), and European Centre for Medium Range Forecasts (ECMWF), are used to derive solar radiation forecasts. These models are on global scales and can provide forecasting for a number of variables useful for modeling solar radiation at a 6 hour refresh rate and at 1 to 3 hours granularity and forecast horizons from 48 to 180 hours. Spatial resolution of these models is variable. The average resolution is on a scale of hundreds of kilometers squared. Many of these models are tuned to a particular forecast location and use cloud cover information integrated with a clear sky model to produce more specific forecasts per location. This can result in 3 by 3 kilometer resolution with hourly updates.
Satellite-based forecasting uses geostationary Earth observing satellites to detect, characterize, track, and predict the future locations of cloud cover. These methods focus on understanding the amount of solar radiation reaching Earth's surface and that blocked by cloud cover, which impacts the intermittency of solar power production. Images from satellites can provide near 1 square kilometer resolution for cloud cover at specific locations. In forecasting applications, motion vectors are used to understand the future position of clouds and possible impact on the solar power generation. The limitations of this approach are the infrequent updates of original images, bad geographical registration of satellite images, poor understanding of cloud altitudes that pose particular problems for sunrise and sunset predictions, and challenges posed by estimating clearness index through calculation of dynamic pixel range.
Similar to satellite-based forecast approaches, total-sky imager forecasting uses near real-time detection of clouds and capacity to predict cloud movement and position in the near future. As well, presence of clouds at a particular pixel of hemispherical sky image is identified through sky imaging forecasting and thresholding algorithms. These forecasts can be used to predict cloud locations in shorter time periods, dependent on the velocity of cloud movements. The approach allows for high spatial and temporal resolution in forecasts at timescales shorter than about five minutes. Cloud forecasting errors increase with increased forecast horizons due to the high variability of cloud cover, especially at coastal sites.
Statistical approaches to solar radiation forecasting, such as log regressions and autoregressive integration moving average models, unobserved component models, transfer functions, neural networks, and hybrid models. In testing, ARIMA models obtained the best forecasting results. Linking statistical modeling with real-time data from monitoring sites could lead to better accuracy of predictions. While ground measurements can be downgraded to coarser time intervals, sub-hourly forecasts cannot be produced if observed radiation data is available in coarser resolutions.
The outputs from any of the above models must be converted into the electric energy that a particular solar photovoltaic plant will produce. This step is done with statistical approaches that try to correlate the amount of available resources with metered power output. The main advantage of these methods is that the meteorological prediction error, which is the main component of global error, might be reduced by taking into account the uncertainty of the prediction. There are also theoretical models that describe how a power plant converts meteorological resources into electric energy. These types of models are accurate, although too sensitive to the meteorological prediction error. Hybrid models are a combination of models intended to offer better performance than other models used in energy production understanding.
Hydro-electric production forecasting can be similar to other forms of electric power production; the flow of the water can be controlled and predicted with greater regularity than other types of renewable energy. The short-term forecasting model consists of three modules: the first module gives an estimation of the daily average power production, the second provides the final forecast of the hourly average power production taking into account operation strategies, and the third allows a dynamic adjustment of the first module estimation by assimilating recent historical production data. The model uses, as inputs, forecasted precipitation values from Numerical Weather Prediction tools and past recorded values of hourly electric production. The structure of the model avoids crossed-influences between the adjustments of such model due to meteorological effects and those due to the operation strategies.
With the increasing popularity of geothermal energy, there is an increased need for geothermal energy forecasting. The internal energy of Earth flows to the surface by conduction at a rate of 44.2 terrawats (TW), and is replenished by radioactive decay of minerals at a rate of 30 TW. These power rates are double the current consumption rate from all primary sources, but most of it is not recoverable. In addition, the top layer of the surface to a depth of 10 meters is heated by solar energy during the summer and releases that energy in the winter. Outside of seasonal variations, the geothermal gradient of temperatures through the crust is 25 to 30 degrees Celsius per kilometer of depth. The conductive heat flux averages 0.1 MW/Km. These values are higher nearer tectonic plate boundaries. The geothermal pump can extract heat from the ground in this case, either to provide warmth for homes or for the generation of electricity. The estimates of the potential for electricity generation vary depending on the scale of investments, with upper estimates of geothermal resources assuming wells as deep as 10 kilometers while existing geothermal wells are usually 3 kilometers deep.
