Advancing Solar Irradiance Prediction with NVIDIA Earth-2

As global electricity demand continues to rise, traditional sources of energy are increasingly unsustainable. Energy providers are facing pressure to reduce reliance on fossil fuels while ensuring a fully supplied and stable grid. In this context, solar energy has emerged as a vital renewable resource, being one of the most abundant clean energy sources available. However, effectively harnessing solar power requires precise and reliable forecasts of surface solar irradiance (SSI).

Accurate SSI predictions enable operators to forecast solar power availability, helping to synchronize electricity production with consumption needs, especially in high-demand periods. Traditional methods of solar irradiance prediction, such as numerical weather prediction (NWP) models with radiative transfer, are expensive to run and take hours to produce a forecast. Nowcasting models, such as those based on optical flow, are faster but are limited in their ability to provide long-range, adaptive forecasts, as they depend on real-time measurements such as satellite images.

NVIDIA, in collaboration with the Institute for Atmospheric and Climate Science of ETH Zurich, has developed a new SSI model. The approach improves SSI modeling by combining advances in NWP and AI to form a model that is more scalable and robust than traditional approaches. For more details, see Data-Driven Surface Solar Irradiance Estimation Using Neural Operators at Global Scale.

Combining the powerful FourCastNet SFNO forecasting model and this new SSI model from NVIDIA enables accurate prediction of solar irradiance at global scale for multiple days without the need for real-time data that is dependent on satellite coverage (Figure 1).

Animation of a global surface solar irradiance forecast. — *Figure 1. Seven-day SSI forecast generated by NVIDIA SSI diagnostic models and FourCastNet SFNO*

These breakthroughs were made possible by NVIDIA Earth-2, a full-stack platform designed for building climate digital twins. Earth-2 provided the necessary tools to easily build a pipeline for solar irradiance prediction that seamlessly integrates with custom models such as the newly developed NVIDIA SSI model. This integrated approach enables straightforward production of high-resolution, real-time predictions that are crucial for applications in climate science, renewable energy, and weather-dependent industries.

This post demonstrates how the new SSI model can be trained with NVIDIA tools, integrated into an Earth-2 pipeline, and used to produce data that can be easily visualized and postprocessed with Earth2Studio.

NVIDIA Earth-2 reference pipeline for solar irradiance prediction

The Earth-2 platform enables fast and accurate weather and climate physics modeling that leverages NVIDIA accelerated computing capabilities. It also provides tools for training and inference of AI models for forecasting, downscaling, interpolation, and other applications. As a user, you can also interact with the output weather and climate data through an advanced set of visualization and postprocessing tools.

Figure 2 illustrates NVIDIA Earth-2 reference pipeline for solar irradiance forecasting. It begins with the the atmospheric state at time $t$ (that is, IFS analysis data), which initializes a forecasting (prognostic) model. This model generates forecasts of various atmospheric variables at time $t + \Delta$ time.

Earth-2 workflow for SSI forecasting, including icons labeled IFS analysis, atmospheric state, forecast model, diagnostic model, surface solar irradiance, downstream process, and user. — *Figure 2. NVIDIA AI Earth-2 deployment workflow applied to solar irradiance forecasting*

Often, the variable of interest may not be present in the set of forecasted variables, such as SSI. To retrieve the SSI forecast, users can leverage a diagnostic model, which acts as an estimation model and translates the atmospheric variables available into the desired target variable. The diagnostic model can, then, be integrated with different forecasting models, leveraging the recent advances in weather forecasts.

Training custom AI weather diagnostic models

NVIDIA Modulus is an open-source framework for training physics-ML models, such as AI weather prediction models. For medium-range weather forecasting, Modulus includes FourCastNet, SFNO, the NVIDIA implementation of GraphCast, Pangu-Weather, and DLWP, with other architectures being added. To make training easier, Modulus also includes pipelines to train on ERA5 and similar datasets. For more details, visit NVIDIA/modulus on GitHub to see training workflow examples and notebooks. All of the implemented architectures are optimized for training on GPUs.

Earth-2 users can train a customized diagnostic model on top of the forecast model output by using Modulus as shown in Figure 2. The model published in Data-Driven Surface Solar Irradiance Estimation Using Neural Operators at Global Scale, trained with Modulus, is an improved diagnostic model that utilizes Adaptive Fourier Neural Operators (AFNOs), enabling it to estimate 6-hourly SSI for the full globe.

Figure 3 shows the Modulus training pipeline applied for the SSI diagnostic model training. The ERA5 Reanalysis atmospheric data is used as input for the diagnostic model, which learns how to map the atmospheric variables to a new target variable. The ground-truth variable (SSI fields) can be retrieved from ERA5 or other datasets for higher accuracy, such as the satellite-based SARAH3.

Diagram of the NVIDIA Modulus pipeline for diagnostic models training, including ERA5 / SARAH3, atmospheric state, diagnostic model, target, predicted SSI, and Modulus trainer. — *Figure 3. NVIDIA Modulus training workflow for the SSI diagnostic model*

Streamlining solar irradiance inference with NVIDIA Earth2Studio

After training a custom model, the easiest way to make use of it is through NVIDIA Earth2Studio, a Python library designed for AI-driven weather model inference. Earth2Studio provides a comprehensive set of building blocks tailored for both deterministic and probabilistic weather predictions. The library supports a wide array of data sources right out of the box, including but not limited to the Climate Data Store (CDS), ARCO, the Global Forecast System (GFS), the Integrated Forecasting System (IFS), and the High-Resolution Rapid Refresh (HRRR).

One of the standout features of Earth2Studio is its flexibility in model selection. As a user, you have the option to choose from a range of pretrained models present in NVIDIA GPU Cloud (NGC). The models can be easily deployed for immediate inference (Figure 4). You can also seamlessly integrate your own custom models into the framework, such as a model trained with Modulus.

Using Earth-2, you can select from a variety of data sources, as well as a wide range of prognostic and diagnostic models from the NGC model registry (Figure 4). The different components are seamlessly integrated into the pipeline with minimal code changes.

Diagram showing the Earth2Studio inference pipeline, including access to a model registry for defining the diagnostic and prognostic models. — *Figure 4. Select from a range of datasets and select prognostic and diagnostic models from a model registry in Earth2Studio*

Running a weather forecast using Earth2Studio

The following code example in Figure 2 is a reference implementation using prebuilt NVIDIA models to forecast SSI. After importing the necessary data sources (IFS), IO operations (ZarrBackend), and models (FourCastNet SFNO, SolarRadiationAFNO) from the model registry (NGC), the prognostic and diagnostic models are loaded. At this point, users can specify a custom model. In this example, the AFNO model was trained in Modulus and added to NGC for convenience. The data source, output storage and number of timesteps are also specified. The run.diagnostic function is then called to perform the simulation. The forecasts can be then postprocessed with downscaling tools such as CorrDiff.

from earth2studio.data import IFS
from earth2studio.io import ZarrBackend
from earth2studio.models.px import SFNO
from earth2studio.models.dx import SolarRadiationAFNO

# Load the SFNO model package, including the model weights, from NGC
model = SFNO.load_model(SFNO.load_default_package())

# Load the Solar Radiation model package, including the model weights, from NGC
diagnostic_model = SolarRadiationAFNO.load_model(SolarRadiationAFNO.load_default_package()))

# Use the GFS analysis as the data source
data = GFS()

# Use a Zarr archive to store the outputs
io = ZarrBackend(file_name="solar_irradiance_prediction.zarr”)

nsteps = 8 # the number of 6-hour time steps

# Run the SSI global forecast
io = run.diagnostic(
    	["2024-06-01"], # start the forecast on 01 June 2024, 00:00 UTC
nsteps, 
prognostic_model, 
diagnostic_model, 
data, 
io
)

Earth2Studio is open source, providing tools for the scientific research community to explore and experiment with AI models for weather and climate and to foster collaboration and innovation within the community. Enterprise developers building applications can access with NVIDIA NIM microservices and task-specific diagnostic models (such as the one used in the previous example), as well as enterprise support, through NVIDIA AI Enterprise. This enterprise-level integration ensures that organizations can build scalable and performant solutions using cutting-edge AI technologies and benefit from NVIDIA expertise in AI-driven solutions.

Visualizing the weather forecast

The Earth-2 platform provides visualization pipelines leveraging NVIDIA Omniverse, a 3D development platform designed to facilitate interoperability between different applications and offers a collaborative environment where developers and designers can work together seamlessly. With the help of NVIDIA RTX real-time rendering technology, Omniverse enables the creation of high-resolution, interactive visualizations, making it ideal for a wide range of industry use cases.

Leveraging these capabilities through Earth-2 platform, you can build interactive, detailed representations of the Earth and atmosphere, facilitating better understanding and decision-making. Figure 5 shows a 7-day SSI forecast constructed with the help of NVIDIA Omniverse. High irradiance is indicated by a bright color overlaid on top of a satellite base image.

This provides a glimpse of what’s possible in terms of building a visualization pipeline for high-fidelity rendering of Earth system data. Such reference pipelines are modular and open to extension, enabling a flexible integration of sophisticated visualizations with custom applications. Learn more about tools for scientific visualization.

Animation showing an SSI forecast mapped on a 3D representation of the Earth. — *Figure 5. Seven-day SSI forecast visualized with NVIDIA Omniverse*

Get started with NVIDIA Earth-2

NVIDIA Earth-2 is a platform for accelerated training and inference of weather and climate simulation models. It provides key tools to use AI weather models and seamlessly integrates the newly developed model in the inference pipeline. The main components of the Earth-2 platform are NVIDIA Modulus for model training and Earth2Studio for model inference. Both enable you to select from a wide range of data sources, architectures, and pretrained models.

To get started with Modulus, quickly set up your environment with the Modulus start guide. To train your model, visit NVIDIA/modulus on GitHub to see a few examples.

To get started with Earth2Studio, install the Python package following the Earth2Studio Install Guide. Follow the NVIDIA/earth2studio GitHub tutorials to run your first weather forecast.