Extending intraday solar forecast horizons with deep generative models

Surface solar irradiance (SSI) plays a crucial role in tackling climate change-as an abundant, non-fossil energy source, exploited primarily via photovoltaic (PV) energy production. With the growing contribution of SSI to total energy production, the stability of the latter is challenged by the intermittent character of the former, arising primarily from cloud effects. Mitigating this stability challenge requires accurate, uncertaintyaware, near real-time, regional-scale SSI forecasts with lead times of minutes to a few hours, enabling robust real-time energy grid management. State-of-the-art nowcasting methods typically meet only some of these requirements. Here we present SHADECast, a deep generative diffusion model for the probabilistic spatiotemporal nowcasting of SSI, conditioned on deterministic aspects of cloud evolution to guide the probabilistic ensemble forecast, and based on near real-time satellite data. We demonstrate that SHADECast provides improved forecast quality, reliability, and accuracy in all weather scenarios. Our model produces realistic and spatiotemporally consistent predictions extending the state-of-the-art forecast horizon by 26 min over different regions with lead times of 15-120 min. Our physics-informed generative approach leads to up to 60% performance improvement in extreme value prediction over the state-of-the-art deterministic models, showcasing the advantage of probabilistic modeling of cloudiness over the classical deterministic approach. It also surpasses the probabilistic benchmarks in predicting extreme values. Finally, SHADECast empowers grid operators and energy traders to make informed decisions, ensuring stability and facilitating the seamless integration of PV energy across multiple locations simultaneously.