Computer Vision based Solar Irradiance Forecasting

Project Overview

This research developed a comprehensive hybrid deep learning pipeline for very short-term solar irradiance forecasting, targeting a 20-minute prediction horizon. The system integrates multimodal data sources—high-resolution fish-eye all-sky imagery and time-series meteorological observations—to achieve robust performance under dynamic cloud cover conditions.

Key Components and Methodology

1. Data Preprocessing and Image Correction
   Fish-eye camera images were undistorted using a cosine-weighted hemispheric sampling technique. This method preserves angular fidelity across the hemispherical field of view, ensuring accurate representation of cloud positions and solar disk location critical for irradiance estimation.

2. Cloud Detection and Segmentation
   A multi-stage cloud identification pipeline was implemented:
   • UcloudNet: A lightweight U-Net variant trained for semantic segmentation of cloud pixels.
   • Red-to-Blue Ratio (RBR) Thresholding: Physics-based clear-sky reference model for binary mask generation.
   • Hybrid Fusion: Ensemble decision logic combining segmentation confidence with RBR to produce high-reliability binary cloud masks.
     Comparative ablation studies validated the superiority of the hybrid approach in minimizing false positives under thin cirrus and broken cloud regimes.
3. Dual-Stream Deep Learning Architecture
   • Vision Branch: Sequences of 5 consecutive undistorted all-sky images (1-minute intervals) processed by a Vision Transformer (ViT) encoder to extract spatio-temporal cloud motion features.
   • Time Series Branch: 30-minute historical window of meteorological variables (GHI, DNI, DHI, ambient temperature, barometric pressure, solar zenith/azimuth angles) encoded via a Time Series Transformer (TST).
   • Fusion and Regression: Feature vectors from both branches concatenated and passed through a multilayer perceptron (MLP) head for multi-step GHI and DNI regression.
4. Training and Evaluation
   The model was trained end-to-end on a dataset collected from a tropical rooftop station in Sri Lanka (2023–2024), comprising over 150,000 synchronized image–measurement pairs. Performance was benchmarked against persistence, smart persistence, and single-modality baselines using MAE, RMSE, and FS (forecast skill) metrics.

Technical Contributions

• Designed and implemented the complete end-to-end deep learning pipeline, including data ingestion, preprocessing, model architecture, and evaluation.
• Developed custom undistortion and hybrid cloud segmentation modules optimized for real-time deployment.
• Conducted extensive ablation studies on distortion correction methods and segmentation algorithms to quantify their impact on forecast accuracy.
• Demonstrated state-of-the-art performance for 20-minute ahead GHI/DNI forecasting in highly variable tropical sky conditions.

Visual Results

Acknowledgments

This work was conducted with the guidance from Prof. Roshan Godaliyadda, Prof. Parakrama Ekanayake from the Department of Electrical and Electronic Engineering, University of Peradeniya.