skip to main content

Photovoltaic power prediction based on sky images and tokens-to-token vision transformer

1Power Dispatching and Control Center, State Grid Jiangsu Electric Power Co. Ltd., Nanjing, China

2Power Dispatching and Control Center, State Grid Taizhou Electric Power Co. Ltd, Taizhou, China

Received: 4 Aug 2023; Revised: 3 Oct 2023; Accepted: 19 Oct 2023; Available online: 24 Oct 2023; Published: 1 Nov 2023.
Editor(s): H Hadiyanto
Open Access Copyright (c) 2023 The Author(s). Published by Centre of Biomass and Renewable Energy (CBIORE)
Creative Commons License This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Citation Format:

Photovoltaic (PV) power generation has high uncertainties due to the randomness and imbalance nature of solar energy and meteorological parameters. Hence, accurate PV power forecasts are essential in the operation of PV power plants (PVPP) for short-term dispatches and power generation schedules. In this paper, a new deep neural network structure based on vision transformer is proposed to combine sky images and Tokens-To-Token(T2T) for photovoltaic power prediction. The method uses an incremental tokenization module to aggregate neighboring image patches into tokens, which capture the local structural information of the clouds. Then, an efficient T2T-ViT backbone network is used to extract the global attentional relationships of the tokens for power prediction. In order to evaluate the performance of the proposed model, the method was compared with several deep learning architectures such as ResNet and GoogleNet on a dataset collected by the National Renewable Energy Laboratory in Colorado, USA. The results of power prediction were analysed using training loss, prediction error, and linear regression, and they show that the proposed method achieves higher prediction accuracy and lower error compared to the existing methods, especially in short- and ultra-short-term prediction. The paper demonstrates the potential of applying Transformer models to computer vision tasks for renewable energy forecasting. The results show that the proposed method achieves higher prediction accuracy and lower error than several deep learning architectures, such as ResNet and GoogleNet, especially in short- and ultra-short-term prediction.

Fulltext View|Download
Keywords: Photovoltaic power prediction; short term prediction; sky image; Deep learning; T2T-ViT

Article Metrics:

  1. Agoua, X. G., Girard, R., & Kariniotakis, G. (2018). Short-term spatio-temporal forecasting of photovoltaic power production. IEEE Trans. Sustain. Energy, 9(2), 538–546. https.//
  2. Ajith, M. & Martínez-Ramón, M. (2021). Deep learning based solar radiation micro forecast by fusion of infrared cloud images and radiation data, Applied Energy, 294.
  3. Bi, J., Zhu Z., & Meng, Q. (2021). Transformer in Computer Vision. 2021 IEEE International Conference on Computer Science, Electronic Information Engineering and Intelligent Control Technology (CEI), Fuzhou, China, pp. 178-188
  4. Biswas, A. K., Ahmed, S. I., Bankefa, T., Ranganathan, P., and Salehfar, H. (2021). Performance Analysis of Short and Mid-Term Wind Power Prediction using ARIMA and Hybrid Models. IEEE Power and Energy Conference at Illinois (PECI), Urbana, IL, USA, 2021.
  5. Bochie, K., Gilbert, M. S., Gantert, L., Barbosa, M. S. M., Medeiros, D. S. B., & Campista, M. E. M. (2021). A survey on deep learning for challenged networks: Applications and trends, Journal of Network and Computer Applications, 194, 2021.
  6. Boland, J., David M, & Lauret P. (2016). Short term solar radiation forecasting: island versus continental sites. Energy .113:186e92.
  7. Burnham, K., Anderson D., & Huyvaert KK. (2011). Aic model selection and multimodel inference in behavioral ecology: some background, observations, and comparisons. Behav Ecol Sociobiol.65:23e35.
  8. Dolara, A., Leva, S., & Manzolini, G. (2015). Comparison of different physical models for pv power output prediction. Sol Energy. 119:83e99.
  9. Fan, H., Xiong, B., Mangalam, K., Li, Y., Yan, Z., Malik, J., & Feichtenhofer, C. (2021). Multiscalevision transformers. In: Proceedings of the IEEE/CVF international conference oncomputer vision.
  10. Fukushima, D. & Ishikawa, T. (2022). Experiments And Discussions On Vision Transformer (ViT) Parameters For Object Tracking, 2022 Nicograph International (NicoInt), Tokyo, Japan, pp. 64-68
  11. Gao, Y., Shi, S., Sun, Z. , & Ling, C.. (2022). The combination of transformer and CNN in computer vision," 2022 IEEE 4th International Conference on Civil Aviation Safety and Information Technology (ICCASIT), Dali, China, pp. 321-325
  12. Graditi, G., Ferlito, S., & Adinolfi, G. (2016). Comparison of Photovoltaic plant power production prediction methods using a large measured dataset, Renew Energy, 90, 513-519.
  13. Han, K., Wang, Y., Chen, H., Chen, X., Guo, J., Liu, Z., Tang, Y., Xiao, A., Xu, C., Xu, Y., Yang, Z., Zhang, Y., Tao, D. (2022). A Survey on Vision Transformer. in IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 87-110.
  14. Jaouhari, Z. El, Zaz, Y., & Masmoudi, L. (2015). Cloud tracking from whole-sky ground-based images," 2015 3rd International Renewable and Sustainable Energy Conference (IRSEC), Marrakech, Morocco, pp. 1-5
  15. Lauret, P., Voyant, C., Soubdhan, T., David, M., Poggi, P. (2015). A benchmarking of machine learning techniques for solar radiation forecasting in an insular context, Sol Energy, 112, 446-457.
  16. Liu, J., Zang, H., Cheng, L., Ding, T., Wei, Z., & Sun, G., (2023). A Transformer-based multimodal-learning framework using sky images for ultra-short-term solar irradiance forecasting, Applied Energy, 342, 121160,
  17. Liu, S. (2023). Application and Analysis of Convolutional Neural Networks and Vision Transformer Models in Fruit Recognition, 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA), Shenyang, China, pp. 1530-1533.
  18. Liu, W., Ren, C., & Xu, Y. (2021). PV generation forecasting with missing input data. A super-resolution perception approach. IEEE Trans. Sustain. Energy, 12(2), 1493–1496. https.//
  19. Limouni, T., Yaagoubi, R. (2022). Univariate and Multivariate LSTM Models for One Step and Multistep PV Power Forecasting. International Journal of Renewable Energy Development, 11(3), 815-828.
  20. Lu, Z., Wang, K., & Li, X. (2021). A Preprocessing and Feature Extraction Method of Ground-based Cloud Images for Photovoltaic Power Prediction. 2021 40th Chinese Control Conference (CCC), Shanghai, China, pp. 6823-6828
  21. Ma, D., (2022). Recent advances in deep learning based computer vision. 2022 International Conference on Computers, Information Processing and Advanced Education (CIPAE), Ottawa, ON, Canada, pp. 174-179
  22. Nascimento E. G. S., Talison A.C. de Melo, Davidson M. Moreira. (2023). A transformer-based deep neural network with wavelet transform for forecasting wind speed and wind energy. Energy.
  23. Nhat, N. N. V., Huu, D. N. (2023). Evaluating the EEMD-LSTM model for short-term forecasting of industrial power load: A case study in Vietnam. International Journal of Renewable Energy Development, 12(5), 881-890
  24. Nie, Y., & Zamzam, Ahmed S. (2021). Adam Brandt,Resampling and data augmentation for short-term PV output prediction based on an imbalanced sky images dataset using convolutional neural networks, Solar Energy, 224, 341-354.
  25. Peng, Z., Yu, D., Huang, D., Heiser, J., Yoo, S., &Kalb, P. (2015). 3D cloud detection and tracking system for solar forecast using multiple sky imagers, Sol. Energy 118, 496–519.
  26. Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., & Barrow, D. K. (2022). Souhaib Ben Taieb Forecasting: theory and practice, International Journal of Forecasting, 38(3), 705-871.
  27. Qu, J., Qian, Z., & Pei, Y., (2021). Day-ahead hourly photovoltaic power forecasting using attention-based CNN-LSTM neural network embedded with multiple relevant and target variables prediction pattern, Energy, 232.
  28. Ragmani, A., Elomri, A., Abghour, N., Moussaid, K., Rida, M., & Badidi, E. (2020). Adaptive fault-tolerant model for improving cloud computing performance using artificial neural network, Procedia Computer Science, 170. 929-934.
  29. Rizk, Y., & Awad, M.(2019).On extreme learning machines in sequential and time series prediction. A non-iterative and approximate training algorithm for recurrent neural networks, Neurocomputing, 325. 1-19,
  30. Smith, A. (2018). Machine Learning – A Review. Journal of Artificial Intelligence Research, 45(3). 589-604.
  31. Sun, S., Ernst, J., Sapkota, A., Ritzhaupt-Kleissl, E., Wiles, J., Bamberger, J., Chen, T. (2014). Short term cloud coverage prediction using ground based all sky imager, 2014 IEEE International Conference on Smart Grid Communications (SmartGridComm), Venice, Italy, pp. 121-126.
  32. Sun, X., & Zhang, T. (2017). Solar Power Prediction in Smart Grid Based on NWP Data and an Improved Boosting Method. 2017 IEEE International Conference on Energy Internet (ICEI), Beijing, China. pp. 89-94.
  33. Takeda, M. & Yanai, K., (2022). Continual Learning in Vision Transformer. 2022 IEEE International Conference on Image Processing(ICIP), Bordeaux, France, pp. 616-620.
  34. Taskaya-Temizel, T., Casey, M. C. (2005). A comparative study of autoregressive neural network hybrids Neural Networks, 18 (5-6) 781-789.
  35. Trigo-González, M., Cortés-Carmona, M., Marzo, A., Alonso-Montesinos, T., Martínez-Durbán, M., López, G., Portillo, C., & Batlles, F. J. (2023). Photovoltaic power electricity generation nowcasting combining sky camera images and learning supervised algorithms in the Southern Spain, Renewable Energy, 206, 251-262
  36. Tyass, I. & Khalili, T. (2023). Wind Speed Prediction Based on Statistical and Deep Learning Models. International Journal of Renewable Energy Development, 12(2), 288-291
  37. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, AN., & Kaiser, Ł., & Polosukhin I. (2017). Attention is all you need. In: Advances in neural informationprocessing systems. pp. 5998–6008.
  38. Voyant, C., Haurant, P., Muselli, M., Paoli, C., &Nivet, M.-L. (2014). Time series modeling and large scale global solar radiation forecasting from geostationary satellites data, Sol Energy, 102, 131-142.
  39. Wang, X., Kan, M., Shan, S., & Chen, X. (2019). Fully Learnable Group Convolution for Acceleration of Deep Neural Networks. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. pp. 9041-9050.
  40. Wei, L., Zhu, T., Guo, Y., Ni C., and Zheng, Q. (2023). CloudpredNet: An Ultra-Short-Term Movement Prediction Model for Ground-Based Cloud Image. in IEEE Access
  41. Wu S. -H. & Wu, Y. -K. (2020). Probabilistic Wind Power Forecasts Considering Different NWP Models. 2020 International Symposium on Computer, Consumer and Control (IS3C), Taichung City, Taiwan. pp. 428-431.
  42. Xiao, Y., Zhang, Y., & Ni, P. (2022). Ensemble Long Short-Term Tracking with ConvNeXt and Transformer. 2022 7th International Conference on Image, Vision and Computing (ICIVC), Xi’an, China, pp. 688-693
  43. Xiong, R., Yang, Y., He, D., Zheng, K., Zheng, S., Xing, C., Zhang, H., Lan, Y., Wang, L., & Liu, T. (2020). On layer normalization in the transformer architecture. In: International conference on machine learning. PMLR. pp. 10524–33.
  44. Zhang, H., Wang, L., & Liu, J. (2020). Comparative analysis of deep learning architectures for predicting future PV power output. Solar Energy, 456(3), 78-92.
  45. Zhang, J., Verschae, R., Nobuhara, S., & Lalonde, J., (2018). Deep photovoltaic nowcasting, Solar Energy, 176. 267-276.
  46. Zhang, L., Wilson, R., Sumner, M., & Wu, Y., (2023). Advanced multimodal fusion method for very short-term solar irradiance forecasting using sky images and meteorological data: A gate and transformer mechanism approach, Renewable Energy, 216, 118952,
  47. Zhang, Y., Qian, W., Ye, Y., Li, Y., Tang, Y., Long, Y., & Duan, M. (2023). A novel non-intrusive load monitoring method based on ResNet-seq2seq networks for energy disaggregation of distributed energy resources integrated with residential houses, Applied Energy, 349, 121703.

Last update:

No citation recorded.

Last update: 2023-11-29 23:10:01

No citation recorded.