Adaptive Dynamic Programming and Its Application to Economic Dispatch in Microgrid: A Brief Overview
Abstract - 225
PDF

Keywords

Microgrid
Optimal control
Economic dispatch
Adaptive dynamic programming

How to Cite

Chen, Z., Deng, Q., & Chen, K. (2022). Adaptive Dynamic Programming and Its Application to Economic Dispatch in Microgrid: A Brief Overview. Journal of Advances in Applied & Computational Mathematics, 9, 13–31. https://doi.org/10.15377/2409-5761.2022.09.2

Abstract

Both adaptive dynamic programming and other intelligent algorithms can solve the economic dispatch problem in the microgrid. Adaptive dynamic programming can reduce the computational burden, which the intelligent algorithms suffer from, by using function approximation structure to approximate performance index function. In recent years, it has been also widely used in economic dispatch in the microgrid. In this article, we introduce some recent research trends within the field of adaptive dynamic programming based economic dispatch. Adaptive dynamic programming is firstly reviewed. Then, the current research works about adaptive dynamic programming based economic dispatch are summarized and compared. Furthermore, we point out some topics for future studies.

https://doi.org/10.15377/2409-5761.2022.09.2
PDF

References

Werbos PJ. Computational intelligence for the smart grid-history, challenges, and opportunities. IEEE Computational Intelligence Magazine, 2011; 6(3): pp. 14-21. https://doi.org/10.1109/MCI.2011.941587

Venayagamoorthy GK. Dynamic, stochastic, computational, and scalable technologies for smart grids. IEEE Computational Intelligence Magazine, 2011; 6(3): pp. 22-35. https://doi.org/10.1109/MCI.2011.941588

Nguyen DT, Le LB. Optimal bidding strategy for microgrids considering renewable energy and building thermal dynamics. IEEE Transactions on Smart Grid, 2014; 5(4): pp. 1608-1620. https://doi.org/10.1109/TSG.2014.2313612

Talari S, Yazdaninejad M, Haghifam M. Stochastic-based scheduling of the microgrid operation including wind turbines, photovoltaic cells, energy storages and responsive loads. IET Generation, Transmission Distribution, 2015; 9(12): pp. 1498-1509. https://doi.org/10.1049/iet-gtd.2014.0040

Changjin X, et al. Further exploration on bifurcation of fractional-order six-neuron bi-directional associative memory neural networks with multi-delays. Applied Mathematics and Computation, 2021; 410: 126458. https://doi.org/10.1016/j.amc.2021.126458

Chen Z, Wang J, Ma K, et al. Fuzzy adaptive two‐bits‐triggered control for nonlinear uncertain system with input saturation and output constraint. International Journal of Adaptive Control and Signal Processing, 2020; 34(4): 543-559. https://doi.org/10.1002/acs.3098

Changjin X, et al. Bifurcation Dynamics in a Fractional-Order Oregonator Model Including Time Delay. Match-Communications in Mathematical and in Computer Chemistry, 2022; 87(2): 397-414. https://doi.org/10.46793/match.87-2.397X

Changjin X, et al. Bifurcation control strategy for a fractional-order delayed financial crises contagions model. AIMS Mathematics, 2022; 7(2): 2102-2122. https://doi.org/10.3934/math.2022120

Lewis FL. Vrabie D. Reinforcement learning and adaptive dynamic programming for feedback control. IEEE circuits and systems magazine, 2009; 9(3): pp. 32-50. https://doi.org/10.1109/MCAS.2009.933854

Lewis FL, Vrabie D, Vamvoudakis KG. Reinforcement learning and feedback control: Using natural decision methods to design optimal adaptive controllers. IEEE Control Systems Magazine, 2012; 32(6): pp. 76-105. https://doi.org/10.1109/MCS.2012.2214134

Nguyen TA, Crow ML. Stochastic optimization of renewablebased microgrid operation incorporating battery operating cost. IEEE Transactions on Power Systems, 2016; 31(3): pp. 2289-2296. https://doi.org/10.1109/TPWRS.2015.2455491

Werbos P. Advanced forecasting methods for global crisis warning and models of intelligence. General System Yearbook, 1977; pp. 25-38.

Wei Q, Song R, Li B, Lin X. Self-Learning Optimal Control of Nonlinear Systems. Springer, 2018; 103: https://doi.org/10.1007/978-981-10-4080-1

Si J, Wang Y-T. Online learning control by association and reinforcement. IEEE Transactions on Neural networks, 2001; 12(2): pp. 264-276. https://doi.org/10.1109/72.914523

Al-Tamimi A, Lewis FL, Abu-Khalaf M. Discrete-time nonlinear hjb solution using approximate dynamic programming: Convergence proof. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2008; 38(4): pp. 943-949. https://doi.org/10.1109/TSMCB.2008.926614

Wang D, Liu D, Wei Q, Zhao D, Jin N. Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming. Automatica, 2012; 48(8): pp. 1825-1832. https://doi.org/10.1016/j.automatica.2012.05.049

Wei Q, Liu D, Lin H. Value iteration adaptive dynamic programming for optimal control of discrete-time nonlinear systems. IEEE Transactions on cybernetics, 2015; 46(3): pp. 840-853. https://doi.org/10.1109/TCYB.2015.2492242

Wei Q, Lewis FL, Liu D, Song R, Lin H. Discrete-time local value iteration adaptive dynamic programming: Convergence analysis. 2018; 48(6): pp. 875-891. https://doi.org/10.1109/TSMC.2016.2623766

Liu D, Wei Q. Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems, 2013; 25(3): pp. 621-634. https://doi.org/10.1109/TNNLS.2013.2281663

Wang J, Zhang H, Ma K, Liu Z, Chen CLP. Neural Adaptive Self-Triggered Control for Uncertain Nonlinear Systems With Input Hysteresis. IEEE transactions on neural networks and learning systems, 2021; https://doi.org/10.1109/TNNLS.2021.3072784

Wang J, Gong Q, Huang K, et al. Event-triggered Prescribed Settling Time Consensus Compensation Control for a Class of Uncertain Nonlinear Systems with Actuator Failures, IEEE transactions on neural networks and learning systems, 2021; https://doi.org/10.1109/TNNLS.2021.3129816

Xue S, Luo B, Liu D. Event-triggered adaptive dynamic programming for zero-sum game of partially unknown continuous-time nonlinear systems. IEEE Transactions on Systems, Man, and Cybernetics, 2019; pp. 1-11.

Vamvoudakis KG, Lewis FL, Hudas G. Multi-agent differential graphical games: Online adaptive learning solution for synchronization with optimality. Automatica, 2012; 48(8): pp. 1598-1611. https://doi.org/10.1016/j.automatica.2012.05.074

Abouheaf M, Lewis FL, Vamvoudakis KG, Haesaert S, Babuska R. Multi-agent discrete-time graphical games and reinforcement learning solutions. Automatica, 2014; 50(12): pp. 3038-3053. https://doi.org/10.1016/j.automatica.2014.10.047

Wei Q, Liu D. Data-driven neuro-optimal temperature control of water-gas shift reaction using stable iterative adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 2014; 61(11): pp. 6399-6408. https://doi.org/10.1109/TIE.2014.2301770

Huang T, Liu D. A self-learning scheme for residential energy system control and management. Neural Computing and Applications, 2013; 22(2): pp. 259-269. https://doi.org/10.1007/s00521-011-0711-6

Boaro M, Fuselli D, De Angelis F, Liu D, Wei Q, Piazza F. Adaptive dynamic programming algorithm for renewable energy scheduling and battery management. Cognitive Computation, 2013; 5(2): pp. 264-277. https://doi.org/10.1007/s12559-012-9191-y

Fuselli D, De Angelis F, Boaro M, Squartini S, Wei Q, Liu D, Piazza F. Action dependent heuristic dynamic programming for home energy resource scheduling. International Journal of Electrical Power & Energy Systems, 2013; 48: pp. 148-160. https://doi.org/10.1016/j.ijepes.2012.11.023

Xu Y, Liu D, Wei Q. Action dependent heuristic dynamic programming based residential energy scheduling with home energy interexchange. Energy Conversion and Management, 2015; 103: pp. 553-561. https://doi.org/10.1016/j.enconman.2015.06.048

Liu D, Xu Y, Wei Q, Liu X. Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming. IEEE/CAA Journal of Automatica Sinica, 2017; 5(1): pp. 36-46. https://doi.org/10.1109/JAS.2017.7510739

Shuai H, Fang J, Ai X, Wen J, He H. Optimal real-time operation strategy for microgrid: An adp-based stochastic nonlinear optimization approach. IEEE Transactions on Sustainable Energy, 2018; 10(2): pp. 931-942. https://doi.org/10.1109/TSTE.2018.2855039

Shuai H, Fang J, Ai X, Tang Y, Wen J, He H. Stochastic optimization of economic dispatch for microgrid based on approximate dynamic programming. IEEE Transactions on Smart Grid, 2018; 10(3): pp. 2440-2452. https://doi.org/10.1109/TSG.2018.2798039

Wei Q, Liao Z, Song R, Zhang P, Wang Z, Xiao J. Self-Learning Optimal Control for Ice-Storage Air Conditioning Systems via Data-Based Adaptive Dynamic Programming. in IEEE Transactions on Industrial Electronics, 2021; 68(4): pp. 3599-3608. https://doi.org/10.1109/TIE.2020.2978699

Wei Q, Liu D, Liu Y, Song R. Optimal constrained self-learning battery sequential management in microgrid via adaptive dynamic programming. IEEE/CAA Journal of Automatica Sinica, 2016; 4(2): pp. 168-176. https://doi.org/10.1109/JAS.2016.7510262

Wei Q, Shi G, Song R, Liu Y. Adaptive dynamic programmingbased optimal control scheme for energy storage systems with solar renewable energy. IEEE Transactions on Industrial Electronics, 2017; 64(7): pp. 5468-5478. https://doi.org/10.1109/TIE.2017.2674581

Shi G, Liu D, Wei Q. Echo state network-based q-learning method for optimal battery control of offices combined with renewable energy. IET Control Theory & Applications, 2017; 11(7): pp. 915-922. https://doi.org/10.1049/iet-cta.2016.0653

Zhu Y, Zhao D, Li X, Wang D. Control-limited adaptive dynamic programming for multi-battery energy storage systems. IEEE Transactions on Smart Grid, 2019; 10(4): pp. 4235-4244. https://doi.org/10.1109/TSG.2018.2854300

Lewis FL, Vrabie D, Syrmos VL. Optimal control. John Wiley & Sons, 2012. https://doi.org/10.1002/9781118122631

Lewis FL, Vamvoudakis KG. Reinforcement learning for partially observable dynamic processes: Adaptive dynamic programming using measured output data. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2010; 41(1): pp. 14-25. https://doi.org/10.1109/TSMCB.2010.2043839

Vrabie D, Lewis F. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks, 2009; 22(3): pp. 237-246. https://doi.org/10.1016/j.neunet.2009.03.008

Liu D, Wei Q. Finite-approximation-error-based optimal control approach for discrete-time nonlinear systems. IEEE Transactions on Cybernetics, 2013; 43(2): pp. 779-789. https://doi.org/10.1109/TSMCB.2012.2216523

Wei Q, Liu D, Shi G. A novel dual iterative q-learning method for optimal battery management in smart residential environments. IEEE Transactions on Industrial Electronics, 2014; 62(4): pp. 2509-2518. https://doi.org/10.1109/TIE.2014.2361485

Wei Q, Liu D, Shi G, Liu Y. Multibattery optimal coordination control for home energy management systems via distributed iterative adaptive dynamic programming. IEEE Transactions on Industrial Electronics, 2015; 62(7): pp. 4203-4214. https://doi.org/10.1109/TIE.2014.2388198

Venayagamoorthy GK, Sharma RK, Gautam PK, Ahmadi A. Dynamic energy management system for a smart microgrid. IEEE transactions on neural networks and learning systems, 2016; 27(8): pp. 1643-1656. https://doi.org/10.1109/TNNLS.2016.2514358

Wei Q, Lewis FL, Shi G, Song R. Error-tolerant iterative adaptive dynamic programming for optimal renewable home energy scheduling and battery management. IEEE Transactions on Industrial Electronics, 2017; 64(12): pp. 9527-9537. https://doi.org/10.1109/TIE.2017.2711499

Wei Q, Liu D, Lewis FL, Liu Y, Zhang J. Mixed iterative adaptive dynamic programming for optimal battery energy control in smart residential microgrids. IEEE Transactions on Industrial Electronics, 2017; 64(5): pp. 4110-4120. https://doi.org/10.1109/TIE.2017.2650872

Hong SH, Yu M, Huang X. A real-time demand response algorithm for heterogeneous devices in buildings and homes. Energy, 2015; 80: pp. 123-132. https://doi.org/10.1016/j.energy.2014.11.053

Yang X, Zhang Y, He H, Ren S, Weng G. Real-time demand side management for a microgrid considering uncertainties. IEEE Transactions on Smart Grid, 2019; 10(3): pp. 3401-3414. https://doi.org/10.1109/TSG.2018.2825388

Wang F-Y, Jin N, Liu D, Wei Q. Adaptive dynamic programming for finite-horizon optimal control of discrete-time nonlinear systems with -error bound. IEEE Transactions on Neural Networks, 2010; 22(1): pp. 24-36. https://doi.org/10.1109/TNN.2010.2076370

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Copyright (c) 2022 Zitao Chen, Quanbin Deng, Kairui Chen