基于深度强化学习的氢燃料电池电动拖拉机能量管理策略

Deep Reinforcement Learning Energy Management Strategy for Hydrogen Fuel Cell Electric Tractors

摘要:

针对氢燃料电池电动拖拉机（Hydrogen fuel cell electric tractor，HFCET）能量管理策略在线运行时对工况适应性差的问题，提出一种基于深度Q网络（Deep Q-networks，DQN）学习的混合能量管理策略。将深度强化学习方法用于氢燃料电池（Hydrogen fuel cell，HFC）电动拖拉机对提高燃料经济性和延长燃料电池使用寿命具有重要作用。首先，以燃料电池氢耗量为目标，将Q-学习算法与DQN算法进行对比，并与动态规划（Dynamic programming，DP）方法进行比较。将燃料电池性能退化因子纳入目标函数，通过调整性能退化因子与氢耗量实现氢燃料电池经济性和系统性能退化之间的动态平衡。通过电动拖拉机实际运行工况验证所提策略的有效性。实际运行工况试验结果表明，在训练中纳入氢燃料电池性能退化因子时，能量管理策略（Energy management strategy，EMS）能耗下降2.46%，达到实际运行工况DP方法EMS的87.63%，有效抑制了氢燃料电池性能衰退。同时，与DP方法相比，计算效率提高78%以上。

Abstract:

In order to solve the problem of poor adaptability of energy management strategy for hydrogen fuel cell electric tractor (HFCET) when running online, a hybrid energy management strategy was proposed based on deep Q-networks (DQN) learning. Applying deep reinforcement learning method to hydrogen fuel cell (HFC) electric tractor played an important role in improving the fuel cell economy and prolonging the service life of fuel cell. Firstly, Q-learning method was compared with deep Q-networks learning method and dynamic programming (DP) method with the fuel cell hydrogen consumption as the target. Secondly, the fuel cell performance degradation factor was incorporated into the objective function, and the dynamic balance between the hydrogen fuel cell economy and system performance degradation was achieved by adjusting the performance degradation factor and hydrogen consumption. Finally, the effectiveness of the proposed strategy was verified by the actual operating condition of the electric tractor. The experimental results of actual operating conditions showed that when hydrogen fuel cell performance degradation factor was included in the training, the energy consumption of energy management strategy (EMS) was decreased by 2.46%, reaching 87.63% of the actual operating condition of dynamic programming method energy management strategy, which effectively inhibited the decline of hydrogen fuel cell performance. At the same time, compared with dynamic programming method, the computational efficiency was increased more than 78%.