基于改进双Q学习线性自抗扰的水下机器人运动控制研究

Motion Control of Underwater Robot Based on Improved Double Q-Learning and Linear Active Disturbance Rejection Control

摘要:

在小型遥控水下机器人(Remotely operated vehicle,ROV)运动控制中存在模型较难描述、抗扰动能力差等问题,线性自抗扰控制器（Linear active disturbance rejection controller,LADRC）因对模型依赖程度低,对扰动有较好的观测补偿能力等,常被用于解决以上问题。而LADRC控制器在复杂环境中如参数固定会导致抗扰能力下降,因此提出一种基于改进双Q学习的LADRC参数自适应控制算法。采用一种新型分段式奖励函数指导参数调节方向,以加快调节速度,提高算法控制精度;引入双Q算法降低训练中单Q学习的过估计问题;根据系统运动状态实时调节LADRC参数,增强系统对外界扰动的观测补偿能力。Matlab/Simulink转艏、定深试验结果表明:双Q-LADRC相较传统LADRC与PID控制器,阶跃扰动下最大超调量降低1.54%和39.78%,稳定时间降低38.2%和48.4%,在镇定控制上具有更快的收敛速度与更低的超调量,更小的跟踪误差。通过水池试验证明了双Q-LADRC算法的有效性,能较好跟踪预期信号。

Abstract:

Small remotely operated vehicles (ROVs) often suffer from difficulties in accurate model description and weak disturbance rejection in motion control. Although the linear active disturbance rejection controller (LADRC) has been widely used because of its low model dependence and strong disturbance estimation and compensation capability,its control performance may degrade in complex underwater environments when controller parameters remain fixed. To address this problem,an adaptive LADRC parameter tuning method based on an improved double Q-learning algorithm was proposed. A novel piecewise reward function was designed to guide the parameter adjustment process,thereby accelerating convergence and improving control accuracy. Meanwhile,the double Q-learning strategy was introduced to alleviate the overestimation problem in conventional Q-learning training. The proposed method enabled online adjustment of LADRC parameters according to the system state,which enhanced the disturbance observation and compensation capability of the controller. Matlab/Simulink simulations for heading and depth control showed that,compared with conventional LADRC and PID controllers,the proposed double Q-LADRC reduced the maximum overshoot under step disturbances by 1.54% and 39.78%,respectively,and shortened the settling time by 38.2% and 48.4%,respectively. In addition,it achieved faster convergence,low overshoot,and small tracking errors in stabilization control. Pool experiments further verified the effectiveness of the proposed method,demonstrating that it can track the desired signals well and maintain satisfactory control performance.