基于PPO–SAC的杂交水稻制种授粉机组非规则路径协同控制研究

Collaborative Control of Irregular Paths for Hybrid Rice Seed Production Pollination Units Based on PPO – SAC

摘要:

授粉是杂交水稻制种的关键环节,针对杂交水稻制种机械拉绳碰击式授粉在非规则田块中难以实现精准位速协同控制的问题,提出了一种基于近端策略优化(Proximal policy optimization,PPO)和软性演员评论家(Soft actor-critic,SAC)的双环强化学习机组协同控制算法。将授粉车辆的运动控制解耦为具有相对独立目标的子系统,横向上将路径曲率及其变化率融合到PPO算法框架实现对田间弯曲路径的精确跟踪,纵向上将机组成员相对纵向位置与输出转向角融合到SAC算法框架实现机组车辆位速控制,构成双环PPO–SAC强化学习算法实现授粉车辆的精准协同,通过仿真试验与模拟验证试验验证算法的有效性。双环PPO–SAC算法表现优于单框架的强化学习算法,其平均路径跟踪偏差为0.161 m,相比于单PPO算法降低29.9%,机组纵向相对距离偏差相比于单SAC算法减少22.5%,算法验证试验中机组纵向相对距离偏差控制在0.325 m内,田间实地作业平均协同偏差为0.585 m,满足作业要求。本文研究提供了一套适用于杂交水稻制种授粉的非规则路径协同控制算法,该算法具有精准的路径跟踪性能和机组协同性能,为杂交水稻制种的现代化和自动化作业奠定了基础。

Abstract:

Pollination is a crucial link in hybrid rice seed production. Aiming at the problem that precise position-velocity collaborative control is difficult to achieve in irregular fields for mechanical rope-impact pollination in hybrid rice seed production, a dual-loop reinforcement learning-based collaborative control algorithm for pollination units was proposed, which integrated proximal policy optimization (PPO) and soft actor-critic (SAC). The motion control of pollination vehicles was decoupled into subsystems with relatively independent objectives: in the lateral direction, path curvature and its rate of change were integrated into the PPO algorithm framework to achieve accurate tracking of curved field paths; in the longitudinal direction, the relative longitudinal positions of unit members and output steering angles were fused into the SAC algorithm framework to realize position-velocity control of master-slave vehicles. A dual-loop PPO – SAC reinforcement-learning algorithm was constructed to realize precise coordination of pollination vehicles. The effectiveness of the algorithm was verified through simulation tests and field experiments. The results demonstrated that the dual-loop PPO – SAC algorithm outperformed single-framework reinforcement learning algorithms. Its average path tracking error reached 0.161 m, which was 29.9% lower than that of the single PPO algorithm; the longitudinal relative distance error was reduced by 22.5% compared with that of the single SAC algorithm. In the simulated field operation test, the longitudinal relative distance error was controlled within 0.325 m, and the average coordination error in actual field operation was 0.585 m, which met the operational requirements. The research result can provide a set of irregular path collaborative control algorithms suitable for hybrid rice seed production pollination, featuring precise path tracking performance and collaborative operation capabilities, which laied a foundation for the modernization and automation of hybrid rice seed production.