Abstract:Accurate estimation of the harvesting pose of tomato peduncles is critical for achieving low-damage, high-efficiency robotic harvesting. The slender structure and diverse growth orientations of tomato peduncles make single-fruit harvesting via peduncle cutting particularly challenging. A method for estimating the harvesting pose of tomato peduncles was proposed by combining instance segmentation and point cloud analysis. Firstly, a dataset for instance segmentation of tomato fruits and peduncles was constructed. The YOLO v8s-seg model, demonstrating balanced performance during evaluation, was then selected for segmenting tomato fruits and peduncles. Nextly, the midpoints of the skeleton lines in the predicted peduncle regions were extracted as harvesting points. A linear fitting process was applied to the spatial point cloud corresponding to the peduncle skeleton lines to determine the peduncle growth direction, thereby generating a harvesting pose that aligned the end-effector perpendicular to the peduncle. Additionally, the ripeness of the fruit was determined by calculating the percentage of red area within each fruit's segmented region, and a greedy matching method was used to pair fruits and peduncles to enable selective harvesting. A tomato harvesting experimental platform was set up to validate the harvesting pose estimation method in a greenhouse. Experimental results showed that the trained instance segmentation model achieved a precision, recall, and mAP of 85.2%, 80.6%, and 86.9% on the test set, respectively. The accuracy of the proposed ripeness recognition method reached 97.17%, and the success rate of the fruit-peduncle matching method was 92.25%. For forward-growing fruit clusters, the detection rates of peduncles and fruits were 93.63% and 96.36%, respectively. The harvesting point identification and pose estimation accuracies were 96.11% and 89.32%, respectively, with an overall success rate of reaching the harvesting point of 60.91%. The proposed method demonstrated feasibility for peduncle-cutting tomato harvesting tasks, providing a reference for autonomous operation of tomato-picking robots in greenhouse environments.
