基于视觉触觉双重迁移学习的番茄成熟度检测方法

Tomato Maturity Detection Method Based on Visual and Haptic Double Transfer Learning

摘要:

针对当前自动化采摘过程中仅依赖视觉技术无法准确识别番茄成熟度的问题，提出了一种基于视觉触觉双重迁移学习的番茄成熟度检测方法。该方法首先采用视觉触觉双重迁移学习融合算法作为特征提取融合模块，解决无法有效提取番茄特征信息的问题。其次，将软参数共享-多标签分类方法作为分类模块，通过增加不同分类任务之间的关联性，避免出现过拟合的现象。本文主要针对成熟后为红、黄果等单一颜色的番茄品种，并在新开发的视觉触觉数据集进行实验研究。实验表明，软参数共享-多标签检测模型参数量为1.882×107，成熟度AUC分值达到0.977 3，对比不确定性加权损失、自适应硬参数共享、十字绣网络和软参数共享等检测模型，参数量分别下降3.08×106、6.16×106、3.08×106和3.08×106，成熟度AUC分值分别提高0.017 5、0.017 9、0.026 7和0.008 9。这表明该方法在一定程度上提高了自动化采摘过程中对番茄成熟度的检测能力，为番茄成熟度检测问题提供了一种有效的解决方法。

Abstract:

Aiming at the problem that tomato ripeness cannot be accurately recognized by relying only on visual technology in the current automated picking process, a tomato ripeness detection method based on visual-haptic dual migration learning was proposed. The method firstly adopted the visual and haptic double transfer learning fusion algorithm as the feature extraction fusion module to solve the problem of not effectively extracting tomato feature information. Secondly, the soft parameter sharing-multilabel classification method was used as the classification module to avoid overfitting by increasing the correlation between different classification tasks. Focusing on tomato varieties that ripened to a single color, such as red and yellow fruits, experimental studies on a newly developed visual and tactile dataset were conducted. The experiments showed that the parameter count of the soft parameter sharing-multilabel detection model was 1.882×107, and the ripeness AUC score reached 0.977 3. Compared with the detection models of uncertainty weighted loss, adaptive hard parameter sharing, cross-stitch network, and soft parameter sharing, the parameter counts dropped by 3.08×106, 6.16×106, 3.08×106, and 3.08×106, and the ripeness AUC scores increased by 0.017 5, 0.017 9, 0.026 7 and 0.008 9, respectively. This indicated that the method improved the detection of tomato ripeness during automated picking to a certain extent and provided an effective solution to the tomato ripeness detection problem.