Tomato Maturity Detection Method Based on Visual and Haptic Double Transfer Learning

Abstract:

Aiming at the problem that tomato ripeness cannot be accurately recognized by relying only on visual technology in the current automated picking process, a tomato ripeness detection method based on visual-haptic dual migration learning was proposed. The method firstly adopted the visual and haptic double transfer learning fusion algorithm as the feature extraction fusion module to solve the problem of not effectively extracting tomato feature information. Secondly, the soft parameter sharing-multilabel classification method was used as the classification module to avoid overfitting by increasing the correlation between different classification tasks. Focusing on tomato varieties that ripened to a single color, such as red and yellow fruits, experimental studies on a newly developed visual and tactile dataset were conducted. The experiments showed that the parameter count of the soft parameter sharing-multilabel detection model was 1.882×107, and the ripeness AUC score reached 0.977 3. Compared with the detection models of uncertainty weighted loss, adaptive hard parameter sharing, cross-stitch network, and soft parameter sharing, the parameter counts dropped by 3.08×106, 6.16×106, 3.08×106, and 3.08×106, and the ripeness AUC scores increased by 0.017 5, 0.017 9, 0.026 7 and 0.008 9, respectively. This indicated that the method improved the detection of tomato ripeness during automated picking to a certain extent and provided an effective solution to the tomato ripeness detection problem.