基于轻量化多模态Blend-CNN模型的小麦病虫害识别方法

Wheat Disease and Pest Recognition Method Based on Lightweight Multimodal Blend-CNN Model

摘要:

小麦作为全球重要粮食作物之一，对其病虫害进行及时诊断与防治可有效减少粮食损失。然而，深度学习方法通常依赖于大量的训练数据和高性能计算资源，在小样本学习数据和资源有限情况下存在限制。为了解决上述问题，提出一种基于知识增强的轻量化多模态小麦病虫害识别模型Blend-CNN（Multi-modal blend convolutional neural network）。该模型以双分支卷积神经网络为主体框架，在采用EfficientNet骨干网络提取病虫害图像特征基础上，引入多路TextCNN骨干网络提取病虫害文本描述特征，以得到更多的病虫害特征信息，从而提高模型识别准确率；提出基于卷积网络的多模态学习方法，使模型能够从全局上更优地融合2种模态信息。为了减少传统多模态方法的准确率损失，构建了梯度拼配损失函数；为了验证所提模型的有效性，在包含880个样本的自建数据集上进行了对比实验。结果表明，所提模型不仅在该数据集上取得最高识别准确率（96.95%），并且与其他对比模型相比，参数量更少、复杂度更低，更加轻量化，可以用于边缘设备，可为小样本和资源有限场景下的小麦病虫害识别提供一定理论支撑。

Abstract:

Wheat, as one of the world’s essential food crops, requires timely diagnosis and control of diseases and pests to significantly reduce yield losses, which is crucial for global food security. However, deep learning methods typically rely on large amounts of training data and high-performance computing resources, which pose limitations in scenarios with small sample sizes and limited resources. To address these issues, a knowledge-enhanced lightweight multi-modal wheat diseases and pests identification model named multi-modal blend convolutional neural network (Blend-CNN) was proposed. The model was structured around a dual-branch convolutional neural network framework. It utilized an EfficientNet backbone network to extract image features of diseases and pests and incorporated a multi-branch TextCNN backbone network to extract textual features of diseases and pests descriptions, thereby obtaining more feature information and improving identification accuracy. Additionally, an innovative convolutional network-based multi-modal fusion method was introduced, allowing the model to optimally integrate information from both modalities globally. Furthermore, to mitigate the accuracy loss common in traditional multi-modal methods, the gradient blend loss function was developed. Finally, to verify the model’s effectiveness, comparative experiments were conducted on a constructed dataset containing 880 samples. The results demonstrated that the proposed model achieved the highest identification accuracy of 96.95% on this dataset, compared with other models, it had fewer parameters, lower complexity, and it was more lightweight which was applicable for edge devices, offering theoretical support for wheat diseases and pests identification in scenarios with limited sample sizes and resources.