基于因果推断的生菜缺水等级多模态信息融合检测算法

Multi-modal Information Fusion Detection Algorithm for Lettuce Water Deficit Levels Based on Causal Inference

摘要:

针对温室土培生菜智能灌溉中面临的缺水状况识别问题，本文提出了一种基于Transformer自注意力机制与因果推断的多模态信息融合算法，构建了基于图像和时序特征多模态融合的模型框架。通过图像处理与时间序列数据特征提取技术，获取多源异构分类特征，随后引入多模态注意力融合模块对图像及时序特征进行深度融合，在此基础上，基于因果推断的模型蒸馏技术减少图像特征中的噪声，提高神经网络对生菜缺水状态的检测性能。为验证模型性能，构建了包含不同缺水状态的温室生菜多模态图像与环境数据集，试验结果表明，本文模型相较ResNet、DenseNet、EfficientNet、ViT、Swin、ConvNeXt和MambaVision等主流模型及其变体模型在检测性能、迁移学习能力和泛化性上具有明显的优势，在测试集上识别准确率为94.08%，精确率为94.81%，召回率为94.52%。研究结果可为温室水资源优化管理和智能灌溉决策提供参考，具有重要的工程价值。

Abstract:

Aiming to address the challenge of identifying water-deficit conditions in intelligent irrigation for greenhouse-grown lettuce using soil-based cultivation, a multi-modal information fusion algorithm was proposed based on the Transformer self-attention mechanism and causal inference. A model framework was developed to fuse image and time-series features across modalities. Firstly, heterogeneous classification features were extracted from processed images and time-series environmental data. Then, a multi-modal attention fusion module was introduced to achieve deep integration of the two modalities. On this basis, a causal inference-based model distillation technique was employed to reduce noise in the image features, thereby enhancing the neural network's ability to detect water-deficit conditions in lettuce. To evaluate the model's performance, a multi-modal dataset comprising greenhouse lettuce images and environmental data under various water-deficit conditions was constructed. Experimental results demonstrated that the proposed model significantly outperformed mainstream architectures such as ResNet, DenseNet, EfficientNet, ViT, Swin, ConvNeXt, MambaVision, and their respective variants in terms of classification accuracy, transfer learning capability, and generalization. The model achieved an identification accuracy of 94.08%, a precision of 94.81%, and a recall of 94.52% on the test set. The research result can provide practical insights into efficient water management and intelligent irrigation strategies in greenhouse settings, offering notable application value.