基于自注意力机制数据扩充的土壤有机质含量近红外光谱预测方法

Prediction of Soil Organic Matter Content Using Data Augmentation Based on Self-attention Mechanisms

摘要:

作为反映农田土壤肥力状况的关键要素,土壤有机质含量在农业生产中起着举足轻重的作用。 将近红外光谱技术和深度学习相结合,为土壤有机质含量的快速且精准估测提供了全新的有效途径。 然而,由于深度学习以大数据为基础,获得大量土壤样本数据,就需对大量土样进行理化分析,这通常既耗时又耗力。 借助生成对抗网络(GAN)对样本实施扩充操作,是一种被广泛认可的解决策略。 考虑到土壤有机质近红外光谱特征波段存在特征差异,构建了一种经自注意力机制优化的生成对抗网络(SA-GAN)用于扩充土壤有机质数据,使基于近红外光谱技术预测有机质含量的准确性得到进一步提升。 采集了 120 份中国华北平原土壤样本及其对应的近红外光谱数据和有机质含量真值,分别使用 SA-GAN 和传统深度卷积生成对抗网络(DCGAN)对采集的数据进行扩充。 在数据扩充的基础上,设计了 CNN 网络,并分别对基于支持向量机(SVM)、偏最小二乘回归(PLSR)和 CNN 网络的土壤有机质含量估测模型性能进行了分析评估。 实验结果表明,SA-GAN 生成样本数据的光谱曲线趋势和有机质含量统计分布均优于 DCGAN 生成的样本数据,生成数据更接近真实数据。 将 SA-GAN 生成的模拟数据添加到 3 个预测模型的训练集中,可以比 DCGAN 更显著地提高模型预测精度。 最优情况下,将 SA-GAN 生成的仿真数据添加到模型训练数据集中可以将 SVM、PLSR、CNN 预测模型的 R 分别从 0. 51、0. 49、0. 68 提高到 0. 76、0. 73、0. 88。 结果表明,利用基于自注意力生成对抗网络来扩充土壤有机质光谱数据,并以此开展模型训练,进而提升模型的预测精度是切实可行的。

Abstract:

Soil organic matter (SOM)content is a critical indicator of soil fertility, and its accurate estimation is essential for guiding agricultural production and ensuring global food security. Traditional methods for measuring SOM content are labor-intensive and time-consuming, making rapid and non- destructive techniques highly desirable. Aiming to develop an improved method for predicting SOM content using near-infrared spectroscopy (NIR)combined with deep learning, specifically by expanding limited soil sample data using a self-attention mechanism-based generative adversarial network (SA-GAN), 120 soil samples were collected from the North China Plain, along with their corresponding NIR spectral data and true SOM content. Two data expansion methods were employed to address the challenge of small sample sizes: the proposed SA GAN and the conventional deep convolutional generative adversarial network (DCGAN). The expanded datasets were then used to train a convolutional neural network (CNN)for SOM content prediction. Additionally, the performance of three predictive models, support vector machine (SVM), partial least squares regression (PLSR), and CNN was evaluated by comparing their prediction accuracies with and without the inclusion of expanded data. The results showed that SA GAN-generated spectral curves exhibited trends closer to real data with significantly less noise than DCGAN. The statistical distribution of SOM content generated by SA GAN also more closely resembled that of real samples. When the expanded data were added to the training sets of the three prediction models, SA-GAN-generated data consistently improved model performance more effectively than DCGAN-generated data. Specifically, the R2 values of SVM, PLSR, and CNN models were increased from 0. 51, 0. 49, and 0. 68 to 0. 76, 0. 73, and 0. 88, respectively, while root mean square error (RMSE)values were decreased significantly. These findings demonstrated that SA-GAN could effectively enhance the accuracy of SOM prediction models by generating high-quality synthetic data. In conclusion, it successfully validated the feasibility of using SA-GAN to expand SOM spectral data for improving prediction accuracy. This approach not only addressed the limitations of small sample sizes but also provided a valuable tool for guiding smart agricultural practices. Future research would focus on optimizing the network structure and expanding the diversity of soil samples to further enhance model generalizability and accuracy.