基于多教师蒸馏的猕猴桃种植知识图谱多跳问答模型

Multi-hop Question Answering Model for Kiwifruit Planting Knowledge Graph Based on Multi-teacher Distillation

摘要:

针对知识图谱多跳问答存在推理路径偏差导致推理性能下降的问题，本文提出一种基于多教师蒸馏的猕猴桃种植知识图谱多跳问答模型。该模型由并行推理模型、混合推理模型和学生模型构成，分别用于整合双向推理路径中的潜在信息、提供完整的上下文信息以及高效继承教师模型的推理能力。学生模型基于神经状态机实现，通过置信度分布机制从多个教师模型及其中间推理过程中选择性学习信息，并引入标准化的Logits函数动态调整蒸馏温度，以平衡信息传递精度与模型学习效果。模型在自建的猕猴桃数据集KiwiKG（Kiwi kowledge graph multi-hop dataset）及PQ（Pathquestion）、PQL（Pathquestion-large）多跳数据集中开展试验，试验结果表明：该模型在KiwiKG上命中率Hit@1达到90.92%，与EmbedKGQA模型相比提高1.82个百分点；在公开多跳数据集PQ、PQL也达到了最佳效果，模型还兼顾推理性能与参数效率，验证了其在多跳问答中的有效性与泛化能力。

Abstract:

Aiming to address the issue of inference performance degradation caused by reasoning path bias in multi-hop question answering over knowledge graphs, a novel model for multi-hop question answering in the domain of kiwifruit cultivation, was proposed based on multi-teacher knowledge distillation. The proposed model consisted of a parallel reasoning module, a hybrid reasoning module, and a student model, which were designed to integrate latent information from bidirectional reasoning paths, provide comprehensive contextual information, and efficiently inherit the reasoning capabilities of the teacher models, respectively. The student model was implemented by using a neural state machine, which selectively learned from multiple teacher models and their intermediate reasoning processes through a confidence-based distribution mechanism. Additionally, a normalized Logits function was introduced to dynamically adjust the distillation temperature, thereby balancing the precision of information transfer and the effectiveness of model learning. Experiments were conducted on the self-constructed kiwifruit dataset as kiwi kowledge graph multi-hop dataset (KiwiKG), as well as on public multi-hop question answering datasets as pathquestion (PQ) and pathquestion-large (PQL). The results demonstrated that the proposed model achieved a Hit@1 of 90.92% on KiwiKG, representing a 1.82 percentage point improvement over the EmbedKGQA model. Moreover, the model achieves state-of-the-art performance on the public datasets PQ and PQL. In addition to its strong reasoning capability, the model also maintained parameter efficiency, validating its effectiveness and generalization ability in multi-hop question answering tasks.