融合解耦注意力与度量学习的文本匹配方法

A Text Matching Method Integrating Decoupled Attention and Metric Learning

  • 摘要: 针对文本匹配任务中存在语义稀疏和长度差异显著的问题,提出了一种基于SBERT模型的架构——DeRankBERT,通过3种技术改进SBERT模型. 首先,利用基于三路径编码的解耦注意力机制,独立建模词的内容–内容、内容–位置和位置–内容关系,增强对关键语义信息的理解能力. 其次,设计基于欧式距离的排序损失函数,利用样本对的排序相似性标签优化SBERT网络,通过度量学习提高语义空间的一致性,缓解语义稀疏性. 最后,使用平均池化机制代替CLS池化,通过对词向量取平均生成句子的全局表示,有效平滑噪声并归一化文本长度的影响. 在多个公开数据集上进行实验,结果表明,DeRankBERT在文本匹配任务中表现优异,模型预测值的排名顺序与真实值的排名顺序具有高度的一致性,尤其在处理语义稀疏和长度差异显著的文本时,明显优于目前主流的方法,具有较强的鲁棒性.

     

    Abstract: Aiming at the problems of semantic sparseness and significant length difference in text matching task, a framework based on SBERT model, DeRankBERT, is proposed, which improves the SBERT model through three technologies.Firstly, the decoupling attention mechanism based on three path coding is used to independently model the content content, content position and position content relationships of words, so as to enhance the understanding ability of key semantic information.Secondly, a ranking loss function based on Euclidean distance is designed to optimize the SBERT network by using the ranking similarity tags of sample pairs, which can improve the consistency of semantic space and alleviate semantic sparsity through metric learning.Finally, the average pooling mechanism is used to replace CLS pooling.By averaging the word vectors to generate the global representation of sentences, the noise is effectively smoothed and the influence of text length is normalized. Experiments on several public datasets show that DeRankBERT performs well in text matching tasks.The ranking order of the predicted values of the model is highly consistent with that of the real values.Especially when dealing with texts with sparse semantics and significant length differences, DeRankBERT is significantly superior to the current mainstream methods and has strong robustness.

     

/

返回文章
返回