Publications
Please see my google scholar for full paper list.
2024
- arXivAlignXIE: Improving Multilingual Information Extraction by Cross-Lingual AlignmentYuxin Zuo, Wenxuan Jiang, Wenxuan Liu, Zixuan Li, Long Bai, Hanbin Wang, Yutao Zeng, Xiaolong Jin, Jiafeng Guo, Xueqi ChengarXiv preprint
Empirical evidence suggests that LLMs exhibit spontaneous cross-lingual alignment. Our findings suggest that although LLMs also demonstrate promising cross-lingual alignment in Information Extraction, there remains significant imbalance across languages, revealing an underlying deficiency in the IE alignment. To address this issue, we propose AlignXIE, a powerful code-based LLM that significantly enhances cross-lingual IE alignment through two strategies. Firstly, AlignXIE formulates IE across different languages, especially non-English ones, as code generation tasks, standardizing the representation of various schemas using Python classes to ensure consistency of the same ontology in different languages and align the schema. Secondly, it incorporates an IE cross-lingual alignment phase through a translated instance prediction task proposed in this paper to align the extraction process, utilizing ParallelNER, an IE bilingual parallel dataset with 257,190 samples, generated by our proposed LLM-based automatic pipeline for IE parallel data construction, with manual annotation to ensure quality. Ultimately, we obtain AlignXIE through multilingual IE instruction tuning. Although without training in 9 unseen languages, AlignXIE surpasses ChatGPT by 30.17% and SoTA by 20.03%, thereby demonstrating superior cross-lingual IE capabilities. Comprehensive evaluations on 63 IE benchmarks in Chinese and English under various settings, demonstrate that AlignXIE significantly enhances cross-lingual and multilingual IE through boosting the IE alignment.
- A New Pipeline for Knowledge Graph Reasoning Enhanced by Large Language Models Without Fine-TuningIn Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
Conventional Knowledge Graph Reasoning (KGR) models learn the embeddings of KG components over the structure of KGs, but their performances are limited when the KGs are severely incomplete. Recent LLM-enhanced KGR models input KG structural information into LLMs. However, they require fine-tuning on open-source LLMs and are not applicable to closed-source LLMs. Therefore, in this paper, to leverage the knowledge in LLMs without fine-tuning to assist and enhance conventional KGR models, we propose a new three-stage pipeline, including knowledge alignment, KG reasoning and entity reranking. Specifically, in the alignment stage, we propose three strategies to align the knowledge in LLMs to the KG schema by explicitly associating unconnected nodes with semantic relations. Based on the enriched KGs, we train structure-aware KGR models to integrate aligned knowledge to original knowledge existing in KGs. In the reranking stage, after obtaining the results of KGR models, we rerank the top-scored entities with LLMs to recall correct answers further. Experiments show our pipeline can enhance the KGR performance in both incomplete and general situations. Code and datasets are available.
- NLPCC-2024Retrieval-Augmented Code Generation for Universal Information ExtractionYucan Guo, Zixuan Li, Xiaolong Jin, Yantao Liu, Yutao Zeng, Wenxuan Liu , Xiang Li, Pan Yang, Long Bai, Jiafeng Guo, Xueqi ChengIn Proceedings of the 13th CCF International Conference on Natural Language Processing and Chinese Computing
Information Extraction (IE) aims to extract structural knowledge (e.g., entities, relations, events) from natural language texts, which brings challenges to existing methods due to task-specific schemas and complex text expressions. Code, as a typical kind of formalized language, is capable of describing structural knowledge under various schemas in a universal way. On the other hand, Large Language Models (LLMs) trained on both codes and texts have demonstrated powerful capabilities of transforming texts into codes, which provides a feasible solution to IE tasks. Therefore, in this paper, we propose a universal retrieval-augmented code generation framework based on LLMs, called Code4UIE, for IE tasks. Specifically, Code4UIE adopts Python classes to define task-specific schemas of various structural knowledge in a universal way. By so doing, extracting knowledge under these schemas can be transformed into generating codes that instantiate the predefined Python classes with the information in texts. To generate these codes more precisely, Code4UIE adopts the in-context learning mechanism to instruct LLMs with examples. In order to obtain appropriate examples for different tasks, Code4UIE explores several example retrieval strategies, which can retrieve examples semantically similar to the given texts. Extensive experiments on five representative IE tasks across nine datasets demonstrate the effectiveness of the Code4UIE framework.
- KnowCoder: Coding Structured Knowledge into LLMs for Universal Information ExtractionZixuan Li, Yutao Zeng, Yuxin Zuo, Weicheng Ren, Wenxuan Liu, Miao Su , Yucan Guo, Yantao Liu , Xiang Li, Zhilei Hu, Long Bai , Wei Li, Yidan Liu, Pan Yang, Xiaolong Jin, Jiafeng Guo, Xueqi ChengIn Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics
In this paper, we propose KnowCoder, a Large Language Model (LLM) to conduct Universal Information Extraction (UIE) via code generation. KnowCoder aims to develop a kind of unified schema representation that LLMs can easily understand and an effective learning framework that encourages LLMs to follow schemas and extract structured knowledge accurately. To achieve these, KnowCoder introduces a code-style schema representation method to uniformly transform different schemas into Python classes, with which complex schema information, such as constraints among tasks in UIE, can be captured in an LLM-friendly manner. We further construct a code-style schema library covering over 30,000 types of knowledge, which is the largest one for UIE, to the best of our knowledge. To ease the learning process of LLMs, KnowCoder contains a two-phase learning framework that enhances its schema understanding ability via code pretraining and its schema following ability via instruction tuning. After code pretraining on around 1.5B automatically constructed data, KnowCoder already attains remarkable generalization ability and achieves relative improvements by 49.8% F1, compared to LLaMA2, under the few-shot setting. After instruction tuning, KnowCoder further exhibits strong generalization ability on unseen schemas and achieves up to 12.5% and 21.9%, compared to sota baselines, under the zero-shot setting and the low resource setting, respectively. Additionally, based on our unified schema representations, various human-annotated datasets can simultaneously be utilized to refine KnowCoder, which achieves significant improvements up to 7.5% under the supervised setting.
- arXivTemporal Knowledge Graph Question Answering: A SurveyarXiv preprint
Knowledge Base Question Answering (KBQA) has been a long-standing field to answer questions based on knowledge bases. Recently, the evolving dynamics of knowledge have attracted a growing interest in Temporal Knowledge Graph Question Answering (TKGQA), an emerging task to answer temporal questions. However, this field grapples with ambiguities in defining temporal questions and lacks a systematic categorization of existing methods for TKGQA. In response, this paper provides a thorough survey from two perspectives: the taxonomy of temporal questions and the methodological categorization for TKGQA. Specifically, we first establish a detailed taxonomy of temporal questions engaged in prior studies. Subsequently, we provide a comprehensive review of TKGQA techniques of two categories: semantic parsing-based and TKG embedding-based. Building on this review, the paper outlines potential research directions aimed at advancing the field of TKGQA. This work aims to serve as a comprehensive reference for TKGQA and to stimulate further research.
- An In-Context Schema Understanding Method for Knowledge Base Question AnsweringIn Proceedings of the 17th International Conference on Knowledge Science, Engineering and Management
The Knowledge Base Question Answering (KBQA) task aims to answer natural language questions based on a given knowledge base. Recently, Large Language Models (LLMs) have shown strong capabilities in language understanding and can be used to solve this task. In doing so, a major challenge for LLMs is to overcome the immensity and heterogeneity of knowledge base schemas.Existing methods bypass this challenge by initially employing LLMs to generate drafts of logic forms without schema-specific details. Then, an extra module is used to inject schema information to these drafts. In contrast, in this paper, we propose a simple In-Context Schema Understanding (ICSU) method that enables LLMs to directly understand schemas by leveraging in-context learning. Specifically, ICSU provides schema information to LLMs using schema-related annotated examples. We investigate three example retrieval strategies based on raw questions, anonymized questions, and generated SPARQL queries. Experimental results show that ICSU demonstrates competitive performance compared to baseline methods on both the KQA Pro and WebQSP datasets.
- Class-Incremental Few-Shot Event DetectionIn Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation
Event detection is one of the fundamental tasks in information extraction and knowledge graph. However, a realistic event detection system often needs to deal with new event classes constantly. These new classes usually have only a few labeled instances as it is time-consuming and labor-intensive to annotate a large number of unlabeled instances. Therefore, this paper proposes a new task, called class-incremental few-shot event detection. Nevertheless, this task faces two problems, i.e., old knowledge forgetting and new class overfitting. To solve these problems, this paper further presents a novel knowledge distillation and prompt learning based method, called Prompt-KD. Specifically, to handle the forgetting problem about old knowledge, Prompt-KD develops an attention based multi-teacher knowledge distillation framework, where the ancestor teacher model pre-trained on base classes is reused in all learning sessions, and the father teacher model derives the current student model via adaptation. On the other hand, in order to cope with the few-shot learning scenario and alleviate the corresponding new class overfitting problem, Prompt-KD is also equipped with a prompt learning mechanism. Extensive experiments on two benchmark datasets, i.e., FewEvent and MAVEN, demonstrate the superior performance of Prompt-KD.
- Nested Event Extraction upon Pivot Element RecognitonWeicheng Ren, Zixuan Li, Xiaolong Jin, Long Bai, Miao Su, Yantao Liu, Saiping Guan, Jiafeng Guo, Xueqi ChengIn Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation
Nested Event Extraction (NEE) aims to extract complex event structures where an event contains other events as its arguments recursively. Nested events involve a kind of Pivot Elements (PEs) that simultaneously act as arguments of outer-nest events and as triggers of inner-nest events, and thus connect them into nested structures. This special characteristic of PEs brings challenges to existing NEE methods, as they cannot well cope with the dual identities of PEs. Therefore, this paper proposes a new model, called PerNee, which extracts nested events mainly based on recognizing PEs. Specifically, PerNee first recognizes the triggers of both inner-nest and outer-nest events and further recognizes the PEs via classifying the relation type between trigger pairs. The model uses prompt learning to incorporate information from both event types and argument roles for better trigger and argument representations to improve NEE performance. Since existing NEE datasets (e.g., Genia11) are limited to specific domains and contain a narrow range of event types with nested structures, we systematically categorize nested events in the generic domain and construct a new NEE dataset, called ACE2005-Nest. Experimental results demonstrate that PerNee consistently achieves state-of-the-art performance on ACE2005-Nest, Genia11, and Genia13. The ACE2005-Nest dataset and the code of the PerNee model are available at https://github.com/waysonren/PerNee .
- Selective Temporal Knowledge Graph ReasoningIn Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation
Temporal Knowledge Graph (TKG), which characterizes temporally evolving facts in the form of (subject, relation, object, timestamp), has attracted much attention recently. TKG reasoning aims to predict future facts based on given historical ones. However, existing TKG reasoning models are unable to abstain from predictions they are uncertain, which will inevitably bring risks in real-world applications. Thus, in this paper, we propose an abstention mechanism for TKG reasoning, which helps the existing models make selective, instead of indiscriminate, predictions. Specifically, we develop a confidence estimator, called Confidence Estimator with History (CEHis), to enable the existing TKG reasoning models to first estimate their confidence in making predictions, and then abstain from those with low confidence. To do so, CEHis takes two kinds of information into consideration, namely, the certainty of the current prediction and the accuracy of historical predictions. Experiments with representative TKG reasoning models on two benchmark datasets demonstrate the effectiveness of the proposed CEHis.
- JCIP-2024基于多历史序列联合演化建模的两阶段时序知识图谱推理中文信息学报
近年来，随着互联网技术和应用模式的迅猛发展，互联网数据规模爆炸式增长，其中包含大量带有时序信息的动态事件知识。为了建模这类动态事件知识，时序知识图谱在传统知识图谱的基础上引入时间信息，以带时间戳的知识图谱序列刻画这类知识。时序知识图谱推理任务旨在根据过去发生的事件四元组（主语实体，关系（事件类型），宾语实体，时间戳）预测未来发生的事件。为此，模型需要充分建模实体的历史演化过程。然而，巨大的实体数目以及它们对应的大量历史事件给时序知识图谱推理任务带来了巨大挑战。为了降低待建模历史的规模，已有方法选择建模查询实体的长程历史或者全部实体的短程历史，都丢失了一部分历史信息。实际上，由于不同实体对于一个查询的相关程度不同，模型需要更充分地建模相关实体的历史信息。基于此，该文提出了基于多历史序列联合演化建模的两阶段时序推理模型MENet（Multi-sequence Evolution Network）。具体而言，其在第一阶段采用了一种基于启发式规则的候选实体筛选策略，选择最有可能发生事件的候选实体，从而有效地降低了需要建模的实体数目；在第二阶段，其采用了一个多历史序列联合演化模型: 首先通过组合多个实体各自的长程历史信息，得到需要建模的图序列，进而通过考虑该图序列上同时刻发生事件之间的结构依赖、事件发生的时间数值信息以及不同时刻之间的时序依赖，从而更精准地建模实体演化过程。在三个标准数据集上的实验结果表明，上述模型相比于当前最先进的方法模型具有更好的推理性能。
- Tree-of-Reasoning Question Decomposition for Complex Question Answering with Large Language ModelsKun Zhang, Jiali Zeng, Fandong Meng, Yuanzhuo Wang, Shiqi Sun, Long Bai, Huawei Shen, Jie ZhouIn Proceedings of the AAAI Conference on Artificial Intelligence
Large language models (LLMs) have recently demonstrated remarkable performance across various Natual Language Processing tasks. In the field of multi-hop reasoning, the Chain-of-thought (CoT) prompt method has emerged as a paradigm, using curated stepwise reasoning demonstrations to enhance LLM’s ability to reason and produce coherent rational pathways. To ensure the accuracy, reliability, and traceability of the generated answers, many studies have incorporated information retrieval (IR) to provide LLMs with external knowledge. However, existing CoT with IR methods decomposes questions into sub-questions based on a single compositionality type, which limits their effectiveness for questions involving multiple compositionality types. Additionally, these methods suffer from inefficient retrieval, as complex questions often contain abundant information, leading to the retrieval of irrelevant information inconsistent with the query’s intent. In this work, we propose a novel question decomposition framework called TRQA for multi-hop question answering, which addresses these limitations. Our framework introduces a reasoning tree (RT) to represent the structure of complex questions. It consists of four components: the Reasoning Tree Constructor (RTC), the Question Generator (QG), the Retrieval and LLM Interaction Module (RAIL), and the Answer Aggregation Module (AAM). Specifically, the RTC predicts diverse sub-question structures to construct the reasoning tree, allowing a more comprehensive representation of complex questions. The QG generates sub-questions for leaf-node in the reasoning tree, and we explore two methods for QG: prompt-based and T5-based approaches. The IR module retrieves documents aligned with sub-questions, while the LLM formulates answers based on the retrieved information. Finally, the AAM aggregates answers along the reason tree, producing a definitive response from bottom to top.
2023
- Temporal Knowledge Graph Reasoning Based on N-tuple ModelingIn Findings of the Association for Computational Linguistics: EMNLP 2023
Reasoning over Temporal Knowledge Graphs (TKGs) that predicts temporal facts (e.g., events) in the future is crucial for many applications. The temporal facts in existing TKGs only contain their core entities (i.e., the entities playing core roles therein) and formulate them as quadruples, i.e., (subject entity, predicate, object entity, timestamp). This formulation oversimplifies temporal facts and inevitably causes information loss. Therefore, we propose to describe a temporal fact more accurately as an n-tuple, containing not only its predicate and core entities, but also its auxiliary entities, as well as the roles of all entities. By so doing, TKGs are augmented to N-tuple Temporal Knowledge Graphs (N-TKGs). To conduct reasoning over N-TKGs, we further propose N-tuple Evolutional Network (NE-Net). It recurrently learns the evolutional representations of entities and predicates in temporal facts at different timestamps in the history via modeling the relations among those entities and predicates. Based on the learned representations, reasoning tasks at future timestamps can be realized via task-specific decoders. Experiment results on two newly built datasets demonstrate the superiority of N-TKG and the effectiveness of NE-Net.
- arXivProtoEM: A Prototype-Enhanced Matching Framework for Event Relation ExtractionarXiv preprint
Event Relation Extraction (ERE) aims to extract multiple kinds of relations among events in texts. However, existing methods singly categorize event relations as different classes, which are inadequately capturing the intrinsic semantics of these relations. To comprehensively understand their intrinsic semantics, in this paper, we obtain prototype representations for each type of event relation and propose a Prototype-Enhanced Matching (ProtoEM) framework for the joint extraction of multiple kinds of event relations. Specifically, ProtoEM extracts event relations in a two-step manner, i.e., prototype representing and prototype matching. In the first step, to capture the connotations of different event relations, ProtoEM utilizes examples to represent the prototypes corresponding to these relations. Subsequently, to capture the interdependence among event relations, it constructs a dependency graph for the prototypes corresponding to these relations and utilized a Graph Neural Network (GNN)-based module for modeling. In the second step, it obtains the representations of new event pairs and calculates their similarity with those prototypes obtained in the first step to evaluate which types of event relations they belong to. Experimental results on the MAVEN-ERE dataset demonstrate that the proposed ProtoEM framework can effectively represent the prototypes of event relations and further obtain a significant improvement over baseline models.
- CFGL-LCR: A Counterfactual Graph Learning Framework for Legal Case RetrievalKun Zhang, Chong Chen, Yuanzhuo Wang, Qi Tian, Long BaiIn Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
Legal case retrieval, which aims to find relevant cases based on a short case description, serves as an important part of modern legal systems. Despite the success of existing retrieval methods based on Pretrained Language Models, there are still two issues in legal case retrieval that have not been well considered before. First, existing methods underestimate the semantics associations among legal elements, e.g., law articles and crimes, which played an essential role in legal case retrieval. These methods only adopt the pre-training language model to encode the whole legal case, instead of distinguishing different legal elements in the legal case. They randomly split a legal case into different segments, which may break the completeness of each legal element. Second, due to the difficulty in annotating the relevant labels of similar cases, legal case retrieval inevitably faces the problem of lacking training data. In this paper, we propose a counterfactual graph learning framework for legal case retrieval. Concretely, to overcome the above challenges, we transform the legal case document into a graph and model the semantics of the legal elements through a graph neural network. To alleviate the low resource and learn the causal relationship between the semantics of legal elements and relevance, a counterfactual data generator is designed to augment counterfactual data and enhance legal case representation. Extensive experiments based on two publicly available legal benchmarks demonstrate that our CFGL-LCR can significantly outperform previous state-of-the-art methods in legal case retrieval.
- Semantic Structure Enhanced Event Causality IdentificationIn Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics
Event Causality Identification (ECI) aims to identify causal relations between events in unstructured texts. This is a very challenging task, because causal relations are usually expressed by implicit associations between events. Existing methods usually capture such associations by directly modeling the texts with pre-trained language models, which underestimate two kinds of semantic structures vital to the ECI task, namely, event-centric structure and event-associated structure. The former includes important semantic elements related to the events to describe them more precisely, while the latter contains semantic paths between two events to provide possible supports for ECI. In this paper, we study the implicit associations between events by modeling the above explicit semantic structures, and propose a Semantic Structure Integration model (SemSIn).It utilizes a GNN-based event aggregator to integrate the event-centric structure information, and employs an LSTM-based path aggregator to capture the event-associated structure information between two events. Experimental results on three widely used datasets show that SemSIn achieves significant improvements over baseline methods.
- Rich Event Modeling for Script Event PredictionIn Proceedings of the AAAI Conference on Artificial Intelligence
Script is a kind of structured knowledge extracted from texts, which contains a sequence of events. Based on such knowledge, script event prediction aims to predict the subsequent event. To do so, two aspects should be considered for events, namely, event description (i.e., what the events should contain) and event encoding (i.e., how they should be encoded). Most existing methods describe an event by a verb together with a few core arguments (i.e., subject, object, and indirect object), which are not precise enough. In addition, existing event encoders are limited to a fixed number of arguments, which are not flexible enough to deal with extra information. Thus, in this paper, we propose the Rich Event Prediction (REP) framework for script event prediction. Fundamentally, it is based on the proposed rich event description, which enriches the existing ones with three kinds of important information, namely, the senses of verbs, extra semantic roles, and types of participants. REP contains an event extractor to extract such information from texts. Based on the extracted rich information, a predictor then selects the most probable subsequent event. The core component of the predictor is a transformer-based event encoder that integrates the above information flexibly. Experimental results on the widely used Gigaword Corpus show the effectiveness of the proposed framework.
2022
- Knowledge-Enhanced Self-Supervised Prototypical Network for Few-Shot Event DetectionIn Findings of the Association for Computational Linguistics: EMNLP 2022
Prototypical network based joint methods have attracted much attention in few-shot event detection, which carry out event detection in a unified sequence tagging framework. However, these methods suffer from the inaccurate prototype representation problem, due to two main reasons: the number of instances for calculating prototypes is limited; And, they do not well capture the relationships among event prototypes. To deal with this problem, we propose a Knowledge-Enhanced self-supervised Prototypical Network, called KE-PN, for few-shot event detection. KE-PN adopts hybrid rules, which can automatically align event types to an external knowledge base, i.e., FrameNet, to obtain more instances. It proposes a self-supervised learning method to filter out noisy data from enhanced instances. KE-PN is further equipped with an auxiliary event type relationship classification module, which injects the relationship information into representations of event prototypes. Extensive experiments on three benchmark datasets, i.e., FewEvent, MAVEN, and ACE2005 demonstrate the state-of-the-art performance of KE-PN.
- HiSMatch: Historical Structure Matching based Temporal Knowledge Graph ReasoningZixuan Li, Zhongni Hou, Saiping Guan, Xiaolong Jin, Weihua Peng, Long Bai, Yajuan Lyu , Wei Li, Jiafeng Guo, Xueqi ChengIn Findings of the Association for Computational Linguistics: EMNLP 2022
A Temporal Knowledge Graph (TKG) is a sequence of KGs with respective timestamps, which adopts quadruples in the form of (subject, relation, object, timestamp) to describe dynamic facts. TKG reasoning has facilitated many real-world applications via answering such queries as (query entity, query relation, ?, future timestamp) about future. This is actually a matching task between a query and candidate entities based on their historical structures, which reflect behavioral trends of the entities at different timestamps. In addition, recent KGs provide background knowledge of all the entities, which is also helpful for the matching. Thus, in this paper, we propose the Historical Structure Matching (HiSMatch) model. It applies two structure encoders to capture the semantic information contained in the historical structures of the query and candidate entities. Besides, it adopts another encoder to integrate the background knowledge into the model. TKG reasoning experiments on six benchmark datasets demonstrate the significant improvement of the proposed HiSMatch model, with up to 5.6% performance improvement in MRR, compared to the state-of-the-art baselines.
- Meta-CQG: A Meta-Learning Framework for Complex Question Generation over Knowledge BasesKun Zhang, Yunqi Qiu, Yuanzhuo Wang, Long Bai , Wei Li, Xuhui Jiang, Huawei Shen, Xueqi ChengIn Proceedings of the 29th International Conference on Computational Linguistics
Complex question generation over knowledge bases (KB) aims to generate natural language questions involving multiple KB relations or functional constraints. Existing methods train one encoder-decoder-based model to fit all questions. However, such a one-size-fits-all strategy may not perform well since complex questions exhibit an uneven distribution in many dimensions, such as question types, involved KB relations, and query structures, resulting in insufficient learning for long-tailed samples under different dimensions. To address this problem, we propose a meta-learning framework for complex question generation. The meta-trained generator can acquire universal and transferable meta-knowledge and quickly adapt to long-tailed samples through a few most related training samples. To retrieve similar samples for each input query, we design a self-supervised graph retriever to learn distributed representations for samples, and contrastive learning is leveraged to improve the learned representations. We conduct experiments on both WebQuestionsSP and ComplexWebQuestion, and results on long-tailed samples of different dimensions have been significantly improved, which demonstrates the effectiveness of the proposed framework.
- AES-2022面向知识库问答的问句语义解析研究综述仇韫琦, 王元卓, 白龙, 尹芷仪, 沈华伟, 白硕电子学报
知识库问答（Knowledge Base Question Answering，KBQA）借助知识库中精度高、关联性强的结构化知识，为给定的复杂事实型问句提供准确、简短的答案.语义解析是知识库问答的主流方法之一，该类方法在给定的问句语义表征形式下，将非结构化的问句映射为结构化的语义表征，再将其改写为知识库查询获取答案。目前，面向知识库问答的语义解析方法主要面临三个挑战：首先是如何选择合适的语义表征形式以表达问句的语义，然后是如何解析问句的复杂语义并输出相应的语义表征，最后是如何应对特定领域中数据标注成本高昂、高质量数据匮乏的问题.本文从上述挑战出发，分析了知识库问答中常用的语义表征的特点与不足，然后梳理现有方法并总结分析其如何应对问句的复杂语义，接着介绍了当前方法在标注数据匮乏的低资源场景下的尝试，最后展望并讨论了面向知识库问答的语义解析的未来发展方向。
- TKDE-2022What is Event Knowledge Graph: A SurveyIEEE Transactions on Knowledge and Data Engineering
Besides entity-centric knowledge, usually organized as Knowledge Graph (KG), events are also an essential kind of knowledge in the world, which trigger the spring up of event-centric knowledge representation form like Event KG (EKG). It plays an increasingly important role in many downstream applications, such as search, question-answering, recommendation, financial quantitative investments, and text generation. This paper provides a comprehensive survey of EKG from history, ontology, instance, and application views. Specifically, to characterize EKG thoroughly, we focus on its history, definition, schema induction, acquisition, related representative graphs/systems, and applications. The development processes and trends are studied therein. We further summarize prospective directions to facilitate future research on EKG.
- Complex Evolutional Pattern Learning for Temporal Knowledge Graph ReasoningZixuan Li, Saiping Guan, Xiaolong Jin, Weihua Peng, Yajuan Lyu, Yong Zhu, Long Bai , Wei Li, Jiafeng Guo, Xueqi ChengIn Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics
A Temporal Knowledge Graph (TKG) is a sequence of KGs corresponding to different timestamps. TKG reasoning aims to predict potential facts in the future given the historical KG sequences. One key of this task is to mine and understand evolutional patterns of facts from these sequences. The evolutional patterns are complex in two aspects, length-diversity and time-variability. Existing models for TKG reasoning focus on modeling fact sequences of a fixed length, which cannot discover complex evolutional patterns that vary in length. Furthermore, these models are all trained offline, which cannot well adapt to the changes of evolutional patterns from then on. Thus, we propose a new model, called Complex Evolutional Network (CEN), which uses a length-aware Convolutional Neural Network (CNN) to handle evolutional patterns of different lengths via an easy-to-difficult curriculum learning strategy. Besides, we propose to learn the model under the online setting so that it can adapt to the changes of evolutional patterns over time. Extensive experiments demonstrate that CEN obtains substantial performance improvement under both the traditional offline and the proposed online settings.
2021
- Integrating Deep Event-Level and Script-Level Information for Script Event PredictionIn Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
Scripts are structured sequences of events together with the participants, which are extracted from the texts. Script event prediction aims to predict the subsequent event given the historical events in the script. Two kinds of information facilitate this task, namely, the event-level information and the script-level information. At the event level, existing studies view an event as a verb with its participants, while neglecting other useful properties, such as the state of the participants. At the script level, most existing studies only consider a single event sequence corresponding to one common protagonist. In this paper, we propose a Transformer-based model, called MCPredictor, which integrates deep event-level and script-level information for script event prediction. At the event level, MCPredictor utilizes the rich information in the text to obtain more comprehensive event semantic representations. At the script-level, it considers multiple event sequences corresponding to different participants of the subsequent event. The experimental results on the widely-used New York Times corpus demonstrate the effectiveness and superiority of the proposed model.
- Rule-Aware Reinforcement Learning for Knowledge Graph ReasoningIn Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021
Multi-hop reasoning is an effective and explainable approach to predicting missing facts in Knowledge Graphs (KGs). It usually adopts the Reinforcement Learning (RL) framework and searches over the KG to find an evidential path. However, due to the large exploration space, the RL-based model struggles with the serious sparse reward problem and needs to make a lot of trials. Moreover, its exploration can be biased towards spurious paths that coincidentally lead to correct answers. To solve both problems, we propose a simple but effective RL-based method called RARL (Rule-Aware RL). It injects high quality symbolic rules into the model’s reasoning process and employs partially random beam search, which can not only increase the probability of paths getting rewards, but also alleviate the impact of spurious paths. Experimental results show that it outperforms existing multi-hop methods in terms of Hit@1 and MRR.
2020
- Bidirectional Dependency-Guided Attention for Relation ExtractionXingchen Deng, Lei Zhang, Yixing Fan, Long Bai, Jiafeng Guo, Pengfei WangIn Proceedings of the 12th Asian Conference on Machine Learning
The dependency relation between words in the sentence is critical for the relation extraction. Existing methods often utilize the dependencies accompanied with various pruning strategies, thus suffer from the loss of detailed semantic information.In order to exploit dependency structure more effectively, we propose a novel bidirectional dependency-guided attention model. The main idea is to use a top-down attention as well as a bottom-up attention to fully capture the dependencies from different granularity. Specifically, the bottom-up attention aims to model the local semantics from the subtree of each node, while the top-down attention is to model the global semantics from the ancestor nodes. Moreover, we employ a label embedding component to attend the contextual features, which are extracted by the dependency-guided attention. Overall, the proposed model is fully attention-based which make it easy for parallel computing. Experiment results on TACRED dataset and SemEval 2010 Task 8 dataset show that our model outperforms existing dependency based models as well as the powerful pretraining model. Moreover, the proposed model achieves the state-of-the-art performance on TACRED dataset.
- Hierarchical Query Graph Generation for Complex Question Answering over Knowledge GraphIn Proceedings of the 29th ACM International Conference on Information & Knowledge Management
Knowledge Graph Question Answering aims to automatically answer natural language questions via well-structured relation information between entities stored in knowledge graphs. When faced with a complex question with compositional semantics, query graph generation is a practical semantic parsing-based method. But existing works rely on heuristic rules with limited coverage, making them impractical on more complex questions. This paper proposes a Director-Actor-Critic framework to overcome these challenges. Through options over a Markov Decision Process, query graph generation is formulated as a hierarchical decision problem. The Director determines which types of triples the query graph needs, the Actor generates corresponding triples by choosing nodes and edges, and the Critic calculates the semantic similarity between the generated triples and the given questions. Moreover, to train from weak supervision, we base the framework on hierarchical Reinforcement Learning with intrinsic motivation. To accelerate the training process, we pre-train the Critic with high-reward trajectories generated by hand-crafted rules, and leverage curriculum learning to gradually increase the complexity of questions during query graph generation. Extensive experiments conducted over widely-used benchmark datasets demonstrate the effectiveness of the proposed framework.
- Entity Type Enhanced Neural Model for Distantly Supervised Relation Extraction (Student Abstract)In Proceedings of the AAAI Conference on Artificial Intelligence
Distantly Supervised Relation Extraction (DSRE) has been widely studied, since it can automatically extract relations from very large corpora. However, existing DSRE methods only use little semantic information about entities, such as the information of entity type. Thus, in this paper, we propose a method for integrating entity type information into a neural network based DSRE model. It also adopts two attention mechanisms, namely, sentence attention and type attention. The former selects the representative sentences for a sentence bag, while the latter selects appropriate type information for entities. Experimental comparison with existing methods on a benchmark dataset demonstrates its merits.
2019
- JCIP-2019基于深度学习的关系抽取研究综述中文信息学报
关系抽取（RE）是为了抽取文本中包含的关系，是信息抽取（IE）的重要组成部分。近年来，研究人员利用深度学习技术在该领域开展了深入研究。由于神经网络类型丰富，基于深度学习的关系抽取方法也更加多样。该文从关系抽取的基本概念出发，对关系抽取方法依据不同的视角进行了类别划分。随后，介绍了基于深度学习的关系抽取方法常用的数据集，并总结出基于深度学习的关系抽取框架。在此框架下，对关系抽取方法在面向深度学习的输入数据预处理、面向深度学习的神经网络模型设计等方面的具体工作进行了分析与评述，最后对未来的研究方向进行了探讨和展望。