FREDA: Few-Shot Relation Extraction Based on Data Augmentation

Abstract

The primary task of few-shot relation extraction is to quickly learn the features of relation classes from a few labelled instances and predict the semantic relations between entity pairs in new instances. Most existing few-shot relation extraction methods do not fully utilize the relation information features in sentences, resulting in difficulties in improving the performance of relation classification. Some researchers have attempted to incorporate external information, but the results have been unsatisfactory when applied to different domains. In this paper, we propose a method that utilizes triple information for data augmentation, which can alleviate the issue of insufficient instances and possesses strong domain adaptation capabilities. Firstly, we extract relation and entity pairs from the instances in the support set, forming relation triple information. Next, the sentence information and relation triple information are encoded using the same sentence encoder. Then, we construct an interactive attention module to enable the query set instances to interact separately with the support set instances and relation triple instances. The module pays greater attention to highly interactive parts between instances and assigns them higher weights. Finally, we merge the interacted support set representation and relation triple representation. To our knowledge, we are the first to propose a method that utilizes triple information for data augmentation in relation extraction. In our experiments on the standard datasets FewRel1.0 and FewRel2.0 (domain adaptation), we observed substantial improvements without including external information.

1. Introduction

Relation extraction (RE) is one of the fundamental tasks in building knowledge graphs (KGs) in natural language processing [1]. Its primary purpose is to identify the types of relations between pairs of entities in a sentence. Since Zheng et al. [2] use CNN-based models for RE tasks and propose methods to capture relevant tokens and sentence-level features automatically, neural-network-based RE methods have replaced mainly traditional algorithms and have been widely used in tasks, such as information retrieval [3], dialogue generation [4], and reading comprehension [5].

Most researchers use supervised learning methods for RE [6,7], training models based on large amounts of labelled data. However, labelling the dataset is time-consuming and laborious in real life. To address this problem, some researchers have proposed the distantly supervised relation extraction (DSRE) methods. Mintz et al. [8] constructed an algorithm that does not rely on labelled data, reducing the dependence on the dataset but not focusing on the information in the bag; Pathore et al. [9] exchanged information about the sentences within the bag to make more use of the data in the available bag. The distant supervision method uses a known knowledge base aligned with a large text corpus to generate labelled datasets, which does not require a large amount of manually labelled data but suffers from a relatively severe noise interference problem, leading to the limited performance of RE. Therefore, the researchers propose the few-shot relation extraction (FSRE) method to solve the data scarcity problem [10].

Using a small number of samples in training models has been a research point that attracts more attention [11,12]. In the field of computer vision, the use of few-shot learning (FSL) [13,14] has shown promising results. Inspired by this, various FSL-based methods have been introduced in the task of FSRE. Prototype networks are one of the most popular algorithms. Snell et al. [15] introduce prototype networks into FSRE based on a meta-learning framework [16], and the accuracy rate is effectively improved.

The FSRE task initially uses only sentence information for model training. Gao et al. [17] and Ye et al. [18] demonstrated the usefulness by comparing them with global-level models and did not focus on information across relations in the dataset; Wang et al. [19] alleviate the relation obfuscation problem using two specific mechanisms. All [17,18,19] try to fully use the valuable information in the dataset to achieve a more informative prototype network. Recent FSRE research efforts introduce external information such as relation information (description of relation information) [20,21,22] and entity concepts [23], thus effectively improving the performance of FSRE tasks. However, two problems remain: (1) The current work is overly dependent on external information and does not pay sufficient attention to the information already available in the dataset [24]. (2) The approach in using prototype networks tends to construct prototypes by averaging the instances of each class, ignoring the interactive information between the support set and the query set instances [25].

We propose a data enhancement approach in this paper to address the above issues. On the one hand, by introducing the relation triple information, data augmentation can be applied to the information of the instances in the support set; on the other hand, by incorporating an interactive attention module, the interaction information between the instances of the support set and the query set, as well as the instances of the relation triple and the query set, can be enhanced. As shown in Figure 1, the relation triple information contains two parts, the relation’s name and the entity pairs, which we can obtain from the support set instances. Our method is as follows. First, we pass the relation triples and sentence information through the same encoder, mapping them to the same semantic space. Next, by utilizing the Euclidean distance algorithm, the interactive attention module assigns higher coefficient values to the parts where the instances in the support set and the instances in the query set, as well as the instances in the relation triple and the instances in the query set, interact intensively with each other. Finally, we directly add the support set and relation triple representations after passing through the interactive attention module.

The difference between our work and other existing results is that our approach only uses information already in the instances and does not incorporate external information. Moreover, our ideas are more suitable for application to FSRE for two reasons: (1) When training models, common approaches to data enhancement are the use of unlabeled datasets [26] and the fuller use of labelled datasets [27]. Our approach is to make full use of the dataset and enhance the accuracy of various types of prototypes by adding the relation triple, which improves the results effectively and has a better domain adaptation capability. (2) To improve performance on the test set, we incorporate the interactive attention module during model training. This module calculates the similarity between the query set instances, support set instances, and relation triple instances. We prioritize the highly interactive parts between instances to utilize the inherent information of the dataset fully.

We conducted experiments on FewRel1.0 [28] and FewRel2.0 (domain adaptation) [29] in four few-shot settings, and both of our methods yielded better boosts, which illustrates the effectiveness of our approach. Our contribution is as follows:

(1)

For the first time, we invoke relation triple information to enhance and correlate the support set’s relation and entity pair features through a data augmentation approach.

(2)

We address the issue of limited resources by incorporating an interactive attention module, which focuses on highly interactive portions among instances, thereby improving resource constraints.

(3)

Our model show improvements in both FewRel1.0 and FewRel2.0 (domain adaptation) datasets, with particularly significant results in FewRel2.0 (achieving a maximum improvement of 9.79 points). This demonstrates the strong domain adaptation capability of our model.

2. Related Work

In this section, we will introduce and explain RE and FSRE.

2.1. Relation Extraction

RE is one of the fundamental tasks of information extraction, and its purpose is to provide fundamental support for the construction of structured knowledge by identifying semantic relations between known entity pairs in an instance. Traditional RE has mainly used kernel-based approaches [30] and feature-based approaches [31] and has achieved good results. However, traditional relation extraction relies excessively on manual annotation and the ability to understand context. With the introduction of deep learning in natural language processing, several neural-network-based models were introduced for RE, such as convolutional neural networks (CNNs) [2], long short-term memory (LSTM) [32], and transformer [33], which continue to improve their effectiveness in supervised RE. The large amount of data that drives the model makes its generalization ability overly dependent on high-quality datasets. Therefore, using supervised learning requires labelling a large amount of high-quality data, and labelling a batch of RE datasets is time-consuming and laborious. To address the difficulty of labelling data, Mintz et al. [8] introduce the idea of distant supervision to RE, which typically automatically generates labelled data by using an existing knowledge base aligned to the text. Riedel et al. [34] refer to Mintz et al.’s idea as the “distant supervision hypothesis” and propose the at-least-one hypothesis to introduce multiple instances into distant supervision of data but ignore the possibility of multiple relations per entity pair. Zeng et al. [35] propose an algorithm combining PCNN with multiple instances to classify and extract the generated data, but this approach still tends to introduce excessive noise problems. The main task of the existing DSRE is to denoise and improve the correctness of the labelled dataset as much as possible [36]. However, this method still cannot solve the long-tail problem that exists in the dataset. In order to improve the performance of the RE model more effectively, the FSRE method is proposed to solve the long-tail problem and the data scarcity problem.

2.2. Few-Shot Relation Extraction

FSRE aims to train the model using limited instances as much as possible through reasonable algorithms and technical means and to accurately identify the semantic relations of entity pairs in unlabeled text. By utilizing FSL techniques, FSRE tasks have addressed the issues of limited dataset instances and long-tail problems. This approach avoids heavy reliance on a large amount of annotated data, significantly reducing the cost of building RE models. FSRE has gained considerable attention and research focus in recent years due to its strong feasibility and applicability in real-life scenarios. The focus of the task is introducing the concept of FSL [37]. It is the computer field that first uses the method of FSL, which already has many excellent algorithms for image segmentation. Zhang et al. [38] came up with the idea of representing images as distributions by learning parameterized Gaussian noise regularization; Finn et al. [39] propose a model-independent meta-learning algorithm. In the FSRE field, metric-based and optimization-based algorithms are the most common.

The metric-based approach achieves semantic relation classification of entity pairs based on comparing distances after encoding embeddings and similarity measures [40]. The parameter-based optimization approaches treat the relations between entities as individual tasks and train a multitask classifier to handle all relations in a multitask manner [41]. The current parameter-based optimization approach is inferior to the metric-based approach, and most scholars have focused their research on the metric-based approach. Koch et al. [42] propose a Siamese network based on a metric learning method, which calculates the data distance after inputting two samples into two identical networks, and follows the distance to determine whether the two instances are of the same class. Gao et al. [26] propose a neural snowball approach based on the Siamese network to classify specific new relations from an open-domain perspective using many unlabeled texts. However, because Siamese network use a comparison-by-comparison approach, they are less efficient in multiclassification tasks. Garcia [14] et al. first use graph convolutional networks (GCNs) in FSRE to predict the edge information of the query set nodes using a known subgraph of the support set. Li et al. [43] propose a graph-based model generation module that enables the support sets, query sets, and relation descriptions to generate different classification models to handle FSRE tasks in different tasks. The use of graph neural networks can enrich the relations of text semantics when the class of relations is uncertain. However, the approach used in computing the adjacency matrix is less efficient than the prototype network.

Snell et al. [15] first propose an efficient classification model, the prototype network, based on the idea that there exists a prototype for each class, which is obtained from the average of the embedding vectors of all instances of the support set, and the type to which the query set belongs is classified according to the closest distance principle between the query set and the various class prototypes. Prototype networks are the dominant direction for FSRE tasks due to their efficiency. The following are some of the more representative models in FSRE using the prototype network approach: Proto-HATT [17] improves model robustness and accelerates model convergence by designing instance-level and feature-level attention schemes; MLMAN [18] encodes the instances in the query set and support set and leverages the interaction of local and instance-level information between them to obtain richer semantic representation; REGRAB [44] incorporates global relation graphs with Bayesian meta-learning into the model; TD-proto [45] introduces relation and entity descriptions to enhance prototype networks; ConceptFERE [23] incorporates an entity multiconcept selection module to enhance key entity concept features; CTEG [46] fine-tuning by assigning pseudolabels to untagged data using the domain; HCRP [20] introduce three modules for hybrid prototype learning, contrast learning, and adaptive focus loss to improve the model; SimpleFSRE [21] simplifies the model using a direct join method of relation information; PRM [22] proposes a parameter-free prototype correction method; and MapRE [47] adds a framework that takes into account label uncertainty and label-aware semantic mapping information. However, these models have inferior domain adaptation capabilities.

Moreover, the above studies using the prototype network approach usually do not focus on the interaction information between the support set instances and the query set instances but simply average the embedding matrix of the support set instances for each relation class to obtain the class prototype, failing to exploit the interaction information between instances. Some of the models that have achieved good results also improve the performance of FSRE tasks by incorporating external information and do not take full advantage of the information in the dataset.

We, therefore, propose a data augmentation-based model FREDA for the FSRE task. Firstly, we extract the relation triple information from the support set to fully use the triple information and augment the triple information in the support set instances. Secondly, we successfully reduce the deviation between the obtained and expected relation prototypes by designing an interactive attention module that computes the instances between the support set, the query set, and the relation triple interactively, focusing on the highly interactive parts between the instances. Our method is efficient in capturing similar features between the query set and the support set to obtain more discriminative relation prototypes, and it also gives excellent results in datasets with different domains with strong domain adaptation capability.

3. Methodology

In this section, the parts of our model will be introduced to illustrate the overall framework of our model. Figure 2 shows the comprehensive framework of our model, where the relation triple information is obtained from the support set. Firstly, the support set, query set, and relation triple information go through the same sentence encoder to map to the same semantic space. Secondly, in the interactive attention module, we perform Euclidean distance similarity between the support set and the relation triple representation simultaneously using the query set representation to obtain the similarity weights, thus focusing on highly interactive instances. Then, we fuse the interactive support set and relation triple information through the prototype fusion module. Finally, we obtain the relation semantics belonging to the entity pairs with unknown relations in the query set after the classifier.

3.1. Task Definition

We followed the FSRE task setup for the experiment. According to the N-way-K-shot setting, there are four categories: 5-way-1-shot, 10-way-1-shot, 5-way-5-shot, and 10-way-5-shot. Each class is divided into a training set, a validation set, and a test set, and each dataset contains the support set $S$ and the query set $Q$. The support set is used for model training, the query set is used for task evaluation, and the support set and query sets have the same relation class N. For example, the 2-way–2-shot setup in

Table 1. 2-way–2-shot. Head entity in red font and tail entity in blue font.

Support Set
R1: country of citizenship	Instance1: Charles Gniette was a Belgian field hockey player who competed in the 1920 Summer Olympics. Instance2: Catherine Loyola (born 1986) is a fashion model and beauty queen from the Philippines.
R2: mother	Instance1: Ariston had three other children by Perictione: Glaucon, Adeimantus, and Potone. Instance2: Dylan and Caitlin brought up their three children, Aeronwy, Llewellyn and Colm.
Query Set
R1 or R2	Rugby League Live2 followed in 2012, again developed by Big Ant Studios.

draws two relations from the support set, two instances are extracted from each relation, and the relations of entity pairs present in the query set instances are determined based on the models trained on the support set instances.

In FSRE, each instance consists of three parts: $(x,e, \mathrm{a}\mathrm{n}\mathrm{d} r)$. Here, $x$ represents the sentence of the instance, $e=(e_h, e_t)$ represent the entities, where $e_h$ is the head entity and $e_t$ is the tail entity, and $r$ represents the relation’s name. In addition, we extract the relation triple information $T=(r,e_h, e_t)$ from the instance directly. It is evident that the relation triple information we extract is already present in the instances and can be directly extracted.

3.2. Sentence Encoder

We choose BERT as the encoder, as shown in Figure 3, given an instance $x=\left\{w_1,\dots ,w_c\right\}$, with $x$ containing c tokens. When the instance is encoded by BERT, its corresponding embedding $X=\left\{W_1,\dots ,W_c\right\}$ is obtained. We will select the instances to go through the encoder to form the hidden state corresponding to the start token mentioned by the head entity and the hidden state corresponding to the start token mentioned by the tail entity as the output of the encoder.

$$S_h=\left\{{S_h}_k^i\in {\mathbb{R}}^d;i=1,\dots ,N,k=1,\dots ,K\right\}$$

(1)

$$Q_h=\left\{{Q_h}_j\in {\mathbb{R}}^d;j=1,\dots ,G\right\}$$

(2)

$$S_t=\left\{{S_t}_k^i\in {\mathbb{R}}^d;i=1,\dots ,N,k=1,\dots ,K\right\}$$

(3)

$$Q_t=\left\{{Q_t}_j\in {\mathbb{R}}^d;j=1,\dots ,G\right\}$$

(4)

The $S_h$ and $S_t$ represent the hidden states corresponding to the start token of the head and tail entity mentions in the support set, respectively. The $Q_h$ and $Q_t$ represent the hidden states corresponding to the start token of the head and tail entity mentions in the query set, respectively. The ${S_h}_k^i$ and ${S_t}_k^i$ represent the hidden states corresponding to the start token of the head entity mentions and tail entity mentions, respectively, in the kth instance of the ith relation in the support set. The ${Q_h}_j$ and ${Q_t}_j$ represent the hidden states corresponding to the start token of the head entity mentions and tail entity mentions, respectively, in the jth instance of the query set.

We only utilize the hidden states corresponding to the start tokens of the two entity mentions. By concatenating these two pieces of information, we obtain the required representation for the support set, denoted as $S$, and the query set, denoted as $Q$. This effectively reduces the introduction of excessive and unnecessary parameters:

$$S=cat(S_h, S_t),$$

(5)

$$Q=cat(Q_h, Q_t)$$

(6)

where $S\in { R}^{2d}$ and $Q_h\in { R}^{2d}$. $d$ is the size of the contextual representation of the sentence encoder.

The relation triple information representation is obtained by encoding the relation triples, and we can obtain two types of embeddings: the [CLS] token embedding $T_c$ and the average of all token embeddings $\widetilde{T_p}$. These embeddings are then concatenated to form the desired relation triple information representation:

$$T=cat(T_c,\widetilde{T_p})$$

(7)

where $T\in { R}^{2d}$ belongs to the spliced relation triple information representation. The average of all token embeddings $\widetilde{T_p}\in { R}^d$ is computed as follows:

$$\widetilde{T_p}=\frac{1}{C}\sum _{j=1}^{C}TP\left(j\right).$$

(8)

The $TP\left(j\right)$ in the equation denotes the embedding matrix of the jth token in a relation triple instance, with a fixed token length of $C$ for each instance.

3.3. Interactive Attention Module

Due to the limited number of instances in the support set and previous works ignoring the class information hidden in the query set, they failed to extract the inter-instance features well. Inspired by Zhang et al. [48], we interact the query set instances and relation triple instances, the query set instances, and support set instances by each other. This approach focuses on capturing highly interactive parts between instances to extract relation features better. Firstly, we use the Euclidean distance algorithm to calculate the similarity ${\alpha }_k^i$ and ${\beta }_k^i$ of the support set representation and query set representation, and the relation triple representation and query set representation. They are calculated as follows:

$${\alpha }_k^i=\frac{\sum _{j=1}^G{{\mathrm{e}\mathrm{x}\mathrm{p}(S}_k^i-Q_j)}^2}{\sum _{k=1}^K{\sum _{j=1}^G{\mathrm{e}\mathrm{x}\mathrm{p}(S}_k^i-Q_j)}^2},$$

(9)

$${\beta }_k^i=\frac{\sum _{j=1}^G{{\mathrm{e}\mathrm{x}\mathrm{p}(T}_k^i-Q_j)}^2}{\sum _{k=1}^K{\sum _{j=1}^G{\mathrm{e}\mathrm{x}\mathrm{p}(T}_k^i-Q_j)}^2}$$

(10)

where ${\alpha }_k^i$ denotes the similarity of the kth instance of the ith relation in the support set to the query set, $S_k^i$ denotes the embedding information of the kth instance of the ith relation in the support set, ${\beta }_k^i$ denotes the similarity of the kth instance of the ith relation in the relation triple to the query set, $T_k^i$ denotes the embedding information of the kth instance of the ith relation in the relation triple, and $Q_j$ represents the embedding information of the jth instance of the query set. As shown in Figure 4, in the 2-way–2-shot setup, the information interaction between the support set and the query set, and the support set and the query set, obtain the similarity between them according to the inverse of the difference of the distances between the instances. Then, the similarity of the query set to the instances of the support set is calculated using Equation (9).

Then, we directly multiply the similarity with the corresponding support set representation and relation triple representation. For example, in the support set, when the similarity of an instance is high it means that this instance contributes more to the model test to obtain the correct result, so we give it a higher coefficient. When multiplied together, the corresponding fused support set’s relation prototype $S^{\prime }$ and relation triple’s relation prototype $T^{\prime }$ are obtained:

$$S^{\prime }=\sum _{k=1}^K{\alpha }_k^i\times S,$$

(11)

$$T^{\prime }=\sum _{k=1}^K{\beta }_k^i\times T.$$

(12)

The $S$ is the embedding matrix of the support set after the encoder, and $T$ is the embedding matrix of the relation triple after the encoder.

We have obtained the relation prototype representation of the support set and the relation prototype representation of the relation triple after the information interaction. To present the algorithm of the interaction attention module more clearly, we summarize the experimental steps of the module through the following pseudocode of Algorithm 1.

Algorithm 1 Dataset transformation steps in the Interactive Attention module
Input: support set embedding $S\in {\mathbb{R}}^{2d}$, query set embeddings $Q\in {\mathbb{R}}^{2d}$ and relation triple information embedding $T\in {\mathbb{R}}^{2d}$.
Output: support set embedding $S^{\prime }$ and query set embedding $T^{\prime }$ after the interactive attention module
1:	for episode n = 1 to 4 do
2:	foreach $S_k^i\in S$, $Q_j\in Q$, $T_k^i\in T$ do
3:	use Equation (5) to calculate the similarity ${\alpha }_k^i$ between each instance of the support set and the query set; use Equation (6) to calculate the similarity ${\beta }_k^i$ between each instance of the relation triplet and the query set.
4:	focusing on instances in the support set and relation triplet that are highly similar to the query set through Equations (7) and (8), we obtain $S^{\prime }$ and $T^{\prime }$
5:	end for
6:	end foreach

3.4. Prototype Fusion

Inspired by the fusion method of relation information and instance prototypes propose by Liu et al. [23], we directly add the relation prototype representation $S^{\prime }$of support set instances and the relation prototype representation $T^{\prime }$ of relation triple instances to obtain the final prototype, preventing the influence of harmful parameters on the model:

$$P=S^{\prime }+T^{\prime }$$

(13)

where $P$ denotes the final prototype representation, $P$ ∈ ${\mathbb{R}}^{2d}$.

Our work is a multiclass classification with an N relation class. After experimenting with different loss classes, we chose cross-entropy loss as our loss function:

$$\mathcal{L}=-\sum _{i=1}^N\left(y_i\times \mathit{log}\left({\widehat{y}}_i\right)\right)$$

(14)

where $y_i$ denotes the value of the ith element in the actual label vector, and ${\widehat{y}}_i$ denotes the value of the ith element in the model’s predicted probability vector.

4. Experiment

This section provides a detailed description of our data set, experimental setup, and an in-depth analysis of our experimental analysis and ablation experiments.

4.1. Dataset and Setup

In this work, we evaluate our model on two datasets, FewRel1.0 and FewRel2.0, as shown in

Table 2. FewRel dataset.

Dataset	Source	Apply	Relation Number	Instance Number
		Train	64	44,800
FewRel1.0	Wiki	Val	16	11,200
		Test	20	14,000
	Wiki	Train	64	44,800
FewRel2.0		Val	10	7000
	PubMed	Test	15	10,500

. The FewRel1.0 dataset is derived from Wikipedia, which first initially processed the data by distant supervision and then addressed the noise problem present in distant supervision by crowdsourcing. It contains a total of 100 relations, each with 700 instances, for a total of 70,000 instances. In addition, each sentence has relation names, head entities, and tail entities; 64 of the 100 relations are used for training, 16 for validation, and 20 for testing. In the FewRel2.0 (Domain Adaptation) dataset, while the training set remains unchanged, the validation set and the test set are extracted from the medical database PubMed. The validation set consists of 10 relations, and the test set consists of 15 relations. Seven hundred instances accompany each relation in all sets. Examples from different domains are taken for the validation and test sets to facilitate the domain adaptation capability of the model. The training, validation, and test sets are further divided into a support set for training the model and a query set for prediction.

We experimentally investigate four RE configurations with N of {5, 10} and k of {1, 5}. In the encoder part, we choose base-uncase BERT and CP modules with a feature dimensionality of 768. The maximum length of each sentence we set to 128, and the learning rates are 1 × $e^{-5}$ and 5 × $e^{-6}$ are used depending on the task. All our experiments were performed on computers with Intel R Core i9 13900K/F CPU@5.8 GHz and two sheets with 24 G of video memory GeForce RTX 3090 GPU cards, and the models were implemented on the Pytorch1.12.0 framework. Because of our lack of equipment, the batch size was set to 2 in the configuration 10-way–5-shot and 4 for everything else. Our number of iterations was 30,000, the optimizer was chosen to be AdamW, and the loss was chosen to be a cross-entropy loss.

4.2. Comparative Models

We compare 14 baselines on FewRel. Depending on the encoder, we divided the baselines into three parts: a CNN-based model, a BERT-based model, and a BERT model with a post-training task. The characteristics of each baseline are specified as follows:

• Proto-HATT (Gao et al.; AAAI, 2019) [17] introduces hybrid attention, focusing on feature extraction.
• MLMAN (Ye et al.; ACL, 2019) [13] improves the model using a multilevel matching and aggregation prototype network.
• BERT-PAIR (Gao et al.; EMNLP, 2019) [29] makes use of the addition of auxiliary sentences.
• Proto-BERT (Garcia et al.; ICLR, 2020) [14] preserves important contextual information by constructing multiprototype word embeddings.
• REGRAB (Qu et al.; ICML, 2020) [44] adds Bayesian learning methods with external global relation graphs.
• TD-proto (Yang et al.; CIKM, 2020) [45] considers that general content words are also essential.
• CTEG (Wang et al.; COLING, 2020) [19] adds two types of external information to enhance the information content.
• ConceptFERE (Yang et al.; ACL, 2021) [23] optimizes entity concept information through the selection of entity concept words.
• HCRP (Han et al.; EMNLP, 2021) [20] introduces a hybrid prototype network with task adaptation for focus loss.
• SimpleFSRE (Liu et al.; ACL, 2022) [21] simplifies the model by making direct use of RE.
• MTB (Soares et al.; ACL, 2019) [33] constructs a task-agnostic relation representation from entity chain text.
• CP (Peng et al.; EMNLP, 2020) [49] proposes a pretraining framework for entity masking contrast for RE to deepen the understanding of textual context and type information.
• MapRE (Dong et al.; EMNLP, 2021) [47] incorporates label uncertainty and a label-aware semantic mapping information framework in pretraining and fine-tuning.
• Proto-CNN (Snell et al.; NIPS, 2017) [15] introduces the first prototype network to an FSRE model.

4.3. Experimental Results

Our model is kept consistent with the baseline model we compare in terms of the settings of the embedding and coding layers.

The experiments in the dataset FewRel1.0 are shown in

Table 3. The results of FSRE accuracy experiments on the FewRel1.0 validation/test set. The top-to-bottom encoder includes the use of CNN, the original BERT, and the addition of additional pretrained BERT. Where FREDA denotes the results of our implementation and the others are from results reported in papers or CodaLab.

Encoder	Model	5-Way–1-Shot	5-Way–5-Shot	10-Way–1-Shot	10-Way–5-Shot
CNN	Proto-HATT	72.65/74.52	86.15/88.40	60.13/62.38	76.20/80.45
	MLMAN	75.01/__ __	87.09/90.12	62.48/__ __	77.50/83.05
BERT	BERT-PAIR	85.66/88.32	89.48/93.22	76.84/80.63	81.76/87.02
	Proto-BERT	84.77/89.33	89.54/94.13	76.85/83.41	83.42/90.25
	REGRAB	87.95/90.30	92.54/94.25	80.26/84.09	86.72/89.93
	TD-proto	__ __/84.76	__ __/92.38	__ __/74.32	__ __/85.92
	CTEG	84.72/88.11	92.52/95.25	76.01/81.29	84.89/91.33
	ConceptFERE	__ __/89.21	__ __/90.34	__ __/75.72	__ __/81.82
	HCRP	90.90/93.76	93.22/95.66	84.11/89.95	87.79/92.10
	SimpleFSRE	91.29/94.42	94.05/96.37	86.09/90.73	89.68/93.47
	FREDA	92.23/94.86	94.59/96.52	85.99/91.66	90.40/93.18
	MTB	__ __/91.10	__ __/95.40	__ __/84.30	__ __/91.80
	CP	__ __/95.10	__ __/97.10	__ __/91.20	__ __/94.70
	MapRE	__ __/95.73	__ __/97.84	__ __/93.18	__ __/95.64
	HCRP (CP)	94.10/96.42	96.05/97.96	89.13/93.97	93.10/96.46
	SimpleFSRE (CP)	96.21/96.63	97.07/97.93	93.38/94.94	95.11/96.39
	FREDA (CP)	95.66/97.26	97.31/98.30	91.81/95.32	95.26/96.68
	$∆$	+5.09	+2.39	+8.25	+2.93
	$∆$(CP)	+2.16	+1.20	+4.12	+1.98

, which contains both CNN-based and BERT-based encoders. Whereas the BERT-based approach partly used the original BERT, the other part is based on the original BERT with additional pretraining using Wikipedia data. The last row of the BERT-based and CP-based modules shows the results of our experiments using identical encoders. As can be seen from the table, our results are significantly improved in the test set. Moreover, our model results show a significant improvement compared to the basic models Proto-BERT and CP.

We also conduct experiments on the dataset FewRel2.0 (domain adaptation) to test the domain adaptation capability of our model, as shown in Figure 5. Our model obtains a vast improvement on FewRel2.0 (domain adaptation), which shows that our model has a strong domain adaptation capability and can be better applied to datasets with different domains.

4.4. Analysis of Results

In order to show more clearly that our model has a more fantastic boost in the CP-based encoder module, we show it in the line graph shown in Figure 6. In the figure, we can see that our model RTFRE (CP) effect is significantly better than the other models in all four few-shot settings, which shows the superiority of our model. However, there is a shortcoming in Table 3, where our model is lower than model SimpleFSRE at 10-way–5-shot in the encoder model for BERT-based, although the overall lift is higher. Our analysis suggests that our model is better suited for states with fewer K values, and the smaller experimental result of the boost from the 5-way–5-shot setting could verify our suspicions. Moreover, because of our lack of equipment, our batch was changed to 2 at the 10-way–5-shot, and the minor tuning of the parameters could also be an influence. However, our model test results have a greater boost than our baseline.

On the other hand, to demonstrate that our model yields better relation prototypes across relation classes and that our model is more aggregative when distinguishing between similar relations, we performed a visualization of relation instances on the validation set of the 5-way–1-shot setup. As shown in Figure 7, we conducted two experimental comparisons. Images a and c randomly select five relations, with the same color corresponding to the same relation between the two images. Image a visualizes the clustering of various relations in the dataset using the SimpleFSRE model, while image c visualizes the clustering of various relations in our model. These two images are used to compare the aggregation capabilities of our model and the SimpleFSRE model across different relations. Images b and d specify the extraction of five similar relations, with the same color indicating the same relation between the two images. Image b visualizes the clustering of similar relations in the dataset using the SimpleFSRE model, while image d visualizes the clustering of similar relations in our model. In the figure, we can observe the following: our model has a stronger aggregation ability than the baseline model in the five randomly selected relation classes; in the five selected similar relations, both b and d have a more vital aggregation ability, while our graph has a somewhat denser aggregation in the same relation class, indicating that our model is more capable of distinguishing similar relations.

4.5. Ablation Experiments

In this section, to analyze the various components of the model that played their part, we conducted ablation experiments on the FewRel1.0 dataset using an encoder of BERT as shown in

Table 4. The results of ablation experiments.

Model	5-Way-1-Shot	5-Way-5-Shot	10-Way-1-Shot	10-Way-5-Shot	Average
SimpleFSRE	91.29/94.42	94.05/96.37	86.09/90.73	89.68/93.47	93.74
+Triple	92.30/94.73	94.65/96.41	85.38/91.39	90.47/92.94	93.86
FREDA	92.23/94.86	94.59/96.52	85.99/91.66	90.50/93.23	94.06

. SimpleFSRE was the baseline of our model. Then, the relation information descriptions were removed, and the relation triple information drawn from the support set samples was added to the experiments. Finally, we added the interactive attention module, focusing on the parts of the query set that were highly interactive with each instance in the support set and relational triple information, respectively, with highly interactive instances receiving higher coefficient values. The test set results of the experiments are submitted to the Codalab website for evaluation, ensuring the fairness and credibility of the results.

Table 4 shows that the test results improve significantly in the k-way-1 setting when only the relation triple information is added but improve less in the 5-way–5-shot setting and drop in the 10-way–5-shot setting instead. This suggests that our model is more suitable in a less sample setting. When the interactive attention module is added, our model improves in all four settings compared to adding only the relation triple information but still relatively more in the K = 1 setting. The analysis of this ablation experiment shows that the addition of both the relational triad information and the interactive attention module helped our model.

5. Conclusions

The metric learning-based FSRE task aims to obtain a better projection function through embedding information in a small number of support set instances under continuous iteration of the model and to better predict the semantic relations of entity pairs in the query set instances. This relies heavily on the quality of the support set instances in the embedding space, but the support set instances by the FSRE are underutilized. Therefore, we include the relation triple information through data augmentation and an interactive attention module through the Euclidean distance algorithm, both of which are designed to improve the quality of the embedding space of the support set. In addition, the model in this paper is conducted without the use of external information and, as we know from the results of previous experiments on models with external information, the addition of external information can help improve the performance of the model, and we will try to add external information to our model in the next experiments. We will also try to apply our model to other domains such as healthcare and food, to achieve its value in reality, as our model has a strong domain adaptation capability.