Zero-Shot Relation Triple Extraction with Prompts for Low-Resource Languages

Abstract

Although low-resource relation extraction is vital in knowledge construction and characterization, more research is needed on the generalization of unknown relation types. To fill the gap in the study of low-resource (Uyghur) relation extraction methods, we created a zero-shot with a quick relation extraction task setup. Each triplet extracted from an input phrase consists of the subject, relation type, and object. This paper suggests generating structured texts by urging language models to provide related instances. Our model consists of two modules: relation generator and relation and triplet extractor. We use the Uyghur relation prompt in the relation generator stage to generate new synthetic data. In the relation and triple extraction stage, we use the new data to extract the relation triplets in the sentence. We use multi-language model prompts and structured text techniques to offer a structured relation prompt template. This method is the first research that extends relation triplet extraction to a zero-shot setting for Uyghur datasets. Experimental results show that our method achieves a maximum weighted average F1 score of 47.39%.

1. Introduction

Relation and triplet extraction (RE) is a critical topic in information extraction (IE), and it may be used for a variety of natural language processing (NLP) tasks [1,2,3]. The vast databases of labeled samples required by conventional approaches, on the other hand, are frequently expensive to annotate [4]. However, preceding zero-shot relation work does not necessitate the extraction of the whole relation triplet. Chen and Li [3] present zero-shot BERT, a unique multi-task learning approach, to directly estimate unknown relations without hand-crafted feature tagging. There are numerous methods in use currently to address the problems caused by data scarcity. Distantly supervised [4,5] methods are fast enough to construct large-scale corpora and corpus annotation is inexpensive, but the quality of annotation produced by this method is not as good as the results of manual human annotation. The work objective could also be written so that the label space is unrestricted.

However, since relation triplet extraction is a structured prediction problem rather than a sequence classification task, the existing formulations cannot be used in this manner [6]. John Koutsikakis et al. [7] present GREEK-BERT, a BERT-based language model for contemporary Greek that is monolingual. Xin Xu et al. [8] present an empirical investigation into the development of relational data extraction systems in low-resource environments. Based on recent pre-trained language models, they comprehensively investigate two methods for evaluating efficacy in low-resource settings: data augmentation technologies and self-training to generate more labeled in-domain data. Pundlik et al. [8] worked on a multi-domain Hindi dataset. The architecture described in the article [8] consisted of two stages. Yadav et al. [9] showed that Sentiment Analysis for the mixed Hindi language could be performed using three approaches. The first way was to use a neural network to classify previously known words.

Wikipedia [10] constructs this dataset using a snapshot of English Wikipedia. The titles of Wikipedia articles are considered brief texts, whereas the abstracts of all Wikipedia articles are used to build the associated lengthy documents. This dataset includes a total of 30,000 short texts and 1,747,988 long documents. This dataset is compiled by DBLP [10] using the bibliography database. The titles of computer science literature represent brief texts, whereas the abstracts of all published papers are collected to form the corresponding lengthy documents. This dataset consists statistically of 81,479 short readers and 480,558 long records. Programmable web [11] generates a real-world dataset for service recommendation and description augmentation. Fei Cheng et al. [12] provide an open-source natural language processing toolbox for extracting Japanese medical data. They create a pipeline system consisting of three components for identifying medical entities, categorizing entity modalities, and extracting relations. Existing relation extraction datasets for the Russian language are few and comprise a limited number of instances. Hence, D.I. Gordeev et al. [13] chose to develop RURED, a new Ontonotes-based sentence-level named entity and relation extraction dataset. The collection includes over 500 annotated texts and over 5000 labeled relationships. Leonhard Hennig et al. [14] present a German-language dataset, which is human-annotated with 20 coarse and fine-grained entity types and entity-linking information for geographically linkable entities. This first German-language dataset combines annotations for Named Entity Recognition and Relation Extraction. Ahmed Hamdi et al. [15] present the creation of the NewsEye resource, a multilingual dataset for named entity identification and linking, enhanced with attitudes towards named entities. The collection consists of diachronic historical newspaper articles published between 1850 and 1950 in French, German, Finnish, and Swedish.

Due to the lack of low-resource datasets, research on relation and triplet extraction has mostly been restricted to the English, German, French, Russian, Japanese, Finnish, Swedish, Arabic, and Hindi languages [11]. In this study, we present an alternate relation extraction method for a low-resource language (Uyghur) that can extract facts of new kinds that were not previously stated or observed. Despite the lack of annotated training examples, including the test relation labels, our method aims to extract triplets of the subject, object, and relation type from each sentence.

Table 1. Task examples of triplet extraction.

Sentence	Subject	Object
Mr. Alim died in Urumqi.	Mr. Alim	In Urumqi

In Table 1, Sentence, Subject, and Object refer to the sentence, first entity, and second entity, respectively.

and

Table 2. Task parameters against our proposed Zero-Shot prompts retreated to triplet extraction.

Task Setting	Input	Output	Supervision
Relation Classification	$S\mathrm{e}ntence,entit{y}_{head},entit{y}_{tail}$	$label$	Full
Zero-Shot Relation Classification	$S\mathrm{e}ntence,entit{y}_{head},entit{y}_{tail}$	$label$	Zero-Shot
Relation Triple Extraction	$Sentence$	$entit{y}_{head},entit{y}_{tail},label$	Full
Zero-Shot-Prompts Relation Triple Extraction	$Sentence$	$entit{y}_{head},entit{y}_{tail},label$	Zero-Shot

In Table 2, $S\mathrm{e}ntence,entit{y}_{head},entit{y}_{tail}$ refer to the sentence, first entity, and second entity, respectively.

provide a task example and overview of task settings for easy comparison. This is the first research that, to our knowledge, extends relation triplet extraction to a zero-shot setting for Uyghur datasets. As a result, we suggest Relation Prompt, which redefines the zero-shot task as the production of new data for Uyghur languages. The fundamental idea is to use relational prompts to motivate a language model to produce an artificial training sample capable of expressing the necessary relations. These synthetic data can then be used to train another process of extracting a model of the triplet [16].

Overall, our significant contributions are as follows:

• In English, words and prepositions are separated by spaces, but in Uyghur, the entity and the suffixes following the entity are linked together. In Uyghur triplet extraction, entities and suffixes are not handled like Uyghur root extraction because entities may contain more than one word, so we have to consider the suffix of the last word in the entity.
• The Zero-Shot relation triple extraction dataset, known as Zero-Shot-Prompt-RE, is made up of Uyghur. In our collection, we have over 23,652 examples. There are 24 related classes that can each be an instance.
• This is the first research that, to our knowledge, extends relation triplet extraction to a zero-shot setting for Uyghur datasets.
• We use prompt-build synthetic relations to provide structured texts in order to make Zero-Shot-Prompt-Relation-Extraction.

2. Related Work

RE is a crucial pre-requisite job for creating large-scale knowledge graphs (KG), which can be used for automated responses, information retrieval, and other natural language processing (NLP) activities [17,18,19,20,21]. Previous papers on RE had a pipelined approach: first, all entities in the input phrase are identified using named entity recognition (NER), then, relational classification (RC) is applied to pairs of extracted entities [13]. Pipelined approaches typically have an issue with error propagation and ignore the connection between the two phases [20]. Ref. [21] provides a reading comprehension method in which one or more natural-language questions are associated with each relation slot. To improve the performance of relation classification systems, Ref. [22] performs zero-shot relation classification by using relation descriptions, current textual entailment models, and freely available textual entailment datasets. Ref. [23] proposes a unique dataset for Reading Comprehension (RC) for neural techniques in language understanding. Using GPT-3 as a case study, Ref. [24] shows that zero-shot prompts can significantly outperform those that use few-shot examples. This analysis motivates the rethinking of the role of prompts in controlling and evaluating powerful language models. Refs. [25,26,27] explore techniques for exploiting the capacity of narratives and cultural anchors to encode nuanced intentions. In natural language processing, prompting-based methods have shown promise in novel paradigms for zero-shot or few-shot inference [28]. Another advantage is adapting extensive language models to new jobs without needing somewhat expensive fine-tuning [29]. In order to combine many pathways and learn to share strength in a single RNN expressing logical composition across all relations, Refs. [30,31,32] present joint learning and neural attention modeling. However, such methods have many disadvantages for more difficult tasks such as triplet extraction [33,34]. Refs. [35,36] construct a Uyghur grammatical dictionary and provide extensive Uyghur grammatical information. Ref. [37] proposed a hybrid joint extraction model for supervised Uyghur datasets. To extend Uyghur RE research, Ref. [38] proposed a hybrid method to expand the Uyghur-named entity relation corpus. Sonali Rajesh Shah et al. [39] analyze, review, and discuss the methods, algorithms, and challenges researchers face when performing indigenous language recognition. The primary objective of this assessment is to comprehend the recent work undertaken in South Africa for indigenous languages. The trends in the field of SA are being uncovered by analyzing 23 articles. Machine learning, deep learning, and sophisticated deep learning algorithms were utilized in 67% of the reviewed publications. Twenty-nine percent of researchers have employed a lexicon-based methodology. SVM (Support Vector Machine) and LR (Logical Regression) outperformed all other machine learning approaches. CHENHE DONG et al. [39] provide a historical overview of deep learning research on natural language generation, highlighting the unique nature of the problems to be addressed. It begins with a contrasting generation with language comprehension, establishing fundamental concepts regarding the objectives, datasets, and deep learning methods. Following is a section of evaluation metrics derived from the output of generation systems, illustrating the possible performance types and the areas of difficulty. Finally, a list of open problems presents natural language generation research’s main challenges and future orientations. Nasrin Taghizadeh [40] proposed a cross-language method for relation extraction, which uses the training data of other languages and trains a model for relation extraction from Arabic text. The task is supervised learning, in which several lexical and syntactic features are considered. The proposed method mainly relies on the Universal Dependency (UD) parsing and the similarity of UD trees in different languages. Regarding UD parse trees, all the features for training classifiers are extracted and represented in a universal space. Yuna Hur et al. [41] suggest Entity-Perceived Context representation in Korean to provide a better capacity for grasping the meaning of entities and considering linguistic qualities in Korean.

3. Zero-Shot-Prompt-RE: Methodology

The majority of zero-shot research has been conducted in English. We provide a Uyghur-Relation-Prompt that employs relation types as prompts to produce Uyghur synthetic relation instances of target unknown relation types in order to extract triplets for unknown relation types in Uyghur-Zero-Shot-Prompt-RE. There are more research works on relation extraction using English. However, there needs to be more research on Uyghur, which belongs to a different language family to English and whose grammar is more complex with weaker letters and more or missing affixes. Therefore, this paper implements a rule- and template-based approach to Uyghur relation extraction research.

3.1. Task Definition

In English, words and prepositions are separated by spaces, but in Uyghur, the entity and the suffixes following the entity are linked together. In Uyghur triad extraction, entities and suffixes are not handled like Uyghur root extraction because entities may contain more than one word, so we have to consider the suffix of the last word in the entity. Uyghur-Zero-Shot-Prompt-RE aims to use relational and entity cues to learn and produce synthetic data from seen datasets, $D_{\mathrm{s}}$, and to extract triplets from unseen test datasets, $D_{\mathrm{u}}$. $D_{\mathrm{s}}$ and $D_{\mathrm{u}}$ are described as $D=\left(S,T,R\right)$, where $S$ is the Uyghur sentence: “ئالمىرە بېيجىڭ ئۇنۋېرسىتىدا ئۇقۇيدۇ.—Almira is a student at Peking University” the seen relation is “educated at (P69)”—“ئالىي مەكتەپنى پۈتتۈرگەن (P69)”, T = (ئالمىرە، ئالىي مەكتەپنى پۈتتۈرگەن (P69) بېيجىڭ ئۇنۋېرسىتى،) is the output triplets, and R is the relation. “ئالمىرە—Alimire ” is a person’s name, “بېيجىڭ ئۇنۋېرسىتىدا— at Peking University” is a geographical named entity, a relation triple is defined as $\left(e_{head},e_{tail},y\right)$, which denotes the head entity—“ئالمىرە”, tail entity—“بېيجىڭ ئۇنۋېرسىتى + دا”, and relation type –“educated at (P69)”—“ئالىي مەكتەپنى پۈتتۈرگەن (P69)”, respectively. In the tail entity, “بېيجىڭ ئۇنۋېرسىتىدا—at Peking University”, the entity to be extracted should be Peking University, without the preposition. The seen and unseen relation type sets are represented as $R_{\mathrm{s}}=r_s^1+r_s^2\mathrm{\dots }+r_s^n$ and $R_{\mathrm{u}}=r_u^1+r_u^2\mathrm{\dots }+r_u^m$, where $n=\left|R_{\mathrm{s}}\right|$ and $m=\left|R_{\mathrm{u}}\right|$ are the sizes of the known and unknown relation type sets, respectively.

First entities, second entities, and relation labels are shown in blue, orange, and dark red, respectively, in Figure 1a,b. The relation generator (a) accepts the relation label as input and returns the context and entity combination as output. The relation extractor (b) accepts a phrase as input and returns a relation triplet consisting of an entity couple and a relation label.

Usually, we define the knowledge graph formally as $G=\left\{\epsilon ,R,F\right\}$, where $\epsilon ,R,F$ denote the set of entities, relations, and facts, respectively. A fact is denoted as a triplet, T. The notation of the specific triadic relation extraction design and its description are shown in

Table 3. Common symbolic descriptions in relationship extraction.

Notation	Description
$G=\left\{\epsilon ,R,F\right\}$	A knowledge graph
$F$	Facts
$D=\left(S,T,R\right)$	Sentence, Triplet, Relation
$T_{\mathrm{u}}=\left(e_{head},e_{tail},y\right)$	A Uyghur triple of head, relation, and tail
$D_{\mathrm{s}}$	Seen datasets
$D_{\mathrm{u}}$	Unseen datasets
$U_g$	The Uyghur relation generator
$U_{\mathrm{e}}$	The relation detector
$Y_{\mathrm{s}}$	The set of seen relational types
$Y_{\mathrm{u}}$	The set of unseen relational types
$D_{new\_u}$	The labeled unseen dataset
$D_{new\_s}$	The labeled seen dataset
$W=\left[w_1,w_2,\mathrm{\dots }{w}_n\right]$	The conditional generation probability for each sequence
$R_{\mathrm{s}}=r_s^1+r_s^2\mathrm{\dots }+r_s^n$	The seen relation type sets
$R_{\mathrm{u}}=r_u^1+r_u^2\mathrm{\dots }+r_u^m$	The unseen relation type sets
$m=\left|R_{\mathrm{u}}\right|$	The sizes of the unseen relation type sets
$n=\left|R_{\mathrm{s}}\right|$	The sizes of the seen relation type sets

.

3.2. Relation Generator

By conditioning the language model on the target’s unknown relation types, we use relation prompts to produce synthetic sentences. Uyghur is a minority language with fewer data resources [42,43]. While using fine-tuning, the training cost can be meager: if the method of deriving feature vectors is used for transfer learning, the later training cost is light, the CPU is entirely stress-free, and no deep learning machine can perform it [36,37,38,39]. As regards what is suitable for small data sets, for cases where the dataset itself is small (thousands of sentences), it is not realistic to train an extensive neural network with tens of millions of parameters from scratch because the more significant the model, the greater the amount of data, and overfitting cannot be avoided. We propose the following algorithm to handle the Uyghur-Zero-Shot-Prompt-RE:

As shown in Algorithm 1, the $U_g$ is the Uyghur relation generator; the $U_e$ is the relation detector. As shown in Figure 1a, the relation takes an input: Relation: “label”, which is a structured prompt. The outputs are structured prompts in the form of context: “Mr. Alim died in Urumqi.—”, subject: “Alim— ”, object: “Urumqi— ”. The probability of the next word given all the previous ones can be used to construct a statistical language model [10]. The purpose is to calculate the conditional generation probability for each sequence, $W=\left[w_1,w_2,\mathrm{\dots }{w}_n\right]$:

$$p\left(w\right)={\displaystyle \prod _{i=1}^{n}p\left(w_i|w<i\right)},$$

(1)

Considering the alphabetic weakness of the Uyghur language and the multiple addition or loss of affixes, we use a prompt-based approach. Neural networks typically generate category probabilities using a “softmax” layer that converts the logit [44], $Ou{t}_i$, calculated for each category into a probability, $p$, by comparing $Ou{t}_i$ with other logarithms. The $T$ is the temperature, $V$ is the vocabulary size.

$$p\left(x_i|x_i<i\right)=\frac{\exp \left(Ou{t}_i/T\right)}{{\displaystyle \sum _{j=1}^{\left|V\right|}\exp \left(Ou{t}_j/T\right)}},$$

(2)

By dividing the output sequences based on the particular phrases “Context: Mr. Alim died in Urumqi.—”, first entity: “Alim—”, second Entity: “Urumqi—”. The output sequence of the relationship generator is used to extract the relational triples. Suppose an error occurs on the decoding side of the relationship generator and an entity does not exist in the formed context. In that case, we discard the sample and continue to produce new data until the intended data sample is available.

Algorithm 1: Zero-Shot-Prompt-RE for Uyghur

Input: Dataset = (sentence, triplets, relation types)

Output: Extracted Uyghur Triplets-$T_{\mathrm{u}}$

$\mathbf{Require}\mathbf{:} \mathrm{Uyghur} \mathrm{data} \mathrm{preprocessing},D_{\mathrm{s}}$$, D_{\mathrm{u}}$$, U_g$$U_{\mathrm{e}}$$,Y_{\mathrm{s}}\cap Y_u=\varnothing .$

$1: \mathrm{If} Y_{\mathrm{s}}\cap Y_u=\varnothing$

2: Suffix = {دا،دە،تا،تە،نىڭ،غا،قا،گە،كە....}

3: SELECT Entities with suffixes

$4:\mathrm{Uyghur} \mathrm{entity} \mathrm{and} \mathrm{suffix} \mathrm{separation}\left(D_u,D_s\right)\to \left(D_{new\_u},D_{new\_s}\right)$

$5:Train\left(U_g,D_{new\_s}\right)\to U_{g,finetuning}$

$6:Train\left(U_{\mathrm{e}},D_{new\_s}\right)\to U_{\mathrm{e},finetuning}$

$7:Generate\left(U_{g,finetune},Y_u\right)\to D_{synthetic}$

$8:Train\left(U_{e,finetune},D_{synthetic}\right)\to U_{g,final}$

$9:\mathrm{Pr}edict\left(U_{g,final},S_u\right)\to {\stackrel{\wedge }{Triplet}}_u$

$10:Return Extracted Triplets{\stackrel{\wedge }{Triplet}}_u$

As can be seen from Figure 2, the second entity “ئۈرۈمچىدە = ئۈرۈمچى + دە —In Urumqi”, the correct result is the “ئۈرۈمچى—Urumqi”, without the “ دە—In” suffix.

3.3. Relation Detector

In this section, we use a sequence-to-sequence approach regarding NRE as a novel labeling problem, generated with greedy decoding [44,45,46,47]. We may create model outputs without any initial decoder input to predict a single relation triplet (subject, object, relation type) in a given sentence, s. If an object or relation type is invalid, we regard it as a null prediction for that sample. As shown in Algorithm 1, the prerequisite is that the set of visible relational types, $Y_{\mathrm{s}}$, and the set of invisible relational types, $Y_{\mathrm{u}}$, cannot intersect. $U_e$ is fine-tuned for the labeled dataset, $D_{new\_s}$, (line 6) after processing based on Uyghur entities and entity suffixes (line 1), and then reformulated on the synthetic sentences (line 8). The test sentences, $S_u$, are then applied to predict and extract relation triplets, ${\stackrel{\wedge }{Triplet}}_u$ (lines 8 and 9). Our relationship extractor is not the same as a traditional relationship extractor, which is used as a prompt input sentence: “” in the form of “Context: s”. Additionally, the structured output form is “subject: ${\mathrm{e}}_{head}$, object: ${\mathrm{e}}_{\mathrm{ta}il}$, Relation: $y$” with a single pair of entities, ${\mathrm{e}}_{head}$ and ${\mathrm{e}}_{\mathrm{ta}il}$, meeting the relation type, $y$.

Our method (Zero-Shot-Prompt-Uyghur-RE) accepts “Context:, subject: ${\mathrm{e}}_{head}$, object: ${\mathrm{e}}_{\mathrm{ta}il}$, Relation:” as decoder input and produces $y$ as relation type. As a result, because this modification influences model prediction, our solution naturally includes Zero-Shot-Prompt-RE.

3.4. Extracting Uyghur Multiple Triplets

Uyghur is a multi-featured language, so in this section, we also suggest a generation decoding approach to increase zero-shot extraction performance for phrases with numerous triplets. For the Relation Prompt Production of synthetic data, each sample contains a single relation triplet [48,49,50,51,52,53]. Therefore, traditional relation extraction models are likely to perform poorly with our framework for multi-triplet Zero-Shot-Prompt-RE since they often presume that training data may include several triplets per sentence. There is a multi-turn question-answering inference approach [44] that may alleviate this issue. As a result, we offer Triplet Search Decoding, which enhances the relation extractor’s multi-triplet Zero-Shot-Prompt-RE [54].

When using a traditional relationship extractor [16] that takes a sentence cue as input and outputs a relationship category in an auto-regressive manner, multiple relationships cannot be resolved. Although our triplet search decoding can produce many lines, each of which corresponds to a distinct candidate-related triplet, we then filter the final output sequences using a probability threshold. Starting from the entity sequence “subject: ”, it follows from our template that the next generated token should be the first token of the subject, such as “Alim—”. The probability for the $i$th potential token of the subject is represented as $\mathrm{p}\left({\mathrm{e}}_{head},i\right)$. The specific sub-sequence “object: ${\mathrm{e}}_{tail}$” is then formed after greedy decoding of the sequence. The first token of the second entity, such as “Urumqi—ئۈرۈمچى”, will then come next. The $p\left(e_{tail,j}|e_{head,i}\right)$ represents the probability of the initial token for the $j$th tail entity. As a result, the generation is branched so that there will be an additional $b$ sequence $p\left(e_{tail,j}|e_{head,i}\right)$ for each head entity. The special sub-sequence “Relation:” is then generated after greedy decoding of the sequence. The initial token of the relation label, such as “death—” in “Place of death—”, will be the next token to be generated. We branch the generation so that there will be an additional $b$ sequence $p\left(y_k|e_{head,i},e_{tail,j}\right)$ for each pair of subjects and objects. We only examine the likelihood of the first token because it can primarily predict the entity’s subsequent tokens [55,56,57]. The entire inference probability is represented as:

$$\begin{array}{l}p\left(triple{t}_{i,j,k}\right)=p\left(entit{y}_{head,i},entit{y}_{tail,j},y_k\right)=p\left(y_k|entit{y}_{head,i},entit{y}_{tail,j}\right)\\ \cdot p\left(entit{y}_{tail,j}|entit{y}_{head,i}\right)\cdot p\left(entit{y}_{head,i}\right)\end{array}$$

(3)

Unlike other decoding methods, such as beam search, triplet search decoding takes advantage of the particular triplet structure.

4. Experiments

4.1. Datasets

Uyghur Relation Extraction datasets for Zero-Shot-Prompt-RE are created by a distantly supervised Wikidata knowledge base and Wikipedia articles, as can be seen in

Table 4. Datasets statistics in Uyghur languages.

	Sentence-Length	Entities	Samples	Relations
UyRE-ZSP	20.8	20,742	23,652	24

. We use the same methodology as [3,50,58] to divide the data into known and unknown label sets randomly.

In our experiments, we have compared different sets of unknown relation types of size $m\in \left\{5,10\right\}$ to confirm the effect of the size of the set of relation types on the experimental results.

4.2. Experimental Settings

Based on Uyghur Relation Extraction for Zero-Shot-Prompt-RE (UyRE-ZSP), the l-r is set at 3 × 10⁻⁴ and a batch size of 64. We employ the AdamW optimizer during the training procedure [45]. For each relationship, a set number of phrases will be generated. We fine-tune the pre-trained mBERT [38] for Uyghur, the relation extractor.

To evaluate ZSP RTE, we test triplet extraction separately for phrases with a single triplet and sentences with several triplets. To assess multiple triplet extraction, we use the Micro F1 metric, which is often employed in structured pre-diction tasks [51] and provides precision (P) and recall (R). As each phrase contains just a single possible triplet, the Accuracy metric is used to evaluate single triplet extraction (Acc.). We examine UyRE ZSP by [54] using the Macro F1 measure.

To evaluate the model’s performance for relational triple extraction [28], we introduced Precision (Prec.), Recall (Rec.), and F1-measures as evaluation criteria:

$$P=\frac{TP}{TP+FP}\times 100$$

(4)

$$R=\frac{TP}{TP+FN}\times 100$$

(5)

$$F1\_score=\frac{2PR}{P+R}\times 100$$

(6)

where $TP$ indicates the total number of positive and anticipated positive labels and $TN$ represents the number of tags that are negative and expected to be negative, FP represents the number of brands projected to be positive but negative. The number of positive but expected to be harmful labels is denoted by the symbol $FN$ [28].

4.3. Main Experimental Results

Triplet Extraction: 

Table 5. Results for our method in different relation types in Uyghur languages.

	Method	Single Triplet	Multi Triplet
		Acc.	Prec.	Rec.	F1
m = 5	Table Sequence [51]	11.73	50.00	4.76	8.69
	NoGen [10]	8.32	83.33	6.17	11.49
	Relation Prompt [6]	13.65	71.25	26.51	38.64
	Zero-Shot-RTE for Uyghur	13.28	73.97	30.17	42.86
	Zero-Shot-Prompt-RTE for Uyghur (Our suggested model)	16.35	69.23	35.29	46.75
m = 10	Table Sequence [51]	9.11	50.00	2.38	4.54
	NoGen [10]	6.58	75.00	5.83	0.11
	Relation Prompt [6]	14.56	71.43	9.26	16.39
	Zero-Shot-RTE for Uyghur	11.54	76.25	31.69	44.77
	Zero-Shot-Prompt-RTE for Uyghur (Our suggested model)	16.48	78.46	33.95	47.39

compares Relation Prompt-Uyghur to the baselines on Zero-Shot-Prompt-Relation Triplet Extraction (ZSP_RTE) for the UyRE-ZSP datasets. Regarding accuracy and F1 metrics, our method consistently outperforms the baseline methods in single-triplet and multi-triplet evaluations. Uyghur is one of the most morphologically complex and adhesive languages, in which the composition of words and their meanings rely on complex forms of affixation to represent them. The affixes not only change the root word’s importance but also determine a word’s role in a sentence. Therefore, the correct separation of stems and affixes is necessary to represent the word’s true meaning as a whole correctly. Consequently, it requires extra processing of data; additional letters will be added or lost when the word root is extracted; this should be handled according to the Prompt to avoid the case of an extraction error.

We have conducted comparative experiments between Zero-Shot-Prompt-RE and the baseline method in Table 5. Our method consistently outperformed the baseline method in terms of accuracy and F1 metrics in both single triad and multiple triplet results [10,51]. We also note that Zero-Shot-Prompt-Relation Extraction (ZSP_RE) in Uyghur performs much better than a third-party Table Sequence [50,51,52], especially in multi-trigram extraction, suggesting that the Zero-Shot-Prompt-RE approach in Uyghur is practical. However, our proposed Uyghur entity and suffix processing, relation extractor, and decoder technique successfully resolve this difficulty by automatically listing and arranging many triples accurately at the time of inference, without the extracted entities carrying a suffix, as in previous work [39,44].

Relation Extraction: Zero-Shot-Prompt-RE for Uyghur includes information about entity pairs in the hints and supports zero-shot relation classification work without additional training.

Table 6. Results for our suggested method in Uyghur Zero-Shot-Prompt-Relation Extraction.

	Method	Prec.	Rec.	F1
m = 5	R-BERT [51]	88.40	13.40	23.28
	ZS-BERT [3]	73.81	32.63	45.25
	Relation Prompt [6]	66.67	53.84	59.57
	Zero-Shot-Prompt-RE for Uyghur (Our suggested model)	70.89	57.55	63.35
m = 10	R-BERT [51]	88.73	12.67	22.18
	ZS-BERT [3]	74.32	30.55	43.30
	Relation Prompt [6]	82.25	38.75	52.68
	Zero-Shot-Prompt-RE for Uyghur (Our suggested model)	82.02	48.39	60.87

shows that, compared to previous state-of-the-art ZS-BERT techniques [12], our approach maintains relatively good F1 performance for our relational classification when the set of unseen relational labels increases. It may be possible to scale better to larger sets of unseen tags based on relational hints, which is very friendly for applications of relational triad extraction in open domains. The reason for the low recall is that when the samples are labeled, most positive samples are mistakenly labeled as negative samples. This makes it harder for the model to learn from positive examples, which lowers the accuracy of negative samples while positive samples are predicted as negative samples. To solve the problem, the next step is to fix the samples that were wrongly labeled and watch the results to see if anything has changed.

5. Analysis

5.1. Ablation Study

The generated data’s accuracy is crucial to our Zero-Shot-Prompt-Relation Triplet Extraction (ZSP_RTE) technique’s success. Therefore, we contrast a variety of genuine and artificial data sets. We count the number of distinct words and other items in the texts. The accuracy of the data generated is critical to the success of our Zero-Shot Prompt-Relation Triplet extraction technique. Therefore, we compared the kinds of natural and artificial data samples and performed comparative experiments.

Table 7. Data diversity comparison.

Model	Sentences	Entities	Words
Our Data	2451	2859	10,356
Generated Data	2451	3122	7452

shows that the synthetic data sentences have more entities with a variety of unique components. However, the generated sentences have fewer individual words and less lexical diversity.

However, the generated sentences have a lesser vocabulary diversity. On the other hand, non-entity words account for the majority of the total uncommon words.

Our suggested Zero-Shot-Prompt-Relation Triplet Extraction approach relies significantly on the quality of the produced data. Thus, we examine a variety of samples of actual and fake data. Specifically, we count the unique words and items in the texts. We used the UyRE-ZSP validation set to produce five-label sentences and an equivalent number of synthetic phrases for comparison. Table 7 demonstrates that the created sentences include various unique items.

Nevertheless, the resulting phrases include fewer unique words overall, which may be explained by the fact that entity names are often unique. The generator language model has been exposed to many unique entity names during extensive pre-training. On the other hand, non-entity words account for the majority of the overall number of unique words. By employing prompts to condition the production of sentences explicitly for unknown connection labels, the output sentences may include less context-specific information.

Table 8. Ablation results of Uyghur multi-triplet extraction.

Model	F1	△F1
Full method	26.23
Uyghur Triplet Search Decoding	12.33	−13.9
Extractor fine-tuning	14.54	−11.69

shows the results of an ablation study that was performed to test how well our decoding method and task-specific fine-tuning worked for the Uyghur visible relation set on the multi-triple recombination Zero-Shot-Prompt-Relation Extraction. The comparison was conducted on the UyRE-ZSP validation set, which contains ten invisible identifiers. This large performance disparity indicates that triplet search decoding is essential for Zero-Shot-Prompt-Relation Triplet Extraction for Uyghur with multiple triplets, implying that the enumeration and classification of associative triplet candidates are of adequate quality. Second, we observe that the relation extractor’s performance degrades significantly when the visible relation samples from the column set are not fine-tuned prior to the final fine-tuning of the synthetic samples generated by the invisible labels.

When the size of the unseen label set, $\mathrm{m}$, increases, our technique can retain a relatively good classification F1 performance, while Zero-Shot-Prompt-RE exhibits a more pronounced decline in performance.

5.2. Qualitative Analysis

Figure 3 shows a variety of real and created samples to demonstrate how the relation data generator generalizes to real-world relations. Real sentences and related labels were compiled from factual articles.

Given the relations “Mother—ئاپىسى”, “Spouse—جۆرىسى-”, and “Place of birth—تۇغۇلغان جايى.”, the generator can generate logical statements. It can also discover the proper semantic meaning of the relations. To test the generality of the relational data generator to relationships in the wild, we provide several actual and created examples in Figure 3. The connection labels and actual phrases were culled from articles of fact. With the relations “Mother—ئاپىسى”, “Spouse—جۆرىسى- “, and “Place of birth—-تۇغۇلغان جايى.”, the generator can detect the correct semantics of the relations and generate grammatically sound phrases. The produced head and tail entity pairs may often fit the specified relations and have similar contexts to the actual phrases. In the second instance of the connection ‘spouse’, however, the created entity pairings do not fit the relationship’s intended meaning despite the marriage relationship. In contrast, the connection implied by the produced words is more akin to “get married”. These results imply that a potential area for future development may be to make the head and tail entities created by the system correspond more closely to the specified connection.

5.3. Effect of Generated Data Size

We look into how the number of artificial samples affects how well the multi-triplet Zero-Shot-Prompt-Relation Triplet Extraction for Uyghur works.

Our Zero-Shot-Prompt-Relation Triplet Extraction method depends heavily on the quality of the generated data. Therefore, we compare the diversity of real and synthetic data samples. We measure the number of unique words and entities in the text. We used UyRE-ZSL validation set sentences with five unique tags and generated an equal number of synthetic sentences for comparison. Figure 3 shows that the diversity of unique entities is more significant for the generated sentences. However, the overall unique vocabulary diversity of the generated sentences is lower. These results may be because entity names are unique, and the generator language model has seen many unique entity names in large-scale pre-training. On the other hand, the overall unique vocabulary is mainly determined by the non-entity vocabulary. The diversity of contextual information in the output sentences can be constrained by generating sentences specifically for unseen relational labels using cueing conditions.

Table 9. Effect of Uyghur generated data size On UyRE-ZSP.

	Generated Samples	F1
m = 5	100	19.56
	200	23.24
	500	26.59
	1000	27.54
	1500	24.25
m = 10	100	23.15
	200	26.54
	500	27.42
	1000	25.25
	1500	24.12

shows the results of the evaluation, which is based on a UyRE-ZSP validation set with five and ten labels that are unseen. The F1 score goes up when the number of samples per label goes from 100 to 1500. However, increasing the generated capacity to a maximum of 1500 does not improve efficacy. This suggests that while synthetic data are advantageous for Zero-Shot-Prompt-RE for Uyghur, excessive quantities can result in overfitting due to noise.

5.4. Performance across Relations

To study how the performance varies across distinct relations labels, we evaluate single-triplet Zero-Shot-Prompt-RE for Uyghur on the UyRE-ZSP test set with m = 5, 10 unseen tags, respectively.

Table 10. Separate evaluation on relation labels.

	Relation Labels	F1
m = 5	Part of	13.65
	nationality	23.21
	spouse	25.65
	Educated At	39.87
	Award Received	15.63
	sister	28.25
	Place of death	42.68
m = 10	Part of	11.23
	nationality	25.63
	spouse	36.52
	Educated At	38.23
	Award Received	39.25
	sister	41.58
	Place of death	44.32

shows that the model can perform well for relations such as “sister—ئاچا سېڭىل ” and “ۋاپات بولغان جايى—place of death”.

However, it functions more inadequately for connections such as “Part of—بىر قىسمى ” and “مىللىتى—nationality.”

In order to investigate how the performance varies between different relationship labels, we evaluated the single triple ZeroRTE with five and ten invisible labels on the Wiki-ZSL test set. As can be seen in Table 10, the model performs well for m = 5 relations such as ‘‘Educated at— مەكتەپنى پۈتتۈرگەن’’ and ‘‘place of death’’, the model performs well for relations such as “Part of—بىر قىسمى ” and ‘‘Award received—مۇكاپاتقا ئېرىشكەن ” The model performs well for relations such as “Part of—بىر قىسمى ” and “nationality—دۆلەت تەۋەلىكى ” for m = 10, and worse for relations such as “Part of—ىر قىسمى ” and ‘‘Award received-مۇكاپاتقا ئېرىشكەن ” for m = 10. It performs worse for relations such as “Part of—بىر قىسمى ” and “nationality—دۆلەت تەۋەلىكى ”. These results suggest that Relation Prompt performs best for concrete relations to constrain the output context more effectively.

6. Conclusions

This paper describes the ‘Zero-Shot-Prompt-RE relational triple extraction for Uyghur’ job setting, which was made to get around the main problems with earlier task settings and encourage more research in low-resource relation extraction. To deal with Zero-Shot-Prompt-RE relational triple extraction tasks, we offer RelationPrompt and show that language models can use relation label prompts to create synthetic training data that can be used to create structured sentences. We present an effective and interpretable triplet search decoding approach to address the restriction of extracting many related triplets from a phrase. Using a multi-task learning structure and contextual representation learning quality based on the features of Uyghur entity words, cue learning embeds the input sentences well into the embedding space and significantly improves the performance. We also conducted extensive experiments to investigate different aspects of ZS-BERT, ranging from hyperparametric examples to case studies, ultimately showing that ZSP-RTE can consistently outperform existing relational extraction models in a zero-shot setting. In addition, learning effective relational embeddings using relational prototypes as auxiliary information may also help semi-supervised learning or few-shot learning.

In the future, we plan to apply this method to Knowledge Graph applications, including question answering, text classification, and event extraction.