Meta-Learning for Mandarin-Tibetan Cross-Lingual Speech Synthesis

Abstract

The paper proposes a meta-learning-based Mandarin-Tibetan cross-lingual text-to-speech (TTS) to realize both Mandarin and Tibetan speech synthesis under a unique framework. First, we build two kinds of Tacotron2-based Mandarin-Tibetan cross-lingual baseline TTS. One is a shared encoder Mandarin-Tibetan cross-lingual TTS, and another is a separate encoder Mandarin-Tibetan cross-lingual TTS. Both baseline TTS use the speaker classifier with a gradient reversal layer to disentangle speaker-specific information from the text encoder. At the same time, we design a prosody generator to extract prosodic information from sentences to explore syntactic and semantic information adequately. To further improve the synthesized speech quality of the Tacotron2-based Mandarin-Tibetan cross-lingual TTS, we propose a meta-learning-based Mandarin-Tibetan cross-lingual TTS. Based on the separate encoder Mandarin-Tibetan cross-lingual TTS, we use an additional dynamic network to predict the parameters of the language-dependent text encoder that could realize better cross-lingual knowledge sharing in the sequence-to-sequence TTS. Lastly, we synthesize Mandarin or Tibetan speech through the unique acoustic model. The baseline experimental results show that the separate encoder Mandarin-Tibetan cross-lingual TTS could handle the input of different languages better than the shared encoder Mandarin-Tibetan cross-lingual TTS. The experimental results further show that the proposed meta-learning-based Mandarin-Tibetan cross-lingual speech synthesis method could effectively improve the voice quality of synthesized speech in terms of naturalness and speaker similarity.

1. Introduction

Speech synthesis (or text-to-speech, TTS) has been widely used in applications such as smart homes, navigation, and audiobooks. However, there is still a challenge for a computer to generate natural and expressive human-like speech.

Generally speaking, there are several approaches for generating speech: (a) formant synthesis [1], (b) unit selection-based speech synthesis [2], (c) hidden Markov model (HMM)-based statistical parametric speech synthesis (SPSS) [3], and (d) deep learning-based speech synthesis [4,5,6,7]. Unit selection-based speech synthesis generates the most natural-sounding speech by concatenating small units of prerecorded waveforms with a unit selection algorithm, but its flexibility is limited. HMM-based SPSS has been the most popular TTS technology since the 1990s. The main advantage of HMM-based SPSS over unit selection-based speech synthesis is its flexibility in changing speech features, speaking styles, and emotions. This flexibility is mainly attributed to applying many techniques for controlling variation in speech, such as adaptation [8], interpolation [9], and eigen-voice. However, the synthesized speech of HMM-based SPSS is muffled compared with natural speech [10].

Since 2006, SPSS TTS has improved with the development of deep learning technologies. These deep learning-based statistical parametric acoustic models can be roughly divided into two categories: one adopt restricted Boltzmann machines (RBMs) [11] or deep belief networks (DBNs) [12] to improve the probability density function of HMM states, while the other is to use feed-forward neural networks (FNNs), mixture density networks (MDNs) [13], or long short term memory networks (LSTMs) [14] to model the relationship between linguistic features and acoustic parameters. Because deep learning technologies can represent the mapping between the context linguistic features and acoustic parameters well compared with statistical models commonly used in SPSS (e.g., decision trees, HMM), they help overcome the loss of detailed characteristics in synthesized speech.

Sequence-to-sequence (seq2seq) neural networks have been successfully applied to various tasks such as machine translation [15,16] and speech recognition [17,18]. In [19], Sotelo et al. proposed a seq2seq acoustic model for speech synthesis named Char2Wav, consisting of a reader and a neural vocoder. The reader is an encoder-decoder model with attention. The encoder is a bidirectional recurrent neural network that encodes the input text into hidden feature representations. The decoder with attention takes the hidden feature representations of the encoder to predict intermediate acoustic representations for the neural vocoder. The neural vocoder refers to a conditional extension of SampleRNN [20] that generates raw waveform samples from intermediate representations. Subsequently, the seq2seq TTS represented by Tacotron can be directly trained on the pairs of <text, speech> and automatically learn the alignment and mapping of the characters to spectrogram frames, which are then converted into waveforms by the neural vocoder [21]. Vaswani et al. [22] further proposed an attention mechanisms-based transformer for replacing complex recurrent or convolutional neural networks in encoder and decoder. Recent studies show that transformer-based TTS [6,7] is superior to Tacotron-based TTS in inference speed and robustness. Therefore, the synthesized speech quality of state-of-the-art seq2seq TTS is close to humans’ natural voices.

The rest of the paper is organized as follows. We first introduce related work in Section 2. Then, we illustrate the baseline in Section 3 and present our meta-learning-based Mandarin-Tibetan cross-lingual speech synthesis framework in Section 4. Subsequently, we explain the experimental data and setup in Section 5. The experimental results and analysis are presented in Section 6. Finally, a brief conclusion and future work are provided in Section 7.

2. Related Work

Speech in various languages can be synthesized for a monoglot speaker in cross-lingual speech synthesis. In the early stages, people often use monolingual or simple multilingual TTS to synthesize cross-lingual speech, leading to switching languages accompanied by voice switching. To address this problem, Traber et al. [23] built a polyglot TTS using a bilingual speech corpus recorded by a bilingual speaker. In [24], Mandarin and English context-dependent HMM states were shared, and the mapping was learned from a bilingual dataset recorded by a bilingual speaker. However, these cross-lingual TTS systems are mainly limited to constructing a multilingual corpus, characterized by a time-consuming, tedious and costly development path. Typically, only the data from monoglot speakers in different languages are available. Therefore, modern cross-lingual TTS systems are expected to generate a high-quality same-speaker voice for various languages using a monolingual speech corpus.

Modern cross-lingual speech synthesis mainly faces two challenges. One challenge is that speaker-specific information and language characteristics are directly correlated, i.e., each speaker speaks only one language, making it very hard to maintain consistent voices when language switching, especially within an utterance. In [25], the authors attempted to factorize speaker-specific and language-specific information for HMM-based SPSS. The experiment showed that speaker and language factorization could disentangle speaker-specific and language-specific information. Subsequently, researchers first used the language-specific and speaker-specific components of encoders for seq2seq-based cross-language speech synthesis [26]. Moreover, researchers used the gradient reversal layer (GRL) to further disentangle speaker-specific and language-specific information [27,28]. Another challenge is resolving the inconsistency in the phonetic systems for different languages. One approach is to assume that the input and output of the acoustic model are language-dependent while the middle layers of the acoustic model can be language-independent and shared in the cross-lingual speech synthesis. In [29,30], the authors proposed a multilingual deep learning-based SPSS that shared hidden layers across different languages while the input and output of acoustic models were language-dependent. For cross-lingual seq2seq TTS, researchers used language-dependent text encoders to alleviate the mutual interference of inputs from different languages [26,31], or used language ID to distinguish the input of different languages in the language-independent text encoder [31]. Another approach is first to convert graphemes of different languages to the uniform representations of the Unicode bytes or International Phonetic Alphabet (IPA) [32,33]. The models then consume the Unicode bytes or IPA as the input.

Meta-learning (or learning-to-learn) [34] is a new machine learning approach that has gradually emerged in recent years. As early as in [35], the authors adopted a dynamic network to predict a few parameters in the main network. Specifically, the main network and dynamic network use the same input image. The main network outputs the prediction results, while the dynamic network outputs the convolutional parameters of the main network. Similarly, Ha et al. [36,37] used dynamic networks to generate all weights for convolutional neural networks (CNNs) and LSTMs. The experimental results showed that the proposed method outperformed traditional neural network-based methods. In [38], researchers utilized a dynamic network called contextual parameter generator to generate convolutional parameters of the text encoder in the seq2seq multilingual TTS. The experimental results showed that the proposed meta-learning-based multilingual TTS could produce natural-sounding multilingual speech using more languages and less training data than previous approaches.

Tibetan has three different dialects, including the Lhasa dialect, the Kang dialect, and the Amdo dialect. Because the Lhasa dialect is the standard pronunciation of Tibetan, the paper focuses on realizing Lhasa dialect speech synthesis. Although the above three Tibetan dialects have different pronunciations, they use the same Tibetan characters. A Tibetan character with horizontal and vertical spelling structure differs from English with fully linear spelling. A typical Tibetan character consists of seven parts, as shown in Figure 1. In our previous work, we proposed the HMM-based [39] and deep learning-based [40] Mandarin-Tibetan cross-lingual SPSS. In [39], we realized the Tibetan grapheme-to-phoneme conversion module to convert Tibetan characters into initial and final sequences and used speaker adaptive training to realize Mandarin-Tibetan cross-lingual speech synthesis. In [40], we proposed the deep learning-based Mandarin-Tibetan cross-lingual speech synthesis to realize both Mandarin and Tibetan speech synthesis under a unique framework. Similarly, Xu et al. [41] proposed an HMM-based monolingual Tibetan SPSS. However, these Mandarin-Tibetan cross-lingual and Tibetan monolingual TTS still used SPSS methods. Therefore, the voice quality of the synthesized speech still needs to be further improved. Subsequently, Zhao et al. [42,43] proposed the seq2seq-based Tibetan TTS. In [42], authors explored the Tacotron2-based Lhasa dialect TTS combined with the WaveNet model. In [43], authors presented the Tacotron2-based Tibetan multi-dialect speech synthesis model to synthesize the Lhasa dialect and Amdo dialects. Although the above seq2seq-based monolingual Tibetan TTS models have greatly improved the synthesized voice quality compared to that of SPSS, these methods still have not achieved seq2seq-based Mandarin-Tibetan cross-lingual speech synthesis.

In the paper, we implemented a meta-learning-based Mandarin-Tibetan cross-lingual speech synthesis. Our main contributions are as follows.

(1)

We build two kinds of Tacotron2-based Mandarin-Tibetan cross-lingual seq2seq TTS: the shared encoder Mandarin-Tibetan cross-lingual TTS and the separate encoder Mandarin-Tibetan cross-lingual TTS, using the speaker classifier with gradient reversal layer to disentangle speaker-specific information from the text encoder. At the same time, we design a prosody generator to extract prosodic information from a sentence to explore the syntactic and semantic information adequately.

(2)

We propose a meta-learning-based Mandarin-Tibetan cross-lingual TTS to improve the synthesized speech quality of Mandarin and Tibetan. Based on the separate encoder Mandarin-Tibetan cross-lingual TTS, we use an additional dynamic network to predict the parameters of the language-dependent text encoder that could realize better cross-lingual knowledge sharing in the seq2seq TTS.

3. Baseline of Mandarin-Tibetan Cross-Lingual Speech Synthesis

We use two kinds of Tacotron2-based Mandarin-Tibetan cross-lingual TTS as the baseline system. The two baseline systems are called shared encoder Mandarin-Tibetan cross-lingual TTS and separate encoder Mandarin-Tibetan cross-lingual TTS, respectively.

The shared encoder Mandarin-Tibetan cross-lingual TTS is shown in Figure 2. To make the original Tacotron2-based TTS better adapt to cross-lingual speech synthesis for the shared encoder Mandarin-Tibetan cross-lingual TTS, we change the original Tacotron2-based TTS as follows.

(1)

The residual connection and batch normalization layer are added to the text encoder of the original Tacotron2, named the residual text encoder. The output of the residual text encoder is shown in Equation (1).

$${Encoder}_{\mathrm{res}}\left({\left\{{\mathit{x}}_j\right\}}_{j=1}^N\right)=Encoder\left({\left\{{\mathit{x}}_j\right\}}_{j=1}^N\right)+Phone\_embedding\left({\left\{{\mathit{x}}_j\right\}}_{j=1}^N\right)$$

(1)

where ${\left\{{\mathit{x}}_j\right\}}_{j=1}^N$ represents the input phoneme sequences, $Encoder\left({\left\{{\mathit{x}}_j\right\}}_{j=1}^N\right)$ represents the encoder output of the original Tacotron2-based TTS, and $Phone\_embedding\left({\left\{{\mathit{x}}_j\right\}}_{j=1}^N\right)$ represents the embedding of the input phoneme sequences.

(2)

In the Mandarin-Tibetan cross-lingual speech synthesis task, language embedding and speaker embedding are added to the Tacotron2-based TTS to distinguish the pronunciation characteristics of different speakers in different languages. In particular, the speaker embedding and the output of attention LSTM are concatenated as the input of the decoder LSTM to improve the similarity between the synthesized speech and the original speech.

(3)

We use an adversarial speaker classifier to disentangle the speaker-specific information from the text encoder. The adversarial speaker classifier consists of a gradient reversal layer, two fully connected layers, and a softmax layer. Due to the introduction of the adversarial speaker classifier, the new training loss function is shown in Equation (2).

$${LOSS}_{\mathrm{Total}}={LOSS}_{DEC}+{LOSS}_{\mathrm{Speaker\_classifier}}$$

(2)

where ${LOSS}_{DEC}$ represents the loss between the synthesized speech and the target speech, ${LOSS}_{\mathrm{Speaker\_classifier}}$ represents the loss of speaker classification predicted by the adversarial speaker classifier.

(4)

Chinese and Tibetan characters are different from English and other Roman characters-based languages. Because there is no distinct separator between adjacent words, an occasional word segmentation error leads to semantic confusion and prosodic errors in speech synthesis. In [44], the experiments show that enhanced input text by integrating prosodic information (e.g., prosodic word boundaries, prosodic phrase boundaries, HTS-based context-dependent information) significantly improves the naturalness of the synthesized speech in seq2seq TTS. In [45,46,47], the text embedding extracted by the pre-training model Bidirectional Encoder Representations from Transformers (BERT) is an additional input to add to the seq2seq TTS based on Tacotron2. Because these text embedding features contain linguistic and semantic-related information, they help the speech synthesis system generate more natural speech. To adequately explore the syntactic and semantic information, we design a prosody generator to extract the prosodic information from the sentence [48]. The prosody generator includes a text analyzer, a feature vector extraction module, a question set, and a hidden feature extraction module.

The framework of separate encoder Mandarin-Tibetan cross-lingual TTS is similar to that of the shared encoder Mandarin-Tibetan cross-lingual TTS, as shown in Figure 3. The difference is that Mandarin and Tibetan texts use different residual text encoders. The language ID is used as a mask to select Mandarin or Tibetan text from the output of residual text encoders so that the Mandarin or Tibetan text is encoded into different hidden feature representations.

4. The Framework of Meta-Learning-Based Mandarin-Tibetan Cross-Lingual Speech Synthesis

The meta-learning-based Mandarin-Tibetan cross-lingual speech synthesis framework is shown in Figure 4. We use a dynamic network to predict the parameters of the language-dependent text encoder based on the separate encoder Mandarin-Tibetan TTS. Because generating all weights of RNN [37] is difficult, we replace the bidirectional LSTM of the original text encoder with a fully convolutional encoder from DCTTS [49]. Furthermore, the parameters of the convolutional encoder are generated by corresponding fully connected (FC) layers called the dynamic network to achieve better cross-language knowledge sharing.

5. Experiments

5.1. Experimental Data

We build the Tibetan corpus called Tibetan_Lasa_1 for speech synthesis. Tibetan_Lasa_1 is a standard Lhasa dialect corpus for one female speaker. The utterances are recorded in a professional recording studio. We use high-fidelity microphones and professional audio workstations to record speech corpus and conduct legality checks on all recordings to avoid errors in the recording process. There are 10,000 recordings in total from a 25-year-old female Lhasa dialect broadcaster. All sentences are mainly declarative utterances from news, Tibetan websites, and Tibetan books. All recordings were saved in the Microsoft WAV format (mono-channel, 16-bit depth, sampled at 16 kHz).

Besides, we use the open-source Mandarin speech synthesis dataset of DataBaker. This open-source dataset consists of 10,000 recordings from a 20–30 years old female. All recordings were saved in the Microsoft WAV format (mono-channel, 16-bit depth, sampled at 48 kHz). In the experiment, all recordings were down-sampled from 48 kHz to 16 kHz.

5.2. Experimental Setup

In the experiment, we selected training corpus from the DataBaker and Tibetan_Lasa_1. First, two sets of 150 utterances were randomly selected from the DataBaker to serve as the Mandarin test set and Mandarin development set, respectively. Similarly, two sets of 150 utterances were randomly selected from the Tibetan_Lasa_1 to serve as the Tibetan test set and Tibetan development set, respectively. Then the remaining Mandarin utterances were marked as {M}, while the remaining Tibetan utterances were marked as {T}.

We build the following four TTS models to verify the effectiveness of the proposed meta-learning-based Mandarin-Tibetan cross-lingual TTS.

• seq2seq-M or seq2seq-T: Based on the original Tacotron2, we trained Mandarin and Tibetan monolingual TTS, called seq2seq-T and seq2seq-M, respectively.
• shared-MT: We call the shared encoder Mandarin-Tibetan (MT) cross-lingual TTS as shared-MT, which is trained on training subsets {M} and {T}, and its architecture is shown in Figure 2.
• separate-MT: We call the separate encoder Mandarin-Tibetan cross-lingual TTS as separate-MT, which is trained on training subsets {M} and {T}, and its architecture is shown in Figure 3.
• meta-MT: We call the meta-learning-based Mandarin-Tibetan cross-lingual TTS as meta-MT, which is trained on training subsets {M} and {T}, and its architecture is shown in Figure 4.

The Adam optimizer was used in the experiment to train the above TTS models. The learning rate stepped from ${10}^{-3}$ and halved every 10k step. All models were trained until the loss of the development set started increasing. Two NVIDIA Tesla P4 GPUs were used in the experiment, and batch_size was set to 64. The dynamic network consists of fully connected layers. The input of the dynamic network is the 20-dimensional language embedding, and the size of each convolution kernel determines its output dimension. The bidirectional LSTM of the prosody generator consists of a two layers LSTM with 128 units. The other parameters of all models are similar to the original Tacotron2.

6. Evaluations and Analysis

To evaluate the effectiveness of the proposed meta-learning-based Mandarin-Tibetan cross-lingual TTS, we evaluate the synthesized speech quality of the above models under different training sets. The evaluations mainly include multilingual speech synthesis evaluation and cross-lingual speech synthesis evaluation.

6.1. Multilingual Speech Synthesis Evaluation

In multilingual speech synthesis evaluation, we mainly evaluate the speech quality of the speakers’ native language synthesized by the above four TTS models. The multilingual speech synthesis evaluation includes objective evaluations and subjective evaluations.

6.1.1. Objective Evaluation

We evaluated the Mel-cepstral distortion (MCD) in the objective evaluations. The MCD represents the Mel Frequency Cepstral Coefficients (MFCCs) error calculated from predicted speech samples and natural speech samples. Assuming that the MFCCs of a frame are expressed as y and the predicted MFCCs are expressed as $\widehat{y}$, the MCD of the frame can be calculated as follow:

$$MCD(y;\widehat{y})={\parallel y-\widehat{y}\parallel }_2$$

(3)

where ${\parallel ·\parallel }_2$ represents $L2$ norm.

The MCD of different seq2seq TTS models on the test set is shown in

Table 1. The MCD values of different seq2seq TTS models on the test sets. The seq2seq-M and seq2seq-T are Mandarin and Tibetan monolingual TTS models, respectively. The shared-MT, separate-MT, and meta-MT are Mandarin-Tibetan cross-language TTS models.

Languages	seq2seq-M	seq2seq-T	shared-MT	separate-MT	meta-MT
Mandarin	4.633	-	4.752	4.659	4.652
Tibetan	-	4.185	4.055	4.031	3.985

. We can find that the MCD values of synthesized Mandarin speech of the Mandarin-Tibetan cross-lingual models, including the shared-MT, the separate-MT and the meta-MT, increase by 0.119, 0.026 and 0.019 compared with seq2seq-M, respectively. For Tibetan speech synthesis, the findings can be analyzed that the baseline of Mandarin-Tibetan cross-lingual models, including the shared-MT and the separate-MT, help to improve the voice quality of the synthesized Tibetan speech. The separate-MT has higher modeling accuracy compared with the shared-MT. This may be due to the fact that the separate-MT alleviates the mutual interference of inputs from different languages. Furthermore, the meta-MT seems slightly better than separate-MT for Mandarin and Tibetan speech synthesis. It fulfills our expectations as meta-MT should be more flexible.

6.1.2. Subjective Evaluation

For subjective evaluations, 20 speech samples were randomly selected from the test set. We conducted a mean opinion score (MOS) test and AB preference test to evaluate the voice quality of synthesized speech. In addition, we invited 30 native Mandarin and 30 Tibetan listeners as subjects. Mandarin subjects were invited to evaluate the synthesized Mandarin speech samples, while Tibetan subjects were invited to assess the synthesized Tibetan speech samples.

In the MOS test, subjects were asked to rate the naturalness of the synthesized speech using a 5-point scale score. The average MOS scores of synthesized Mandarin speech samples are shown in Figure 5. We can find that the seq2seq-M has a higher MOS score, and the MOS scores of meta-MT and separate-MT are slightly higher than that of shared-MT from Figure 5. The average MOS scores of synthesized Tibetan speech are shown in Figure 6. The meta-MT has slightly higher average MOS scores for Mandarin and Tibetan speech synthesis compared with the separate-MT and shared-MT. These results are consistent with the objective results.

In the AB preference test, the sentences were the same in both items within a pair. Each pair of synthesized speech samples was played at random. The subjects were asked to listen and judge the quality of which sample was better (or “neutral” means that the subjects had no preference). The preference results of the synthesized Tibetan and Mandarin speech samples are shown in

Table 2. Subjective AB preference score(%) of synthesized Mandarin speech samples with p < $0.01$.

	seq2seq-M	shared-MT	separate-MT	meta-MT	Nertral
1	56.5	31.2	-	-	12.2
2	-	37.8	49.8	-	12.5
3	-	-	22.3	66.2	11.5

and

Table 3. Subjective AB preference score(%) of synthesized Tibetan speech samples with p < $0.01$.

	seq2seq-T	shared-MT	separate-MT	meta-MT	Nertral
1	19.3	70.8	-	-	9.8
2	-	27.3	56.7	-	15.3
3	-	-	24.7	64.8	10.5

. We can find that the AB preference results are consistent with the objective and MOS evaluation results.

6.2. Cross-Lingual Speech Synthesis Evaluation

We build a small-scale dataset containing 50 bilingual sentences for cross-lingual speech synthesis evaluation. Because seq2seq-M and seq2seq-T are monolingual TTS, the speech samples are only generated by the shared-MT, separate-MT, and meta-MT models in Mandarin speaker’s and Tibetan speaker’s voices, respectively. We also randomly selected 20 synthesized speech samples to conduct MOS and AB preference tests for speech naturalness and cross-language speaker similarity preservation of the proposed TTS models. In addition, 20 bilingual speakers in Mandarin and Tibetan were invited to participate in each test.

In the MOS test, subjects were asked to rate the naturalness of the synthesized speech samples using a 5-point scale score. The average MOS scores of synthesized bilingual speech samples in Mandarin speaker’s or Tibetan speaker’s voices are shown in Figure 7. The MOS results of the baseline show that the separate-MT outperforms the shared-MT in cross-lingual speech synthesis, which indicates that the separate-MT benefits more from less language interference when encoding the inputs to hidden representations. Furthermore, the MOS results also show that our proposed meta-MT model can synthesize more natural speech than the baseline in bilingual sentences. Besides, the MOS results show that we can synthesize more natural bilingual speech in a Tibetan speaker’s voice than in a Mandarin speaker’s voice.

In the AB preference test, the listeners were required to provide a speaker similarity choice among three options: (1) similar to the Mandarin speaker’s voice; (2) no preference; (3) similar to the Tibetan speaker’s voice. The AB preference test results are shown in Figure 8. Similar to the previous results, the separate-MT slightly outperforms the shared-MT in cross-lingual speech synthesis. Furthermore, the meta-MT outperforms baseline TTS by successfully maintaining both the Tibetan speaker’s voice and the Mandarin speaker’s voice. Specifically, no models can maintain the Mandarin speaker’s voice as well as the Tibetan speaker’s voice, which may be partially due to the low naturalness of synthesized Mandarin speech.

7. Conclusions and Future Work

The paper proposes a meta-learning-based Mandarin-Tibetan cross-lingual TTS to realize both Mandarin and Tibetan speech synthesis under a unique framework. We first build two kinds of Tacotron2-based Mandarin-Tibetan cross-lingual baseline TTS, including the shared encoder Mandarin-Tibetan cross-lingual TTS and the separate encoder Mandarin-Tibetan cross-lingual TTS. Then, to further improve the synthesized speech quality of Tacotron2-based Mandarin-Tibetan cross-lingual TTS, we propose the meta-learning-based Mandarin-Tibetan cross-lingual TTS using an additional dynamic network to predict the parameters of the language-dependent text encoder. Finally, we synthesize Mandarin speech or Tibetan speech through the unique acoustic model. The baseline experimental results show that the separate encoder Mandarin-Tibetan cross-lingual TTS can handle the input of different languages better than the shared encoder cross-lingual Mandarin-Tibetan TTS. Furthermore, the experimental results also show that the proposed meta-learning-based Mandarin-Tibetan cross-lingual speech synthesis method could effectively improve the voice quality of synthesized speech in terms of naturalness and speaker similarity.

Future work will investigate how to use the least corpus for synthesizing Mandarin-Tibetan cross-lingual speech with satisfactory voice quality. Future work will also improve prosody with other linguistic features (e.g., BERT-derived features).