Bio-Constrained Codes with Neural Network for Density-Based DNA Data Storage

Abstract

DNA has evolved as a cutting-edge medium for digital information storage due to its extremely high density and durable preservation to accommodate the data explosion. However, the strings of DNA are prone to errors during the hybridization process. In addition, DNA synthesis and sequences come with a cost that depends on the number of nucleotides present. An efficient model to store a large amount of data in a small number of nucleotides is essential, and it must control the hybridization errors among the base pairs. In this paper, a novel computational model is presented to design large DNA libraries of oligonucleotides. It is established by integrating a neural network (NN) with combinatorial biological constraints, including constant GC-content and satisfying Hamming distance and reverse-complement constraints. We develop a simple and efficient implementation of NNs to produce the optimal DNA codes, which opens the door to applying neural networks for DNA-based data storage. Further, the combinatorial bio-constraints are introduced to improve the lower bounds and to avoid the occurrence of errors in the DNA codes. Our goal is to compute large DNA codes in shorter sequences, which should avoid non-specific hybridization errors by satisfying the bio-constrained coding. The proposed model yields a significant improvement in the DNA library by explicitly constructing larger codes than the prior published codes.

1. Introduction

The exponential increase in big data demands high density and capacity storage. Inspired by nature, DNA (deoxyribonucleic acid) has various applicable features for digital data storage. DNA comprises four bases: adenine (A), guanine (G), cytosine (C), and thymine (T), collectively called nucleotides. DNA data storage has three key steps [1,2,3,4,5,6,7]: (i) Digital data are converted into binary data, which are encoded into DNA strands with quaternary alphabet (A, C, T, and G) strings/sequences that are called DNA codes or codewords. (ii) These strands are synthesized (data writing) into oligonucleotides by a DNA synthesizer, and the data are stored. (iii) DNA strands are decoded by DNA sequencing (data reading) to retrieve the data. These key steps come under the big umbrella of DNA computing, in which DNA data storage is partially based on information technology (IT) and biotechnology (BT). In IT, data encoding and decoding techniques are employed, comprising computational and mathematical models. In BT, DNA synthesis, storage, and sequencing are carried out with base pairs (A, C, G, and T) in a DNA molecule. It is essential for any DNA computing model to select the DNA molecules and code them efficiently to attain maximum storage density [8].

In the DNA data storage system, various coding techniques, i.e., biological constraint/bio-constraint coding [9] and error correction coding [5], are presented to overcome the different ambiguities. In the literature, GC-content [3], no-run-length [10], reverse-complement ($RC$) constraint [11], and Hamming distance ($d_H$) have been found as major biological constraints coding for DNA synthesis and sequencing errors. For any DNA sequence $\mathsf{\alpha }={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_n$ $\in$ $\sum$ⁿ, the GC-content is the ratio of the sum of bases content $\left(G\mathrm{and}C\right)$ to the total number of bases $\left(\left|G+C\right|/\left|n\right|\right)\times 100\%$ for $n$ sequence length. It is close to 50% in each DNA codeword [12]. No-run-length is the avoidance of repetition of the same quaternary (q-ary) alphabet. Similarly, for the $RC$ constraint, the reverse sequence ${\mathsf{\alpha }}^r={\mathsf{\alpha }}_n{\mathsf{\alpha }}_{n-1}\dots {\mathsf{\alpha }}_1$, complement sequence ${\mathsf{\alpha }}^r={\mathsf{\alpha }}_1^c{\mathsf{\alpha }}_2^c\dots {\mathsf{\alpha }}_n^c$, and reverse-complement sequence are ${\mathsf{\alpha }}^{rc}={\mathsf{\alpha }}_n^c{\mathsf{\alpha }}_{n-1}^c\dots {\mathsf{\alpha }}_1^c$, for which $A^C=T,T^C=A,G^C=C,\mathrm{and}{C}^C=G.$ For instance, in a given DNA sequence TTCAGGA, the reverse is ATGACGT, the complement is AAGTCCT, and the reverse-complement is TACTGCA. $A_4^{GC}\left(n,d,\omega \right)$ and $A_4^{GC,RC}\left(n,d,\omega \right)$ denote the maximum number of codewords in a DNA code satisfying two constraints (GC-content and $d_H$) and satisfying three constraints (GC-content, $RC$ constraint and $d_H$), respectively. DNA libraries satisfying these bio-constraints will have a certain application to DNA computing, particularly DNA data storage [3,10].

A DNA sequence is read through a particular hybridization process in which two complementary single-stranded DNA molecules are combined to form a single-stranded molecule via base pairing. If any sequence in DNA codewords is identical to its reverse-complement, non-specific hybridization will occur, which causes the leading errors in retrieving information [2]. To avoid these, the authors utilized stochastic-based optimization algorithms and neural networks [13,14,15,16,17]. The algorithms in [13,17] were introduced to improve the lower bounds of DNA codes by different iterations and parameters.

Recently, a deep learning model (DLM) was introduced with three different next-generation sequences for DNA information storage [14]. Their DLM utilized four gated recurrent unit (GRU) neural networks grouped into two sets, which took sequences from 5′ to 3′ or from 3′ to 5′. GRUs with two gates have been adopted with a feed-forward neural network (FFNN) to predict the sequence during the hybridization process. The excessive number of hidden nodes and the model’s reliance on an FFNN exacerbate the DLM training. Despite this issue, it initiated an acceptive idea to bring neural networks for DNA data storage. This milestone has motivated us to implement a neural network on DNA codes to provide a novel coding scheme for high-density storage. In [15], a DeepMod system was proposed that integrates the recurrent neural network and long short-term memory (LSTM) models to perceive the DNA codes from various Oxford Nanopore sequences. Likewise, a convolutional neural network was embedded for the generation of DNA bases to achieve high-density storage. It employed a DNA-mapping method consisting of GC-content and homopolymer length constraints to design the DNA codebook [16]. Their work inspired us to seek a model that generalizes the DNA codewords as much as possible with the artificial neural network. However, the neural network (NN) with a fixed number of nodes, i.e., LSTM including forwarding pass, is not well-studied for DNA sequence input. Apart from non-specific hybridization, it is essential for DNA code development to detect the error source to avoid insertion, deletion, and substitution errors with DNA coding bio-constraints [4,5,12]. Several existing studies have been conducted to address these problems with DNA codewords. However, the literature that satisfies the GC-content, Hamming distance, and RC constraints is cited here.

In 2004, DNA codes with GC-content were extensively presented in [18]. It reported the upper and lower bounds on DNA code size with GC-content and Hamming distance to construct the DNA codewords. In a polymerase chain reaction (PCR), a DNA code with huge GC-content (say, >60%) caused the insertion and deletion errors. Thus, it is necessary to consider the GC-content for the stability of DNA sequences by avoiding computation errors. In 2017, a study pioneered by Erlich [3] delivered a seminal work on DNA data storage by proposing a fountain code with GC-content (45–55%) and a minimum Hamming distance (d). They achieved 1.57 net information density; however, they still faced errors in GC-content which propagated severe errors, including mismatches, deletions, and insertions, during the decoding process. In addition, no theoretical lower or upper bounds were presented for those constraints.

In 2018, ref. [10] proposed a novel altruistic algorithm with lower bounds to generate constraint-based stable DNA codes. It also used constant GC-content and minimum Hamming distance and reported an improved number of DNA codewords. However, the storage efficiency was not sufficiently considered for density-based DNA data storage. In 2020, the author [17] proposed a damping multi-verse optimizer algorithm to design the DNA coding sets by constructing the GC-content with no-run-length constraints. Their results revealed 4–16% more improved DNA coding than that of [10], which suggests that the increase in constraints can improve the codes for high-density DNA data storage. In 2021, our previous paper [12] extended the work [10] by proposing a novel algorithm to construct the DNA coding sets with improved lower bounds. The proposed algorithm was applied with GC-content and no-run-length constraints and achieved 30% better lower bounds. However, besides the insertion and deletion errors in DNA codes, another issue of secondary structures (SS) occurs during the reading process [19]. The SS is a base pairing contact of a single-strand sequence that folds back on itself, as presented in [11] (Figure 1). Any DNA sequence with an SS shape will consume the extra resources and energy to be unfolded, which slows the chemical reaction immensely. Therefore, DNA needs to be free from the SS shape before reading DNA sequences in the wet lab. There are few studies on eliminating this severe issue. The author in [11,20] introduced the RC constraint to overcome the SS issue. They subjected the GC-content and RC-content together to improve the DNA coding sets. Their studies furnish the basic idea of combinatorial constraints to generate DNA codes with minimum errors. Although the literature mentioned above [10,11,12,17,20] received high storage DNA code sets and coding rates, these studies do not provide a sufficient method to design higher DNA codes in the shorter sequences that must satisfy the biological constraints, which is enormously important for a stable density-based DNA storage system.

This paper introduces a more efficient coding technique with a novel computational model that is based on biologically inspired computing because it uses a neural network (NN) with biological constraints to obtain a high-density-based DNA data storage. In the proposed model, LSTM as an NN with a forward pass is utilized to open a new door in the NN for DNA code construction. Firstly, the binary data are converted into premiere DNA bases by using the [3] scheme. Then, the yielded premiere DNA strings are passed through the NN model with the forward passing mechanism. A particular criterion trains the activation functions to randomly generate DNA codes. If those DNA codes pass that criterion, we term them optimal DNA codes. Then, the combinatorial constraints are utilized to concatenate these optimal DNA codes. The combinatorial constraints, including GC-content and RC constraint with Hamming distance, are computed to generate a DNA library that is used to store the digital information, for which different propositions and theorems are constructed in the Magma program and proved in this paper. GC-content and Hamming distance are computed, and results are obtained with improved lower bounds. Meanwhile, the RC constraint with Hamming distance is constructed to avoid secondary structure, and it is concatenated with GC to generate the DNA library with the best-known codes. These codes are generated by Magma with different inequalities. These inequalities are based on the previous studies that are used for the comparison of our results. Furthermore, the results are analyzed by the coding rate formula, which helps us to evaluate the data storage density in DNA media.

In general, there are two goals to be delivered for high-density-based DNA data storage with the following features:

• To improve the net information density by storing a large amount of digital data in shorter DNA sequences.
• To construct the DNA codes that satisfy the combinatorial bio-constraints to overcome the reading errors.

In this scenario, these goals are accomplished by the following significant contributions:

• A novel computational model based on the LSTM neural network with a forward pass is proposed to generate the optimal DNA codes from the premiere DNA bases. To the best of our knowledge, such a model has not been studied in the prior studies.
• The combinatorial bio-constraints, including GC-content, RC constraint, and Hamming distance, are constructed for optimal DNA codes to avoid non-specific hybridization by overcoming sequencing errors and secondary structures.
• The results receive many DNA coding sets satisfying the bio-constraints and significantly improving the DNA coding rates compared to the existing studies.

The structure of the rest of the paper is as follows: Section 2 delivers the prior work about deep neural network and combinatorial constraints for DNA data storage. Section 3 presents the preliminaries and notations, Section 4 introduces the proposed model, Section 5 elaborates on the results, and Section 6 concludes this work.

2. Literature Review

This section is divided into two subsections to emphasize our paper’s contributions based on neural networks for DNA codes and DNA coding with combinatorial constraints.

2.1. Deep Neural Networks for DNA Codes

DNA computing has successfully impacted human life due to well-known computation tools based on machine learning and the deep learning community. With the rapid generation of digital data, efficient and effective deep learning architectures (DLAs) have been constructed to compute big data [21]. DLA has been approved in a variety of domains with significant accuracies and predictions. In this article, we consider applying the deep neural network based on DLA. Recurrent neural networks (RNNs) provide the connection between the nodes to form a directed graph along a temporal sequence. The graph exhibits a short-term memory that allows RNNs to remember information from the previous state to the next state [22]. Long short-term memory (LSTM) is a variant of RNN that efficiently learns the long-term dependencies. It has three gates: input, output, and the forget gate. An LSTM unit has a node or cell that accounts for the values over particular time intervals while the rest of the gates regulate the information [23,24].

Various deep neural networks have been applied with different methods and models in various natural computing studies. In 2015, a novel method was proposed for the transformation of DNA sequences into numerical sequences. This method was based on a pulse-coupled neural network and Huffman coding, which used triplet codes to encode the different lengths of DNA sequences [25]. Another study also attempted to encode the data with that method, but it found that encoded sequences are compressed at a close distance, making the results less informative [26]. Although numerous studies have been conducted with deep neural networks for natural computing, deep neural networks are still new models for the DNA data storage system. For instance, in 2021, a GRU (gated recurrent unit)-based deep learning model was presented for DNA information storage with next-generation sequence prediction [14]. Similarly, the author proposed a DeepMod system that integrates the RNN and LSTM models to perceive the DNA codes from various Oxford Nanopore sequences. While RNNs are employed to capture the Nanopore sequencing, LSTM overcomes the vanishing gradient issues in the training of RNNs. The proposed system collectively achieves better DNA codes from the given sequences compared to others [15]. In addition, in [16], the problem of DNA synthesis cost was addressed by delivering a high-density-based DNA data storage system. To achieve a high storage density, convolutional neural networks were embedded to generate the DNA bases. It employed a DNA-mapping method that consisted of GC-content and homopolymer length constraints to design the DNA codebook. It was reported that the proposed scheme efficiently stored and retrieved the information from the DNA storage system with the integration of a deep neural network with the DNA mapping method. These studies provide the motivation for the integrational system of deep neural networks with DNA coding and combinatorial bio-constraints.

2.2. DNA Coding with Combinatorial Bio-Constraints

In DNA synthesizing and sequencing, various errors occur, which are combated with different coding techniques. For instance, error correction coding and bio-constraint coding are mainly used in DNA-based information storage systems [27]. Bio-constraint coding has been practically applied in mass data storage, i.e., magnetic and optical data recording [28]. There are different types of DNA constraint coding reported in the existing literature. Researchers [2,29,30] have formulized single constraints and/or combined the constraints to attain the targeted results by preventing DNA sequence (DNA code) errors.

G. M. Church delivered a pivotal work on DNA data storage by converting 8-bit ASCII (0 to A or C and 1 to G or T) into DNA bases. It pondered the GC-content constraint and the homopolymer run-length constraint by disallowing a run-length greater than 3. It was a groundbreaking step toward storing the digital information into DNA, but it was plagued by huge errors and a lack of competency [7]. N. Goldman presented a different method by compressing the raw data into DNA sequencing with differential coding and a Huffman coding scheme. It employed the run-length constraint of at most 1 and achieved an effective coding rate. It suggested considering the GC-content for better constraint satisfaction [6]. Similarly, R. N. Grass combined the different constraints to deliver an error-correcting scheme. It used Reed–Solomon codes for error control [5]. In comparison, M. Blawat offered a seminal study for DNA data storage by proposing a forward error-correction mechanism. It provided the codewords (codes) that avoided deletion and substitution errors by utilizing the GC-content constraint [4]. Y. Erlich and D. Zielinski proposed another benchmark study by designing the fountain code that considered the GC-content and run-length for DNA synthesis and sequencing. Their study significantly achieved better coding rates compared to existing work; however, in the decoding process, it found error propagation in the information retrieval stage [3]. W. Song and Y. Wang also conducted research on DNA data storage by presenting a mathematical method for DNA code generation by preserving GC-content and Hamming distance constraints [9,27]. D. Limbachiya constructed an altruistic algorithm to create DNA codewords of a specific length. The algorithm also formalized the Hamming distance for each code to satisfy the constraints [10]. In our previous work, a novel algorithm was developed to generate the DNA codes. The obtained DNA codes’ errors have been corrected to a limited extent with GC-content and no-run-length constraints [12].

In conjunction with GC-content, no-run-length, and Hamming distance, a few inspiring and influential constraint studies deal with the reverse-complement constraint. In 2005, Oliver D. King reported the theoretical lower and upper bounds for the maximum size of DNA codes. The report created the codes with the minimum Hamming distance and reverse-complement of any code with the least distance. It stated that obtained DNA codes were larger than ever [18]. In 2010, A. Niema accommodated Oliver D. King’s bounds to design DNA codewords with GC-content, the Hamming distance, and the reverse-complement RC constraint by avoiding non-specific hybridizations. They employed RC constraint to handle the searching of codes related to bases with 0 and 1 points. It obtained many new codes for DNA data storage [20]. In 2021, Benerjee K.G. exhibited the families of those DNA codes which avoid secondary structures. It combined the RC constraint with homopolymers’ run-lengths to construct the dissimilar DNA codes [11].

The prior work on bio-constrained coding established an idea of combinatorial constraints with different mathematical methods and formulations. These studies achieved many DNA codes in their particular methods. However, we have found that the desired DNA codes are still capable of improving the lower and upper bounds for the generation of high-density-based DNA codes. As we discuss a few studies on deep neural networks that impact constructing DNA codes, in this paper, we propose a novel model that integrates a deep neural network with combinatorial constraints to design DNA codewords.

3. Preliminaries and Notations

According to fundamental bio-constraints (Section 1), we design the oligos as sequence α for $\sum =\left\{A,C,G,T\right\}$. If $\mathsf{\alpha }\in {\sum }^n$, the alphabet at the position $i$ in the sequence α is presented as ${\mathsf{\alpha }}_i$. Thus, a sequence ${\mathsf{\alpha }}_i={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_n$ $\in$ ${\sum }^n$ will be generated. In the same way, let another sequence ${\mathsf{\beta }}_i={\mathsf{\beta }}_1{\mathsf{\beta }}_2\dots {\mathsf{\beta }}_n$, $\in$ ${\sum }^n$ be possible if the Hamming distance [31] between both sequences $\left(\mathsf{\alpha },\mathsf{\beta }\in {\sum }^n\right)$, denoted $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$, satisfies the following:

$$d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)=\left|\left\{1\le i\le n:\mathsf{\alpha }\ne \mathsf{\beta }\right\}\right|.$$

(1)

Apart from $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right),$ sequences $\mathsf{\alpha },\mathsf{\beta },\in {\sum }^n$ must satisfy the GC-content and reverse-complement constraints (Section 3) to produce the DNA library $\mathcal{L}$. Hereafter, oligos are denoted by Greek letters, and other notations are a generic set ξ of sequences $\mathsf{\alpha },\mathsf{\beta },\in {\sum }^n$. Here, we need to provide the definition of the DNA library [31].

Definition 1.

A set of DNA bases {A,C,G,T} with n-mer oligos ξ $\subseteq {\sum }^n$ that satisfies the constant GC constraint, reverse-complement constraint, and Hamming distance constraint is called a DNA code/codeword $\left(n, d, \omega \right)$ which collectively forms a DNA library $\mathcal{L}=\sum \left(n, d, \omega \right)$.

If a DNA library is indicated by the number of K-constrained sequences of $q-ary$ strings initiated with a non-zero symbol $i$, Shannon’s relationship [32] can be written as a recurrent relationship:

$${\mathcal{L}}_k\left(n\right)=\left(q-1\right){\sum }_{i-1}^{K+1}{\mathcal{L}}_k\left(n-1\right),\hspace{1em}\hspace{1em}n>K.$$

(2)

If the number of codes n increases, the ${\mathcal{L}}_k\left(n\right)$ increase exponentially by the following:

$${\mathcal{L}}_k\left(n\right)~c{\Gamma }_k^n,\hspace{1em}\hspace{1em}\hspace{1em}n>>1,$$

(3)

where $c~1$ is a constant and ${\Gamma }_k$ is an exponential growth factor which is a real root of

$${\Gamma }_K^{K+2}-q{\Gamma }_K^{K+1}+q-1=0.$$

(4)

In order to store the digital data in the nucleotide, this expression leads to defining the DNA data density [32].

Definition 2.

The maximum number of digital data bits (b) stored per nucleotide (nt) is termed as data density, denoted by$D_k$and defined:

$$D_k=\underset{n\to \infty }{\lim }\left(\frac{1}{n}{\log }_2{\mathcal{L}}_k\left(n\right)\right)={\log }_2{\Gamma }_k \left(\frac{b}{nt}\right).$$

(5)

In high-density-based DNA storage, there is a probability of secondary structures. In experiments, the Nussinov–Jacobson (NJ) algorithm is employed to predict the secondary structures approximately [33]. During the chemical reaction, a DNA sequence ${\mathsf{\alpha }}_i={\mathsf{\alpha }}_1,{\mathsf{\alpha }}_2,\dots ,{\mathsf{\alpha }}_n$ releases the energy to attain stability after forming secondary structures. Thus, this form can be calculated by a DNA property called free energy $\left(E\right)$. This energy relies on the sequence pair $\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right)$, where $1\le i<j\le n$ and the pair releases its energy, which is termed as interaction energy $\mathsf{\phi }\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right)$. Note that $\mathsf{\phi }\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right)$ between sequence ${\mathsf{\alpha }}_i$ and ${\mathsf{\beta }}_j$ in any pair $\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right)$ will be independent of other sequence pairs. In the NJ algorithm, the interaction energies depend on the selected sequence pairs $\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right)$ as non-positive values, while, for the independent interaction energies, the assumption for the NJ algorithm with minimum free energy $E_{i,j}$ for a DNA sequence ${\mathsf{\alpha }}_i={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_n$ is $E_{i,j}=\min \left\{E_{i+1,j-1}+\mathsf{\phi }\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right),E_{i,k-1}+E_{k,j}:K=i+1,\dots ,j\right\}$ for particular conditions and $E_{l,l}=E_{l-1,l}=0$ for $l=1,2,\dots ,n$ [34].

4. Proposed Model

In this paper, the concept of a neural network is embedded with the combinatory constraints $A_4^{GC,RC}\left(n,d,\omega \right)$ to design the DNA codes $\left(n,d,\omega \right)$ of nucleotides that preserve the GC-content, Hamming distance $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$, and RC constraints. The proposed model is built on the three layers listed below:

• Transform the digital data into the sequence of bases (A, C, G, and T).
• Encode the DNA bases into optimal DNA codes.
• Create the bio-constraint codes for the DNA library construction.

This paper’s model novelty is based on layers 2 and 3. In contrast, the first layer is described in prior literature [3], in which, firstly, a digital data file is compressed into binary format. In our case, we compressed an image (cat.jpg) file into a binary file. Next, the binary file is preprocessed with different segments. Furthermore, it reiterates 2 computation processes: the Luby transform and screening. In the Luby transform, different bases are sets for fountain codes, while screening translates the binary droplet to a DNA sequence by converting (00, 01, 10, and 11) to (A, C, G, and T), respectively. Thereafter, we will term these codes premiere codes ξ $\subseteq {\sum }^n$.

After encoding, bio-constraints are applied to screen the sequences. However, in layer 2, we intend to apply the following neural network (NN) over the encoded premiere codes ξ $\subseteq {\sum }^n$ to generate optimal DNA codes ${\mathrm{\Phi }}_{DNA}$ for high-density storage. The 3rd layer introduces the bio-constrained coding to overcome the errors and construct the DNA library $\mathcal{L}=\sum \left(n,d,\omega \right)$. The comprehensive details of these layers (2 and 3) are given in the following subsections. The model diagram for the integration of NN with DNA coding constraints is illustrated in Figure 1.

4.1. NN-Based DNA Codes

The encoded premiere DNA codes ξ $\subseteq {\sum }^n$ are moved through the NN model to obtain the optimal DNA codes ${\mathrm{\Phi }}_{DNA}$. This model is based on 4 layers (encodings, 2 LSTM layers, and forward pass). In this neural network, 128 LSTM units have been considered, while the amount of hidden units is 4 times greater than that of the LSTM units and the dropout rate is set to 0.5 to avoid the overfitting issue. This rate will result in a 50% decrease in the number of neurons in the repetition oligonucleotides. It creates weight = 0 if a code is in a forward pass for a single iteration. During the training process, the trainable parameter is automatically set to false to prevent weight updates. The model is trained on forward and reverse input sequences $\left(\mathsf{\alpha },\mathsf{\beta }\right)$ to append the DNA codes, which must have different oligos in front of each other. In this paper, various primer templates are created from the ξ $\subseteq {\sum }^n$ with length 9 bases to make model learning efficient. The learning sequences are essential to attain the DNA codewords, avoiding identical bases. A sequence ${\mathsf{\alpha }}_i={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_n$, learned from one layer, is concatenated to another layer according to the forward pass mechanism. In the encoding layer, two single-stranded sequences $\left(\mathsf{\alpha },\mathsf{\beta }\right)$ are concatenated by inserting a particular connector token, which serves as an ending token also. In addition, another special token is appended at the beginning of the sequence. Each encoded base $E_i$ is indexed and fed through the LSTM layers. These layers are double stacked, and the unique tokens are transferred through the dense layers. Each sequence $\mathsf{\alpha }\mathrm{or}\mathsf{\beta }$ in these layers interacts with two-headed arrows to present the bi-directional LSTM for readability. All sequence nodes are initiated from 0 and updated based on the next nucleotide’s information. The hidden LSTM nodes predict the potential patterns and the forget gates update all nucleotides in the given DNA sequence. In the last layer, the final sequence updating is passed through the forward pass of LSTM to identify the forward-base DNA code ${\mathrm{\Phi }}_{DNA}$.

To construct the NN model that permits the constant flow of sequences $\left(\mathsf{\alpha },\mathsf{\beta }\right)$ through self-connected units, each oligonucleotide is protected with a self-linear unit $j$. The input gate $i{n}_j$ unit is responsible for protecting the linear unit $j$ from the other irrelevant units’ connection. Next, the critical unit, a memory cell, is designed for each DNA sequence with a linear unit $j$ to stop connecting with different DNA sequences. The memory cell of $j$ unit is indicated by $c_j$, with the current net sequence $ne{t}_{c_j}$ with $c_j$ achieving the input from the multiplicative $ou{t}_j$ unit, which is considered the output gate in the LSTM model. The activation of the input gate $i{n}_j$ and output gate $ou{t}_j$ with iteration time $t$ is indicated by $y^{i{n}_j}\left(t\right)$ and $y^{ou{t}_j}\left(t\right)$, respectively, which can be defined [23]:

$$y^{i{n}_j}\left(t\right)=f_{i{n}_j}\left(ne{t}_{i{n}_j}\left(t\right)\right),$$

(6)

where

$$ne{t}_{i{n}_j}\left(t\right)={\displaystyle \sum }_u{w}_{i{n}_{j}u}{y}^u\left(t-1\right).$$

$$y^{ou{t}_j}\left(t\right)=f_{ou{t}_j}\left(ne{t}_{ou{t}_j}\left(t\right)\right),$$

(7)

where

$$ne{t}_{ou{t}_j}\left(t\right)={\displaystyle \sum }_u{w}_{ou{t}_{j}u}{y}^u\left(t-1\right).$$

where $w$ stands for the number of the weight matrix and $y^u$ is the activation of an arbitrary unit $u$.

These activation functions enable the network to learn the complex features of each DNA sequence at the input gate $i{n}_j$ and output gate $ou{t}_j$. Although there are other weights and vector formulas for the LSTM gates [35,36], we omit them in this paper. However, we generalize the above activation functions to architect the forward pass for the final output. These functions learn the DNA bases to satisfy the following criteria for primer design.

• The DNA primer length is generally 15–30 nt [6]. The best length for PCR amplification primers is usually 20 nt; we also train our model at this limit.
• The length of repeated bases in a primer is generally ≤4 nt [5]. The consecutive appearance of any particular base makes the unstable DNA structure. We set consecutive base lengths ≤ 3 nt.
• The GC ratio of the primers should be 45–55% [3]. The bases A and T are linked by 2 hydrogen bonds, and C and G bases are connected by 3 hydrogen bonds (see Figure 6 [37]). We also consider the GC-content to be 45–55% in this work.

If the primer does not satisfy the above criteria, we alter one base from one primer. For example, if a primer does not satisfy the condition with the sequence AGGTCATC, we alter the first base ‘A’ with ‘T’, because they have a connected hydrogen bond between them, and reconfirm the criterion, while, if the primer meets the criteria, the premiere DNA codes ξ $\subseteq {\sum }^n$ are trained to the multiplicative units. The j-th memory cell block $c_j$, which found the input from multiplicative units $i{n}_j$ and $ou{t}_j$, will have a v-th unit of a memory cell block $c_j^v$ for a net input $ne{t}_{c_j^v}$ for time $t$:

$$ne{t}_{c_j^v}\left(t\right)={\displaystyle \sum }_u{w}_{c_j^{v}u}{y}^u\left(t-1\right).$$

(8)

The internal state $s_{c_j^v}$ and output activation $y^{c_j^v}$ of the v-th memory for a time $t$ with memory cell block $c_j$ will be:

$$s_{c_j^v}\left(t\right)=s_{c_j^v}\left(t-1\right)+y^{i{n}_j}\left(t\right)g\left(ne{t}_{c_j^v}\left(t\right)\right).$$

(9)

$$y^{c_j^v}\left(t\right)=y^{ou{t}_j}\left(t\right)h\left(s_{c_j^v}\left(t\right)\right).$$

(10)

The final net input for the index $k$, which ranges for output units and ranges of the final activation of output with $t$ are:

$$y^k\left(t\right)=f_k\left(ne{t}_k\left(t\right)\right),$$

(11)

where

$$ne{t}_k\left(t\right)={\displaystyle \sum }_{u: u not a gate}{w}_{ku}{y}^u\left(t-1\right).$$

Note that each memory cell has its weight w for the final net input $ne{t}_k\left(t\right).$ The DNA sequence is updated with the latest bases to design the DNA library. Finally, the LSTM cell determines the output by assigning these updates to the output gate $ou{t}_j$. The $ou{t}_j$ computes the final output activation $y^{c_j^v}$ that is passed through the cell as a final optimal DNA code ${\mathrm{\Phi }}_{DNA}$.

4.2. Combinatorial Constraints

This section deliberates the coding method to map the optimal DNA codes ${\mathrm{\Phi }}_{DNA}$ with sequence length $k\left(2n-1\right)$ to the DNA library ${\mathcal{L}}_k\left(n\right)$ with sequence length $kn$ that satisfies the combinatorial constraints ($\mathit{GC},d_H\left(\mathsf{\alpha },\mathsf{\beta }\right),\mathit{RC}$). The basic idea is to combine or concatenate the $k$ optimal sequences $\left(\mathsf{\alpha },\mathsf{\beta }\right)$ of length $n$ to the sequence of length $kn$ by constructing adjacency relations. For instance, if ${\mathsf{\alpha }}_i={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_n$ and ${\mathsf{\beta }}_i={\mathsf{\beta }}_1{\mathsf{\beta }}_2\dots {\mathsf{\beta }}_n$ are sequences with length n, then ${\mathsf{\alpha }}_i{\mathsf{\beta }}_i={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_{kn}{\mathsf{\beta }}_1{\mathsf{\beta }}_2\dots {\mathsf{\beta }}_{kn}$ is the concatenation of ${\mathsf{\alpha }}_i\mathrm{and}{\mathsf{\beta }}_i.$ Since the prescribed parameters of $k$ and $n>3$ to guarantee the sequence α is optimal are met, it is necessary that ${\mathsf{\alpha }}_{i-1}^{\prime }{\mathsf{\alpha }}_i$ will be an optimal sequence, if $\forall i\in \left\{2,3,\dots ,k\right\}$, where ${\mathsf{\alpha }}_{i-1}^{\prime }$ indicates the sequence which must have 3 symbols of ${\mathsf{\alpha }}_{i-1}$.

The constant GC-content $\omega$ can be presented analogously as $A_4^{GC}\left(n,d,\omega \right)$ for the $A_4\left(n,d,\omega \right)$ if all DNA codewords in ${\mathrm{\Phi }}_{DNA}$ have similar melting temperatures and each code desires to be $\omega$. The following are the upper and lower bound constraints for the DNA library ${\mathcal{L}}_k\left(n\right)$ construction. Proposition 1 is based on upper bounds with modified variables for the Hamming distance d, while the original proposition [18] considered only the number of sequences n. Due to new variables, the proof is presented with new codes for this work.

Proposition 1.

For the sequences$\left(\mathsf{\alpha }, \mathsf{\beta }\right)$having the number of codewords$n>0$, with constraint$0\le d\le n$and$0\le \omega \le n$for the upper bound,

(12)

Proof. 

Say that if there are 3 codewords having GC-content $\omega <\frac{n}{3},$ there will be some position $i$ where none of the words has C or G; thus, 2 of 3 words should agree in that position. Hence, $A_4^{GC}\left(n,d,\omega \right)\le 2$ and if $\omega <\frac{n}{3}$, then 2 codewords will be $C^{\omega }{A}^{n-\omega }$ and $G^{\omega }{T}^{n-\omega }.$ In contrast, if there are 4 codewords and none of the codes is agreed at any position $i$, then all 4 nucleotides will occur in each position of $i$. Thus, the average GC-content will be $\frac{n}{2},$ which is based on $A_4^{GC}\left(n,d,\omega \right)\le 3$, and 3 codewords with $\frac{n}{3}\le \omega <\frac{n}{2}$will be $C^{\omega }{A}^{n-\omega },T^{n-\omega }{C}^{\omega }$, and $A^{\lfloor \frac{n-\omega }{2}\rfloor }{G}^{\omega }{T}^{\lfloor \frac{n-\omega }{2}\rfloor }.$ Similarly, if there are 2 codewords and no agreement in any position, there can be 4 codes according to the pigeonhole principle. Thus, the average GC-content will be $\omega$ for $A_4^{GC}\left(n,d,\omega \right)\le 4$ and 4 codewords will be $A^{\omega }{C}^{\omega },C^{\omega }{A}^{\omega },T^{\omega }{G}^{\omega }$, and $G^{\omega }{T}^{\omega }.$ □

From this proposition, Theorem 1 is derived by considering the $n-1$ code length to generate the improved DNA coding sets. In contrast, Theorem 2 is an explicit condition for $d-1$ Hamming distance with constant GC-content to produce the DNA coding sets which satisfy both constraints.

Theorem 1.

A code with length$n$can be smaller than a code with length$n-1$with a minimum Hamming distance of$0\le d\le n$and$0<\omega <n$.

$$A_4^{GC}\left(n, d, \omega \right)\le \lfloor \frac{2n}{\omega }{A}_4^{GC}\left(n-1,d,\omega -1\right)\rfloor ,$$

(13)

$$A_4^{GC}\left(n, d, \omega \right)\le \lfloor \frac{2n}{n-\omega }{A}_4^{GC}\left(n-1,d,\omega \right)\rfloor .$$

(14)

Proof. 

In the case of Equation (13), the sequence ${\mathsf{\alpha }}_i$ for ${\mathsf{\alpha }}_1$ words with length $n$, $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right),$ and GC-content $\omega$, there will be position $j$ where ⌈$\omega {\mathsf{\alpha }}_1/2n$⌉ codewords have C nucleotide, or, at some position, it will be G. Otherwise, the average GC-content can be less than $\omega$. Considering those codewords and deleting position $j$ can generate $n-1$ and GC-content $\omega -1$ codes with minimum $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right).$ In contrast, Equation (14) is analogous, which only differs with GC-content for some position where $\lceil \left(n-\omega \right){\mathsf{\alpha }}_1/2n\rceil$ generates A’s or T’s. □

The inequalities in Equations (13) and (14) are applied to achieve the upper bounds on $A_4^{GC}\left(n,d,\omega \right)$ with $=d,$ $n=\omega$, or $\omega =0$ conditions. Similarly, different bounds can also be obtained by varying different orders; for instance, at constant $n=d$, Equation (13) can still be used after $n=\omega$ and Equation (14) after $\omega =0$.

Theorem 2.

For the maximum code length$n$with minimum distance$d-1$for the GC-content$\omega$in lower bounds,

$$A_4^{GC}\left(n, d, \omega \right)\ge \frac{\left(\begin{array}{c}n\\ \omega \end{array}\right)2^n}{{{\displaystyle \sum }}_{r=0}^{d-1}{{\displaystyle \sum }}_{i=0}^{\min \left\{\lfloor \frac{r}{2}\rfloor , \omega , n-\omega \right\}}\left(\begin{array}{c}\omega \\ i\end{array}\right)\left(\begin{array}{c}n-\omega \\ i\end{array}\right)\left(\begin{array}{c}n-2i\\ r-2i\end{array}\right)2^{2i}}.$$

(15)

Proof. 

By Equation (15), the numerator $\left(\begin{array}{c}n\\ \omega \end{array}\right)2^n$ provides the total codewords with GC-content $\omega$. The denominator yields the codewords with $d-1$ distance for a sequence α, while $\left(\begin{array}{c}\omega \\ i\end{array}\right)\left(\begin{array}{c}n-\omega \\ i\end{array}\right)\left(\begin{array}{c}n-2i\\ r-2i\end{array}\right)2^{2i}$ denotes the lower bounds, which give the codewords of a sequence β with GC-content ω that must satisfy the $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right),\mathrm{for}\mathrm{avoiding}\mathrm{the}\mathrm{error}r$. □

Apart from the GC-content, a reverse-complement constraint is integrated with this paper since we are employing the NJ algorithm for interaction energies $\mathsf{\phi }\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j\right)$ (Section 2) to control free energy for the secondary structures. To unfold the secondary structures before reading, let us consider a set of codewords, $\left\{AG,AC,TC,CA,TT\right\}\in {\sum }^n.$ Any DNA code ξ $\subseteq {\sum }^n$ in DNA codebook with $2n$ length is constructed by defining a bijective map φ between the quinary alphabet $Z_5$ and ${\sum }^n$, and then the net code rate ($R=lo{g}_{4}k/n$, $k$ is number of DNA coding set, and $n$ is number of sequence length) is, in this case, $lo{g}_{4}5/2\approx 0.58$ times of code. The bounds on free energy DNA code ξ $\subseteq {\sum }^n$ are presented in Proposition 2 to determine the secondary structure in a DNA sequence.

Proposition 2.

For all DNA sequences${\mathsf{\alpha }}_i={\mathsf{\alpha }}_1{\mathsf{\alpha }}_2\dots {\mathsf{\alpha }}_n$and${\mathsf{\beta }}_i={\mathsf{\beta }}_1 {\mathsf{\beta }}_2\dots {\mathsf{\beta }}_n$in DNA code ξ $\subseteq {\sum }^n$, the free energy $E_{1,2n}\ge -2n.$

From this proposition, the free energy $E_{i,j}$ reduces for DNA sequences over ξ. Hence, any DNA sequence ${\mathsf{\alpha }}_i$ or ${\mathsf{\beta }}_i$ in ξ $\subseteq {\sum }^n$ will avoid secondary structures. In the above coding sets, with length $2n$, $E_{1,2n}\ge -5\left|\frac{2n}{2}\right|=-5n.$ Now, from this proposition, we need to provide Theorems 3 and 4 to construct the model with which we avoid the secondary structure from any sequence.

Theorem 3.

Any DNA sequence$({\mathsf{\alpha }}_i$ or ${\mathsf{\beta }}_i)$ with length $2n$ in ξ $\subseteq {\sum }^n$ is free from the secondary structure if the stem length $l$ is more than 1 and minimum Hamming distance $d_H=d.$

Proof. 

Note that in any DNA sequence, if there is a secondary structure of stem length, then there will be 2 disjoint sub-sequences (α and β) with length $l$ and there is $\mathsf{\alpha }={\mathsf{\beta }}^{sc}$. The result will be contrapositive, such as if a DNA sequence frees from an SC (secondary-complement) sub-sequence with length $l$, then it will be freed from a secondary structure with a stem length of more than one. □

Theorem 4.

For any DNA code$A_4^{RC}\left(2n, K, d_H\right),$the codeword over ξ, ${\mathsf{\xi }}_{DNA}\cup {\mathsf{\xi }}_{DNA}^c$ will be ($2n, 2K, d_H$) if $d_H\le n$, wherein ${\mathsf{\xi }}_{DNA}^c=\left\{{\mathsf{\alpha }}^c:\mathsf{\alpha }\in {\mathsf{\xi }}_{DNA}\right\}$.

Proof. 

DNA code length and size follow the complement of DNA sequence in RC constraints for a given codeword over ξ, ${\mathrm{\xi }}_{DNA}\cap {\mathrm{\xi }}_{DNA}^c=\varnothing .$ Similarly, note that $d_H\left({\mathsf{\alpha }}^c,{\mathsf{\beta }}^c\right)=d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)\ge d_H$ and $d_H\left({\mathsf{\alpha }}^c,\mathsf{\beta }\right)=d_H\left(\mathsf{\alpha },{\mathsf{\beta }}^c\right)\ge n$. Hence, we have ${\mathrm{\xi }}_{DNA}\cup {\mathrm{\xi }}_{DNA}^c$ with minimum Hamming distance $\min \left\{d_H,n\right\}=d_H$ for the RC constraint. The results will follow the distance property of $d_H.$ □

After constructing Propositions 1 and 2 for the upper and lower bounds’ improvement and avoiding the secondary structure, respectively, we present the combinatorial constraints by utilizing Proposition 3 [20]. This proposition leads to Theorems 5 and 6, improving the lower bounds of k-constraint length and avoiding particular errors from the number of sequences with errors r.

Proposition 3.

Suppose the bounds over$A_4^{GC,RC}\left(n, d, \omega \right)$are concatenated with$GC\left(\mathsf{\alpha }, \mathsf{\beta }\right)$and$RC\left(\mathsf{\alpha }, \mathsf{\beta }\right)$constraints with the Hamming distance$d$for$0\le d\le n$and$0\le \omega \le n$. We have 2 cases:

If$n$is even,

$$A_4^{GC,RC}\left(n, d, \omega \right)=A_4^{GC,R}\left(n, d, \omega \right)$$

(16)

If$n$is odd,

$$A_4^{GC,R}\left(n, d+1, \omega \right)\le A_4^{GC,RC}\left(n, d, \omega \right)\le A_4^{GC,R}\left(n, d-1, \omega \right).$$

(17)

Proof. 

For any set of codewords with length $n$, if their complements in any subset replace all integers, the GC-content will be maintained due to the existence of a Hamming distance $d$ between each codeword of α and β. However, the reverse or reverse-complement and Hamming distance between the codewords are not maintained generally. Subsequentially, if $n$ is even, we can replace codeword ${\mathsf{\alpha }}_i$ by its complements to generate a new codeword ${\mathsf{\beta }}_i$ with the first $n/2$ coordination, and then $H\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j^R\right)=H\left({\mathsf{\beta }}_i,{\mathsf{\beta }}_j^{RC}\right)$ for all codewords ${\mathsf{\alpha }}_i$ and ${\mathsf{\beta }}_j$. In contrast, if $n$ is odd, we can replace codeword ${\mathsf{\alpha }}_i$ by its complements to generate a new codeword ${\mathsf{\beta }}_i$ with the first $\left(n-1\right)/2$ coordination, and then $\left|H\left({\mathsf{\alpha }}_i,{\mathsf{\beta }}_j^R\right)-H\left({\mathsf{\beta }}_i,{\mathsf{\beta }}_j^{RC}\right)\right|\le 1$ for all codewords ${\mathsf{\alpha }}_i$ and ${\mathsf{\beta }}_j$ [20]. □

Theorem 5.

The code with combinatorial constraint is optimal for maximum code length$n$and minimum distance$d=2$for the GC-content$\omega$in lower bounds if$0\le \omega \le n$ and $A_4^{GC,RC}\left(n, 2d, \omega \right)=\left(\begin{array}{c}n\\ \omega \end{array}\right)2^{n-2}.$

Proof. 

By Theorem 2, $A_4^{GC,RC}\left(n,2d,\omega \right)\le \frac{1}{2}{A}_4^{GC}\left(n,2d,\omega \right)=\frac{1}{2}\left(\begin{array}{c}n\\ \omega \end{array}\right)2^{n-1}=\left(\begin{array}{c}n\\ \omega \end{array}\right)2^{n-2}.$ Similarly, by Proposition 3 (16), $A_4^{GC,RC}\left(n,2d,\omega \right)\le \frac{1}{2}{A}_4^{GC,R}\left(n,2d,\omega \right)$, and Theorem 4.5 of [38], $A_2^R\left(n,2\right)=2^{n-2}$. In this argument, the set of all binary words for $2^{n-1}$ does not have palindromes for odd Hamming weight $M$, while the reverse of odd word weight is still odd weight when $n$ is even, so $2^{n-1}$ words are distributed into $2^{n-2}$ pairs $\left\{\mathsf{\alpha },{\mathsf{\alpha }}^R\right\}$, wherein each word from each pair indicates that $A_2^R\left(n,2\right)=2^{n-2}$. Thus, the product lower bounds $A_4^{GC,R}\left(n,d,\omega \right)\ge A_2\left(n,2,\omega \right).A_2^R\left(n,2\right)=\left(\begin{array}{c}n\\ \omega \end{array}\right)2^{n-2}$ for the Hamming distance between 2 separated words of odd weight $M$ should be at least 2; then, the inequality determines the Halving bound, $A_2^R\left(n,2\right)\le \frac{1}{2}{A}_2\left(n,2\right)=2^{n-2}.$□

The lower bounds with deletion or substitution errors ε and with $d\ge 2$ are not tight enough to generate the DNA library for high-density data storage. We can improve the lower bounds of the maximum number of sequences without errors $r$ by constructing the redundancy of explicit DNA codes with $\raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\log M$ by considering Shannon’s relationship [32] (Equation (2)). The purpose of Theorem 6 is to improve the lower bounds of sequences without errors $r$; for which, the lower bounds with deletion and substitution errors ε are considered with fixed numbers of errors.

Theorem 6.

Let$M, k, r, and \mathsf{\epsilon }$be positive integers with$r$and$fixed \mathsf{\epsilon }$. Suppose that$K>3logM+\mathsf{\epsilon }$. Then the redundancy of improved lower bounds is

$$\lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor \log M+\lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor \mathsf{\epsilon }-O\left(1\right).$$

Proof. 

For a sequence $\mathsf{\alpha }\in {\Gamma }_M^{L+2}$, we consider the sequence ${\mathsf{\alpha }}_1{\mathsf{\alpha }}_2{\mathsf{\alpha }}_3\dots {\mathsf{\alpha }}_M$ in the way of descending lexicographic order. Each sequence contains discrete code, so each sequence of length $K_{\mathsf{\epsilon }}$ occurs at most $2^{\mathsf{\epsilon }}$ times. Hence, the number of equivalence classes $m$ is exactly the number of odd weights $M$ with $2K_{\mathsf{\epsilon }}$ runs. This number is known to be (see [39,40], page 360).

$$m={\sum }_{j=0}^{2^{K_{\mathsf{\epsilon }}}}{\left(-1\right)}^j\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ j\end{array}\right)\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}+M-j\left(2^{\mathsf{\epsilon }}+1\right)-1\\ 2^{K_{\mathsf{\epsilon }}}\end{array}\right).$$

This expression for $m$ is inconvenient to work with, so we assign a lower bound on $m$. W.L.O.G., we consider that for $1\le i\le m_1$, where $m_1\le m$ and $m_1<i\le m$ with weight $M$ of discrete codes,

$$m_1=\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ M\end{array}\right)\le \frac{\left(\begin{array}{c}{2}^K\\ M\end{array}\right)}{2^{\mathsf{\epsilon }\mathrm{M}}}.$$

(18)

while the number of equivalence classes with repetitions is

$$m-m_1\le {\displaystyle \sum }_{K=1}^{M-1}\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ K\end{array}\right)K^{M-k}$$

where, in this expression, $\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ K\end{array}\right)$ gives the number of choices of these discrete codes, and $K^{M-K}$ counts the remaining $M-K$ sequences as repetitions of the K discrete ones. Since $L>3logM+\mathsf{\epsilon }$, when $K\le M-2$, we have

$$\frac{\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ K\end{array}\right)K^{M-K}}{\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ K+1\end{array}\right){\left(K+1\right)}^{M-K-1}}=\frac{{\left(K+1\right)}^2}{2^{K_{\mathsf{\epsilon }}}-K}{\left(\frac{K}{K+1}\right)}^{M-k}<1.$$

It follows that $\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ K\end{array}\right)K^{M-K}$ is increasing in $K$; hence,

$$m-m_1={\displaystyle \sum }_{K=1}^{M-1}\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ K\end{array}\right)K^{M-k}\le \left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ M-1\end{array}\right)M^2.$$

(19)

The Equation (18) is larger than Equation (19) w.r.t. discrete codes in each given sequence k:

$$\begin{array}{lll}\frac{\left(\begin{array}{c}{2}^K\\ M\end{array}\right)}{2^{\mathsf{\epsilon }M}}/\left(\left(\begin{array}{c}{2}^{K_{\mathsf{\epsilon }}}\\ M-1\end{array}\right)M^2\right)& =& \frac{\left(2^L-M+1\right)\left(2^K-M+2\right)\left(2^K-M+3\right)\dots 2^K}{M\left(2^{K_{\mathsf{\epsilon }}}-M+2\right)\left(2^{K_{\mathsf{\epsilon }}}-M+3\right)\dots 2^{K_{\mathsf{\epsilon }}}}·\frac{1}{2^{\mathsf{\epsilon }M}{M}^2}\\ & \ge & \frac{\left(2^{K_{\mathsf{\epsilon }}}-M+1\right)\left(2^{K_{\mathsf{\epsilon }}}-M+2\right)\left(2^{K_{\mathsf{\epsilon }}}-M+3\right)\dots 2^{K_{\mathsf{\epsilon }}}}{M^3\left(2^{K_{\mathsf{\epsilon }}}-M+2\right)\left(2^{K_{\mathsf{\epsilon }}}-M+3\right)\dots 2^{K_{\mathsf{\epsilon }}}}\\ & =& \frac{\left(2^{K_{\mathsf{\epsilon }}}-M+1\right)}{M^3}\ge 2^{K_{\mathsf{\epsilon }}-1-3\log M}\ge 1\end{array}.$$

Hence,

$$m\le \frac{\left(\begin{array}{c}{2}^K\\ M\end{array}\right)}{2^{\mathsf{\epsilon }M-1}}.$$

Now, let $\mathrm{Š}$ be an error-correcting code. According to the pigeonhole principle, the sequence size is least $\frac{\left|\mathrm{Š}\right|}{m}$ for a class $\mathrm{Ȿ}$, which indicates $\mathrm{Ȿ}\triangleq \mathrm{Š}$. Therefore,

$$\left|\mathrm{Ȿ}\right|\ge \frac{\left|\mathrm{Š}\right|}{m}\ge \frac{\left|\mathrm{Š}\right|}{\left(\begin{array}{c}{2}^K\\ M\end{array}\right)/2^{\mathsf{\epsilon }M-1}}.$$

(20)

Let $\mathsf{\Sigma }\triangleq {\left\{0,1\right\}}^{\mathsf{\epsilon }}$ and

$${\mathcal{L}}_k\left(n\right)\triangleq \left\{\right({\mathsf{\alpha }}_1\left[K_{\mathsf{\epsilon }}+1,K\right],{\mathsf{\alpha }}_2\left[K_{\mathsf{\epsilon }}+1,K\right],\dots ,{\mathsf{\alpha }}_M\left[K_{\mathsf{\epsilon }}+1,K\right])\in {\mathsf{\Sigma }}^M|\left\{{\mathsf{\alpha }}_1,{\mathsf{\alpha }}_2,{\mathsf{\alpha }}_3,\dots ,{\mathsf{\alpha }}_M\right\}\in \mathrm{Ȿ}\}.$$

We noted that while $\left\{{\mathsf{\alpha }}_1{\mathsf{\alpha }}_2{\mathsf{\alpha }}_3\dots {\mathsf{\alpha }}_M\right\}\in \mathrm{Ȿ}$ is a coding set, at this point, we use the lexicographic order to assign the indices, $({\mathsf{\alpha }}_1\left[K_{\mathsf{\epsilon }}+1,K\right]{\mathsf{\alpha }}_2\left[K_{\mathsf{\epsilon }}+1,K\right]\dots {\mathsf{\alpha }}_M\left[K_{\mathsf{\epsilon }}+1,K\right])\in {\mathsf{\Sigma }}^M.$

We suppose that ${\mathcal{L}}_k\left(n\right)\subseteq {\mathsf{\Sigma }}^M$ is a code of minimum Hamming distance at least $d$; otherwise, if there are two codewords in ${\mathcal{L}}_k\left(n\right)$ that have the Hamming distance at most $d+1$, then both concerned codewords in $\mathrm{Ȿ}$ can be confusable. Thus, by deleting the length of ε suffixes, the concerned codes will be different in ${\mathcal{L}}_k\left(n\right)$. Thus, by using the Hamming bound on $\left|{\mathcal{L}}_k\left(n\right)\right|$ which is same as $\left|\mathrm{Ȿ}\right|$, we have

$$\left|\mathrm{Ȿ}\right|\le \frac{2^{\mathsf{\epsilon }M}}{{\sum }_{i=0}^{\lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor }\left(\begin{array}{c}M\\ i\end{array}\right){\left(2^{\mathsf{\epsilon }}-1\right)}^i}.$$

(21)

By combining the Equations (20) and (21), we have

$$\left|\mathrm{Š}\right|\le \frac{2\left(\begin{array}{c}{2}^K\\ M\end{array}\right)}{{\sum }_{i=0}^{\lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor }\left(\begin{array}{c}M\\ i\end{array}\right){\left(2^{\mathsf{\epsilon }}-1\right)}^i}.$$

Hence,

$$\begin{array}{lll}\log \left(\begin{array}{c}{2}^K\\ M\end{array}\right)-\log \left|\mathrm{Š}\right|& \ge & \log \left({{\displaystyle \sum }}_{i=0}^{\lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor }\left(\begin{array}{c}M\\ i\end{array}\right){\left(2^{\mathsf{\epsilon }}-1\right)}^i\right)-1\\ & =& \lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor \log M+\lfloor \raisebox{1ex}{$r$}\!\left/ \!\raisebox{-1ex}{$2$}\right.\rfloor \mathsf{\epsilon }-O\left(1\right). \square \end{array}$$

In this paper, biologically constrained quaternary codes are pondered to use DNA primers economically. The pseudo-code of Algorithm 1 is utilized to generate the DNA library ${\mathcal{L}}_k\left(n\right)$ which is based on the optimal DNA codes ${\mathrm{\Phi }}_{DNA}$ designed by a neural network. This algorithm produces the codes that satisfy the GC-content $\omega$, reverse constraint, and Hamming distance $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$ using the quaternary encoding.

Algorithm 1. Proposed algorithm to construct DNA library ${\mathcal{L}}_k\left(n\right).$

Input:  Premiere DNA codes ξ $\subseteq {\sum }^n$, optimal DNA codes ${\mathrm{\Phi }}_{DNA}$, GC-content $\omega$, code length $n$, Hamming distance $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$, and reverse constraint;

Output:  DNA library ${\mathcal{L}}_k\left(n\right).$ Convert binary data into ξ $\subseteq {\sum }^n$ by quaternary encoding $\left(4\times 3^{n-1}\right)$ Initiate the NN with activation gates $y^{i{n}_j}\left(t\right)$ using Equation (6) and $y^{ou{t}_j}\left(t\right)$ using Equation (7) to encode the primers; Generate optimal DNA codes ${\mathrm{\Phi }}_{DNA}$ by output activation $y^{c_j^v}\left(t\right)$ using Equation (11) and LSTM layers; Remove the codewords from ${\mathrm{\Phi }}_{DNA}$ if that does not follow the GC-content $\omega$ (Proposition 1 and Theorem 2); Reverse the DNA codes that enable secondary structures (Theorems 3 and 4) and avoid the codes that do not satisfy $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$ $\left(d-1\right);$ Concatenate the bio-constraints $A_4^{GC,RC}\left(n,d,\omega \right)$ for $n$ code length by Proposition 3 and Theorem 5; Construct the error-correcting codes to produce the final DNA library ${\mathcal{L}}_k\left(n\right)$ by Theorem 6.

return: DNA library ${\mathcal{L}}_k\left(n\right)$ for DNA data storage.

5. Result Evaluations

This section elaborates on the improved lower bounds and DNA coding sets obtained by the proposed model of NN and combinatorial bio-constraints. Figure 2 illustrates a random sample of forward and reverse primers for the optimal DNA codes received after the NN implementation. These random DNA sequences were programmed in the Magma program [41] with different sequence lengths and a minimum Hamming distance. The aforementioned propositions and theorems were considered for program construction. As a result, we received $.cod$ files with different lower bounds of DNA codes, satisfying the combinatorial bio-constraints for particular $n$ and $d$. The codes in the $.cod$ files were calculated in the Tables format. In addition, Figure 3 illustrates the numerical analysis by considering the coding rate and storage density of lower bounds given in these tables. Figure 3’s analyses were drawn by using the Prism program.

Table 1. Comparison of our lower bounds with [10] for $A_4^{GC}\left(n,d,\omega \right)$.

n/d	d = 3	d = 4	d = 5	d = 6	d = 7	d = 8	d = 9	d = 10
4	11 12 i
5	17 21	7 7
6	44 59	16 19	6 7
7	110 143 i	36 52 i	11 19 i	4 4
8	289 303	86 115 i	29 36	9 10	4 4
9	662 864 i	199 291 i	59 61	15 31i	8 7 d	4 5
10	1810 1973	525 604 i	141 171 i	43 51 i	7 21 i	5 6	4 4
11	4320 5764 i	1235 1716 i	284 401 i	82 125 i	29 41 i	9 17 i	4 5	4 4
12	12,068 11,618 d	3326 4986 i	662 617 d	190 711 i	58 72	22 29 i	8 11 i	4 4
13	41,867 57,322	7578 8113	1432 2564 i	1201 1391	123 368	39 71 i	13 21	6 8 i

and

Table 2. Comparison of our lower bounds with [17] for $A_4^{GC}\left(n,d,\omega \right)$.

n/d	d = 3	d = 4	d = 5	d = 6	d = 7	d = 8	d = 9
4	12 12
5	20 29 i	8 14 i
6	58 63 i	24 27 i	8 12 i
7	125 118 d	44 51	17 22	7 10 i
8	324 334 i	106 124 i	35 41 i	14 17 i	5 8 i
9	713 921 i	223 237	64 94 i	24 23 d	10 14 i	5 7 i
10	1906 2010	555 913 i	159 163	51 48 d	20 21	10 12 i	4 4

present the lower bounds obtained by our model of GC-content ω and $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$ with the NN. In each row, the upper entries in Table 1 are directly taken from [10], while the upper entries in Table 2 belong to [17]. The lower entries are used to compare our outputs with existing studies. The superscript $i$ represents improved lower bounds and $d$ indicates the decreased lower bounds, while the rest of the other bounds have almost the same lower bounds as compared to [10] and [17], respectively.

In Table 1, the lower bounds are based on GC-content ω and $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$ by deriving Proposition 1 and Theorem 1, and they are compared with Table 1 of [10], which uses the $4\le n\le 13$ and $3\le d\le 10$ inequalities to construct the DNA codes. We have compared our proposed model’s results with [10] by considering the GC-content ω and $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$ with NN. As a comparison, 51% of bounds are improved, 5% have decreased, and 44% are almost the same lower bounds as in [10].

Similarly, in Table 2, the lower bounds are based on RC constraint and $d_H\left(\mathsf{\alpha },\mathsf{\beta }\right)$ by deriving Proposition 1 and Theorem 2 and are compared with the Table 7 of the study [17], which considers the $4\le n\le 10$ and $3\le d\le n$ inequalities to design the DNA codes. In comparison, 64% of bounds are improved in our work, while 11% have decreased and 25% are almost the same as the lower bounds of [17]. However, the limitations of these bounds can be further improved by constructing new theorems or modifying Proposition 1 by varying the values of n and ω.

Apart from the lower bound improvements for the given constraints, the coding rates $(R=\frac{1}{n}lo{g}_{4}L$, $n$ is the sequence length number, and L is the total number of lower bounds in a sequence) have also been improved in a shorter sequence $\left(n-1\right)$. For instance, ref. [10] reported $R=0.3036$ when $n=8\mathrm{and}d=5$, while our work reports the same coding rate (0.3034) with a shorter sequence when $n=7\mathrm{and}d=5$. Similarly, ref. [17] obtained $R=0.4881$ when $n=6\mathrm{and}d=3$; in contrast, this work receives this coding rate (0.4857) when $n=5\mathrm{and}d=3$. The reported improved lower bounds have a better influence on the DNA library ${\mathcal{L}}_k\left(n\right)$ generation, indicating the proposed model’s effectiveness for DNA code construction.

Regardless of this improvement, the 95% confidence interval ($\mathrm{CI}$) mean (Figure 3a) of received bounds presents a breakthrough coding for DNA data storage in DNA computing. The bigger the interval, the more significant the development of coding for DNA data storage. As the purpose of individual RC constraints is to avoid the secondary structures in the DNA sequences, the RC constraints are not concerned with generating the lower bounds separately. However, the studies [11,13,18,20] motivate the idea of integrating the RC constraint with GC-content and Hamming distance in an assembled format to design new DNA coding sets. Taking advantage of their work, we generalize the RC constraint with Proposition 3 and Theorems 5 and 6 to generate the new DNA codes.

Table 3. Comparison of our lower bounds with [20] for $A_4^{GC,RC}\left(n,d,\omega \right)$.

n/d	d = 3	d = 4	d = 5	d = 6	d = 7	d = 8	d = 9	d = 10
4	6 6
5	15 27	3 4
6	44 67	16 21	4 4
7	135 243	36 69	11 19	2 2
8	528 617	128 148	28 42	12 15	2 2
9	1354 1827	275 430	67 121	21 36	8 11	2 2
10	4542 5914	860 1181	210 271	54 77	17 27	8 8	2 2
11	14,405 23,713	2457 6429	477 961	117 557	37 59	14 23	5 8	2 2
12	59,136 67,761	14,784 19,132	1848 2062	924 1092	87 131	29 41	12 18	4 6

is the collection of lower bounds with combinatorial constraints. Each column has upper and lower entries; the former is taken from the Table 8 of the study [20], and the latter is attained by our proposed computational model. The bold entries indicate the outperformed bounds of our proposed model over [20]. Likewise, the coding rates are compared in Figure 3b for $n=8$ of our lower bounds with $n=9$ of [20]. Our model designs the codes with almost the same $R$ with $n-1$ sequences. In addition, the underlined entries indicate the best-known codes of this work that have nine bounds total. In [20], Tables 8 and 9 present the best-achieved bounds which satisfy the GC-content, Hamming distance, and RC constraints; we only compare our results with Table 8 due to its particular inequalities (i.e., $3\le d\le 11$). As Table 1 and Table 2 are also based on these inequalities, we focus on a particular inequality in this paper for all the results.

The new lower bounds delivered by this work are better than the prior work. For instance, for $n=10$ and $d=5$, the size of our DNA codes is 22% greater than that of [20]. In another scenario, if we consider all the sequences at $d=6$, the new improved DNA codes are still 36% better than [20]. These significant improvements are based on our proposed computational model that integrates a neural network with combinatorial bio-constraints. In addition, the size of these DNA codes is still capable of increasing as that of the best-known codes for the highest storage density. The storage density with our DNA codes for $n=9$ to $n=12$ and $d=3\mathrm{to}d=8$ is given in Figure 3c. The high storage density is received in lower Hamming distance, which is also based on the DNA coding sets of each sequence length. For the given particular lower bounds in Figure 3c, the highest density of 4.41 is attained for $d=3.$

Furthermore, the improvements of these lower bounds for any sequence length pioneer the DNA coding rates. A general analysis of Table 3 indicates that the same coding rate (R) is found in 73% of lower bounds with shorter sequences. For example, ref. [20] received $R=\frac{1}{12}lo{g}_{4}87=0.2684$ when $n=12$ and $d=7$, while this work acquires the same coding rate (0.2673) when $n=11$ and $d=7$. Similarly, in another example, when $n=10$ and $d=4$, ref. [20] reported a 0.4874 coding rate; in contrast, this work delivers $R=0.4860$ with the number of sequence lengths $n=9$ at the same Hamming distance. In the case of best-known codes (bold underlined entries), our coding rate is better than [20], with a shorter sequence $\left(n-1\right)$, i.e., $n=8,d=3$, and $R=0.5652$, while this work reports $R=0.5660$ when $n=7\mathrm{and}d=3$.

Thus, these analytical results present that the shorter sequences can achieve the same DNA storage density as the longer sequences. The improved lower bounds in various coding sets indicate the reduction in insertion and deletion errors in the DNA sequences, which enables the proposed computational model to avoid the non-specific hybridization process. In addition,

Table 4. DNA coding sets for DNA library ${\mathcal{L}}_k\left(n\right)$ retrieved when $n=10$ and $d=7$.

GAGTCTAGAC	CTGTATGCAT	TACTAGACAG
GTCTGACATA	CACTACTGAC	ACTGTAGCAT
ATGACTCACT	GATACGACAT	CTACGTAGCA
TACTGTCACG	ACATCTGTCA	TGCACATGAC
AGCATACTCA	TACATCTGCT	GACATGACAG
CGATGTACTG	AGACGATGTC	TGTAGCTACA
CAGTAGATCA	TACGATCGAG	AGATCGACTG
GACTCATGAC	CACGTCTGAT	GCATAGTATC
ACTGACTACT	ACGCAGATAC	TGCGATACTA

presents the DNA library ${\mathcal{L}}_k\left(n\right)$ satisfying the $A_4^{GC,RC}\left(n,d,\omega \right)$ constraints when n = 10 and d = 7, as in Table 3. The satisfaction of combinatorial constraints over the optimal DNA codes from the NN’s output collaborated to improve the lower bounds of DNA coding sets, which emphasizes our proposed computational model.

6. Conclusions

An exciting research challenge in DNA data storage systems is to explore improved lower bounds by avoiding non-specific errors to generate high-density-based storage, which could store a large amount of information in a shorter sequence. In this paper, a novel computational model is offered to construct an extensive DNA library of oligonucleotides. It is accomplished by presenting a three-layer model that integrates a neural network (LSTM) and combinatorial bio-constraints, including GC-content, Hamming distance, and reverse-complement constraints. We derive the recursive expression in propositions and theorems to attain all possible large DNA coding sets by satisfying combinatorial constraints.

All DNA codewords in Table 1 and Table 2 satisfy the GC-content and Hamming distance constraints and improve 51% and 64% of lower bounds compared to [10] and [17], respectively. The lower bounds presented in Table 3 are single error-correcting codes based on the concatenation constraints, while the underlined bounds exhibit the DNA sequences that have avoided secondary structures. Furthermore, the improvements in the lower bounds directly impact the coding rate. For example, results in Section 3 report that the shorter sequences can achieve the same DNA storage density as the longer sequences. It is concluded that the proposed computational model can store a large amount of data in a small number of DNA nucleotides that can improve the data density and reduce the DNA synthesis and sequence cost for a DNA-based data storage system.

In our results, there are still lower bounds that need to be improved by mutation strategies for high-density data storage. Similarly, the insertion and deletion errors can be further controlled by experimenting with the application-oriented bio-constraints, i.e., run-length constraints [42].