Classification of the Human Protein Atlas Single Cell Using Deep Learning

Abstract

Deep learning has made great progress in many fields. One of the most important fields is the medical field, where we can classify images, detect objects and so on. More specifically, deep learning algorithms entered the field of single-cell classification and revolutionized this field, by classifying the components of the cell and identifying the location of the proteins in it. Due to the presence of large numbers of cells in the human body of different types and sizes, it was difficult to carry out analysis of cells and detection of components using traditional methods, which indicated a research gap that was filled with the introduction of deep learning in this field. We used the Human Atlas dataset which contains 87,224 images of single cells. We applied three novel deep learning algorithms, which are CSPNet, BoTNet, and ResNet. The results of the algorithms were promising in terms of accuracy: 95%, 93%, and 91%, respectively.

1. Introduction

Deep learning methods [1] are among the most popular methods of classification and prediction [2]. Currently, deep learning algorithms expand in many areas within different sectors such as gaming [3], image classification [4], autonomous cars [5], and image recognition [6], Plant Diseases [7] IoT Scenarios [8], COVID-19 Detection and management [9,10], Architectural Heritage Images [11], among others.

One of the most common areas that use deep learning algorithms is medical image processing because of its complexity, requiring a lot of time to analyze, with the need to process huge amounts of images [12,13]. In this study, we will apply deep learning algorithms to the medical field and in particular to human cells classification [14], by using the public images dataset called the Human Cell Atlas (HCA). The main aim of this study is to classify the structure of the cell into nineteen classes. This classification helps in the distinction of cell types and the changes happening in it [15].

The beginning of dealing with medical images for prediction and analysis was through dealing with the images resulting from computed tomography, X-rays, radiographs, and so on, to identify a patient’s problem. However, this was not sufficient for early detection of diseases [16]. Then, the quality of the images that are used in the training of deep learning algorithms was developed and that allowed for dealing with different samples of the human body tissues that contain thousands of cells [17]. This helped to improve the outcomes and results, but it is still difficult to determine the changes that occurred within a single cell. Therefore, the type of images that are used in training algorithms has evolved into single-cell images in order to reveal the cell’s structure, the locations of the proteins within it, and any other changes within it; studies have proven the relationship of these changes in the single-cell by discovering diseases early and helping to find the appropriate treatment [18]. The scope of this research is to integrate deep learning, which is part of artificial intelligence, with the single cell, which is part of biology, with medical images that come to us from the medical field as shown in Figure 1.

The contribution of this study is to use a new algorithm called Bottleneck Transformers for Visual Recognition to classify the single-cell for humans. Often, deep learning and computer vision algorithms use batch normalization to obtain the best result [19]. We found a significant improvement on the image classification [20], instance segmentation [21,22], and object detection [23,24,25] by deep convolutional backbone structures [4,26,27]. However, the architecture of Bottleneck Transformers is derived from backbone architectures. Typically in their architectures, they use multiple layers of 3 × 3 convolutions [26]. This is because the warp layers effectively aid in computer vision tasks such as object detection, image classification, position segmentation, and key point detection. This architecture helps us to obtain a powerful solution that is able to scale without adding many layers. In addition, in this research, we used another algorithm called the Cross-State Partial Network. This algorithm has special architectures to give good accuracy with less mathematical complexity. This allows anyone who has a computer with a low requirement to use a deep learning algorithm and obtain a good result. Finally, we used the Residual Networks ResNet algorithm on the single-cell dataset with nineteen classes to see how we can enhance the accuracy of calcification. ResNet was ranked as a top 10 accurate algorithm in the list of best algorithms and won in the ImageNet challenge in 2015.

The remainder of this paper organized as follows: Section 2 presents an overview of Cell Classification and Single Cell Classification using deep learning algorithms in the literature review. Section 3 describes the data set used to train the model. Section 4 presents the pre-processing setup of the project. Section 5 provides the methodology conducted in this study. Section 6 presents the results of the algorithm. Finally, Section 7 summarizes this research and proposes future work to enhance the model.

2. Literature Review

There are billions of humans on this earth, and each of them contains trillions of cells that generate huge amounts of medical images [28]. The researchers used artificial intelligence (AI) algorithms to predict information from medical images. Then, they used deep learning models for classification cell images to obtain more data from them. In addition, to obtain higher accuracy, deep learning algorithms were used on a single cell to classify the components of the cell and the location of proteins in it. The following sections reviews some of the articles in this field and

Table 1. Cells classification.

Ref.	Dataset	Algorithm	Classes	Accuracy
[29]	Human Protein Atlas (HPA)	Project Discovery	29	72%
[29]	Human Protein Atlas (HPA)	Localization Cellular Annotation Tool Loc-CAT	29	72%
[30]	42,774 nonpublic images + HPAv18	DenseNet20 + Network size medium	–	59.3%
[30]	42,774 nonpublic images + HPAv18	DenseNet20 + Focused on data preprocessing	–	57.1%
[30]	42,774 nonpublic images + HPAv18	DenseNet20 + Focused on Data augmentation	–	57.0%

and Table 2 summarizes them.

2.1. Cell Classification

In [29], Sullivan et al. combined two image classification methods for large-scale classification. The dataset used in this paper is public data called Human Protein Atlas (HPA). The first method is a video game called Project Discovery, which helps in the task of image classification for which 322,006 players participated. The second method is used by a deep learning model called the Localization Cellular Annotation Tool (Loc-CAT) [29]. There are 29 classes of proteins in images. After combining these two methods, the accuracy of protein distribution is F1 score of 0.72.

In 2019 [30], a competition was created to develop models for the classification of protein microscope images with a multi-label [31]. Previously, there was a model for classifying the proteins images dependent on cell types of different morphology was called Loc-CAT [29]. However, the performance of this model is less than (0.47) when using the F1 measure. The competition has 2172 teams over a period of three months with a total of 55,213 submissions. The dataset used was 42,774 nonpublic images prepared to be used in this competition. In addition, the teams can use external data such as Human Protein Atlas image collection (HPAv18) contain 78,000 images. There are two challenges in this competition, namely the class imbalance and multi-label problems. The first challenge is the frequencies of classes; the imbalanced such as the most common class, more than other classes, is ‘nucleoplasm’, and there have been (12,885 images in the training and 31,590 images in HPAv18). The second challenge is the multi-label problem; this means the need to assign the multi-classes to each image. After the final ranking, the F1 scores of the top three teams are 0.593, 0.571, and 0.570 sequentially.

2.2. Single Cell Classification

In [32], Durr et al. analyze the high-content screening (HCS) dataset to detect cellular morphology and protein localization. This study suggested three machine learning algorithms that are SVM, LDA, and random forest to classify the features. The result of this model is SVM is 87.6%, the random forest is 89.2%, and LDA is 91.1%. Then, train on Convolutional Neural Net CNN. The result after 100 epochs showed the validation accuracy being on average 0.9313 (SD, 0.0079).

In [33], Meng et al. developed a framework to classify single cells using deep learning. The algorithm used to do the classification was a CNN-based model. In addition, a single-cell image dataset obtained from the Quantitative Phase Imaging QPI system was used and, in the preprocessed phase, the author’s additional channel and augmentation and intensity normalization. Then, each pixel in each channel was normalized to be between 0 and 1. After that, each cell imaging with four different fiber coupling angles was represented. Finally, this model can classify different types of single-cell images automated without any human intervention with an accuracy of 96.5%. In addition, this work was supported in part by the NSFC/RGC under Project N_HKU714/13, and GRF 17245716, and the Croucher Innovation Award.

Table 2. Single cell classification.

Ref.	Dataset	Algorithm	Classes	Accuracy
[33]	QPI	CNN	–	96.5%
[32]	BBBC022v14	SVM	4	87.6%
[32]	BBBC022v14	random forest	4	89.2%
[32]	BBBC022v14	LDA	4	91.1%
[32]	BBBC022v14	CNN	4	93.4%
[32]	BBBC022v14	SVM	3	88.6%
[32]	BBBC022v14	random forest	3	88.6%
[32]	BBBC022v14	LDA	3	94.6%
[32]	BBBC022v14	CNN	3	97.3%

Table 2. Single cell classification.

Ref.	Dataset	Algorithm	Classes	Accuracy
[33]	QPI	CNN	–	96.5%
[32]	BBBC022v14	SVM	4	87.6%
[32]	BBBC022v14	random forest	4	89.2%
[32]	BBBC022v14	LDA	4	91.1%
[32]	BBBC022v14	CNN	4	93.4%
[32]	BBBC022v14	SVM	3	88.6%
[32]	BBBC022v14	random forest	3	88.6%
[32]	BBBC022v14	LDA	3	94.6%
[32]	BBBC022v14	CNN	3	97.3%

3. Dataset

In this study, we will use the public dataset HPA as shown in Figure 2 [34]. There are 87,224 images in train_images with all the images divided into four folders; each folder has a different filter as follows: the protein of interest (green), nucleus (blue), microtubules (red) and endoplasmic reticulum (yellow). In addition, there are 19 labels to predict protein organelle localization as follows: 0. Nucleoplasm, 1. Nuclear membrane, 2. Nucleoli, 3. Nucleoli fibrillar center, 4. Nuclear speckles, 5. Nuclear bodies, 6. Endoplasmic reticulum, 7. Golgi apparatus, 8. Intermediate filaments, 9. Actin filaments, 10. Microtubules, 11. Mitotic spindle, 12. Centrosome, 13. Plasma membrane, 14. Mitochondria, 15. Aggresome, 16. Cytosol, 17. Vesicles and punctate cytosolic patterns, and 18. Negative. In addition, we have a file called train.csv content, the image id, and multi-label classification column. Finally, we have a file called sample_submission.csv; this file is not a multi-label but segments every single cell contained in the image to the class of that cell [35].

4. Pre-Processing

In this section, we present the preprocessing steps carried out on the data so as to improve the accuracy of algorithms. There are many ways to apply preprocessing steps. In this study, we apply different preprocessing approaches to each algorithm as detailed below.

We use different preprocessing for each algorithm such as cropping at a random size, flipping horizontally with a 0.5 axis, flipping vertically with a 0.5 axis, transposing the image which is similar to rotating the image 90 degrees and cutting out parts of the image to make sure it is still able to identify the key points and changing values of the hue to incorporate the changes in lighting …, etc. All this was completed by utilizing a library in Python called Albumentations for image augmentation. The purpose of image augmentation is to create new training samples from the existing data. This library supports all common computer vision tasks such as classification, semantic segmentation, instance segmentation, object detection, and pose estimation. The library provides a simple unified API to work with all data types such as RBG-images, grayscale images, multispectral images, segmentation masks, bounding boxes, and key points. The library contains more than 70 different augmentations to generate new training samples from the existing data. This data augmentation makes the 2×, 4×, or even 8× times the original data and thus helps our model generalize well and give more suitable predictions.

5. Methodology

Since the ultimate objective was to classify single cells in the Human Protein Atlas (which would be referred to as HPA later) dataset, we had to draw out a comparison between the three different architectures that we would make use of. We were given Red, Green, Blue, and Yellow Image channels for a single record so it was quite evident that we would be using Convolutional Neural Networks as the main architecture for our research. CNNs come with a lot of different variants and architectures, each with its own pros and cons and, in this report, we will discuss how each architecture performed against the validation data. In the following, we present the three architectures that we have worked and trained our model on.

5.1. CSPNet Algorithm

We choose the CSPNet model to apply in this study because this algorithm reduces using the memory and the computation cost and increases the accuracy and speed; this the main aim of CSPNet [36]. The algorithm does that by dividing the feature map into two parts and then merging them through a proposed cross-stage hierarchy. In our CSPNet model, we have used transfer learning and made use of a pre-trained model on the CSPNet to save us time and obtain better results at the same time. In addition, we used the rectified linear activation function The ReLU activation function for the hidden layers of our code for its obvious advantages such as the output will be direct if the input is positive, and, if the input is negative, the output will be zero [37]. In addition, we use this activation function to help our model to learn faster and obtain the best result. Because we have a multiclass classification problem, we used the softmax activation function at the output layer to predict the probability of each label [38]. Then, we used a Python library to find the best learning rate for our algorithm.

5.2. ResNet Algorithm

We chose this algorithm because it was ranked top 10 accuracy algorithms in the list of best algorithms. In this project, we made use of PyTorch and its internal dependencies for this project including the Lightning module [39]. There are more features for use ResNet such as it allows us to solve the problem of vanishing gradients; this was a very big problem for deep deep neural networks. We made use of pre-trained ResNet weights online that were trained by different researchers from across the world and used transfer learning to get it accomplished. In a simple way, we downloaded the weights of ResNet50 via the PyTorch library. Our model has 50 layers. We froze the pre-trained weights and added a simple convolution and pooling layer.

5.3. BoTNet Algorithm

The BoTNet is a conceptually simple yet powerful backbone architecture that incorporates self-attention for multiple computer vision tasks by just replacing the spatial convolutions with global self-attention [40] in the final three bottleneck blocks of a ResNet and no other changes. The architecture of our model contains three layers. If you notice, you will find a 512 neuron-sized Multi-Head Self-Attention MHSA [41] layer that stands for Multi Headed Self Attention. This was the breakthrough in achieving even higher accuracy. This MHSA layer was actually a 3 × 3 Convolution layer in ResNet, but in BoTNet, it was replaced by an MHSA layer to improve performance. Since we have used BoTNet, we made use of transfer learning from the pre-trained BoTNet to have the weighted parameters. Then, we added our own layer on top of it in a sequential manner. We added a convolution layer, an average pooling layer, and then flattened it out and passed it to a linear layer that predicted our outputs. For the hyperparameters, we have used the learner PyTorch module, which helped us find the best learning rate for our program which turned out to be 0.030.

6. Results and Evaluation

In this section, we will present the results of the three algorithms that we applied in this study. Then, we will compare them to see which algorithm gives the best accuracy for classifying the single cell.

To start with, we tried the CSPNet algorithm on the original dataset before augmentation steps discussed in previous sections, and we obtained a good result compared to existing research, which was 84% for the accuracy measure, but due to the sensitivity of the classification task at hand, we aimed to improve the results by applying additional preprocessing steps as explained earlier.

The result of the CSPNet algorithm after prepocessing and hyperparameters’ setting is presented below. We ran it against nine Epochs from 0–8 inclusive. The accuracy in Epoch 0 was 94.2%, and the accuracy improved in Epoch 8 and reached 95%; to represent the relation between training and validation to clarify, there is no overfitting as shown in Figure 3.

When we ran the ResNet model for 340 Epochs, our model reached a validation accuracy of 0.91, which is excellent as shown in Figure 4. It was trained on Kaggle TPU. If we had not used transfer learning from ResNet, the accuracy would not have been that much better. Utilizing the pre-training model on images helped us reach a 90%+ accuracy on the validation set. We modeled to a total of 25.6 Million params, and it was trained on Kaggle 8 cored Kaggle TPU.

When we ran the BoTNet model, the accuracy we have is 93.8% in second epochs. In addition, there is no overfitting as shown in Figure 5. The accuracy we reached is quite promising for this model with only a thousand records. The idea of self-attention definitely helped us achieve better results. For every channel, using the idea of self-attention, the accuracy, therefore, improved from 0.91 to 0.937, although, if we had even higher computational power, we could have possibly increased the accuracy by training on even more epochs.

After that, we run the three algorithms on the same training dataset with the different preprocessing steps. We ranked the models on the basis of model loss as follows: CSPNet, BoTNet, and ResNet as shown in Figure 6.

CSPNet outperformed every other model with a accuracy of 95%. This model uses an optimization network that allows the computations to be reduced by 20%, thus giving aid to higher accuracy. Since CSPNet was implemented on top of ImageNet, it has already made use of transfer learning, and it has stable pre-trained parameters from the image net model to give it an accuracy boost. We have also used image augmentation to increase the data and improve predictions. The second on the list was BoTNet, which was developed by a group of researchers specifically for these kinds of situations and recurrent neural networks. This allows us to improve in situations where we have multiple channels or words, and they impact each other. In our case, we had different channels and we were dealing with them separately but BoTNet allowed us to put an impact of one channel on the other one and combined their effect using a phenomenon called self-attention of the transformer neural networks. That is how it was able to perform so well and give an accuracy of 93.9%. Finally, ResNet came in third on our list with accuracy being 0.91%. ResNet is in itself a good model and has won the 2015 image net competition, and the technology it introduced called skip connections has greatly added to the machine computer vision and deep learning community. In addition, it ranks in the top 10 image classification algorithms on the Imagenet accuracy list. The concept of BoTNet self-attention is somehow related to the concept of ResNet skip connections. Nevertheless, it outperformed other models and was able to outperform it for the obvious reasons that they were making use of the pre-trained weights of ResNet and adding further convolutional and pooling layers on top of it. Thus, no doubt they would be better but it does not diminish the importance of ResNet which has been around for years and is always performing well. We summarize the results of three algorithms in this study in the following

Table 3. The result of three algorithms.

Algorithm	Number of Epochs	Train Loss	Valid Loss	Accuracy
CSPNet	9	0.16	0.1528	95%
BoTNet	4	0.19	0.2007	93%
ResNet	340	0.22	0.1528	91%

. Finally, although one network performed better than the others in this case, all of these models have their own usage and specialties, and there is no golden model to pick for every problem that would give the best accuracy in all the datasets and problems.

7. Conclusions and Future Work

In this research, we used deep learning algorithms because it excels at dealing with large and complex data. In addition, we chose three algorithms of deep learning to classify single cells into 19 classes. As it is possible that one image contains more than one class, this will be multi labeling. In addition, we used single-cell classification because that gives us better results than the classification of human tissue, which contains thousands of cells. The algorithms we used were CSPNet, BoTNet, ResNet and we applied them to the Human Protein Atlas dataset with a size of 158.36 GB. After that, we apply different preprocessing approaches to data for each algorithm. The accuracy we obtained from classification of the single-cell image was 95% for the CSPNet algorithm, while the algorithm that obtained the least accuracy was ResNet.

In future work, we seek to increase the accuracy of single-cell classification by trying different algorithms and preprocessing. We will also study the impact of quality or resolution of images on the results. For example, in the used dataset, there were images with different resolutions such as 1728 × 1728, 2048 × 2048, and 3072 × 3072. Further experiments are needed to show the impact of good/bad resolutions or the type of noise in the images on the overall results. We also aim to increase the number of classes in single-cell to reach highly accurate results, which would help discover diseases early and also to help innovate drugs to treat these diseases, in addition to determining the expected disease based on the changes taking place inside the cell. We would like to specifically work on early detection of the two types of cancer that are most common in the world, namely breast and lung cancer. Cancer is a major cause of death worldwide and caused about 10 million deaths in 2020, where the number of people with breast cancer was 2.26 million cases and lung cancer 2.21 million cases, according to the statistics of the World Health Organization. We will do this by collaborating with specialists in the medical field, specifically cancer, and cellular pathologists.