Next Article in Journal
Semantic Reasoning of Product Biologically Inspired Design Based on BERT
Previous Article in Journal
Petri Net Toolbox for Multi-Robot Planning under Uncertainty
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Applied Sciences
Volume 11
Issue 24
10.3390/app112412086
applsci-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Detection of Pitt–Hopkins Syndrome Based on Morphological Facial Features

by
Elena D’Amato
1,
Constantino Carlos Reyes-Aldasoro
1,
Arianna Consiglio
2,
Gabriele D’Amato
3,
Maria Felicia Faienza
4,*,† and
Marcella Zollino
5,6,†
1
School of Mathematics, Computer Science and Engineering City, University of London, London EC1V 0HB, UK
2
Institute for Biomedical Technologies, National Research Council of Italy, 70126 Bari, Italy
3
Neonatal Intensive Care Unit, Department of Women’s and Children’s Health, ASL Bari, “Di Venere” Hospital, 70131 Bari, Italy
4
Paediatric Unit, Department of Biomedical Sciences and Human Oncology, University of Bari “A. Moro”, 70126 Bari, Italy
5
Dipartimento Universitario Scienze della Vita e Sanità Pubblica, Sezione di Medicina Genomica, Università Cattolica del Sacro Cuore, Facoltà di Medicina e Chirurgia, 00168 Rome, Italy
6
Medicina Genomica, Policlinico Universitario A. Gemelli, IRCSS, 00168 Rome, Italy
*
Author to whom correspondence should be addressed.
†
These authors contributed equally to this work.
Appl. Sci. 2021, 11(24), 12086; https://doi.org/10.3390/app112412086
Submission received: 23 October 2021 / Revised: 12 December 2021 / Accepted: 14 December 2021 / Published: 18 December 2021
(This article belongs to the Topic Machine and Deep Learning)
Browse Figures
Versions Notes

Abstract

:
This work describes a non-invasive, automated software framework to discriminate between individuals with a genetic disorder, Pitt–Hopkins syndrome (PTHS), and healthy individuals through the identification of morphological facial features. The input data consist of frontal facial photographs in which faces are located using histograms of oriented gradients feature descriptors. Pre-processing steps include color normalization and enhancement, scaling down, rotation, and cropping of pictures to produce a series of images of faces with consistent dimensions. Sixty-eight facial landmarks are automatically located on each face through a cascade of regression functions learnt via gradient boosting to estimate the shape from an initial approximation. The intensities of a sparse set of pixels indexed relative to this initial estimate are used to determine the landmarks. A set of carefully selected geometric features, for example, the relative width of the mouth or angle of the nose, is extracted from the landmarks. The features are used to investigate the statistical differences between the two populations of PTHS and healthy controls. The methodology was tested on 71 individuals with PTHS and 55 healthy controls. The software was able to classify individuals with an accuracy rate of 91%, while pediatricians achieved a recognition rate of 74%. Two geometric features related to the nose and mouth showed significant statistical difference between the two populations.

1. Introduction

Pitt–Hopkins syndrome (PTHS) is a rare neurodevelopmental disorder determined by haploinsufficiency of the TCF4 gene on chromosome 18, as a result of deletion or intragenic mutations [1,2]. This syndrome was first identified in 1978 by David Pitt and Ian Hopkins at the Royal Children’s Hospital in Melbourne Australia [3,4]. The subjects affected by PTHS are characterized by an intellectual disability associated with developmental delay, uncommon patterns of breathing, such as breath-holding or hyperventilation, epilepsy, and distinctive facial features. Some of the facial characteristics include sunken eyes, a squared forehead, a larger than normal nose with a broad nasal bridge, a nasal tip that is arched with flaring nostrils, an evident curved shape of the upper lip (also known as Cupid’s bow), and wide spaces between teeth [2,5,6].
Diagnosis of PTHS typically starts on a clinical consultation in which facial signs and other factors, such as absent speech, EEG abnormalities or breathing abnormalities, are evaluated [7,8,9]. In some cases, the diagnosis is confirmed by genetic testing, looking for either deletions of the chromosome region in which the protein transcription factor 4 (TCF4) is located or the identification of a heterozygous variant of TCF4 [10]. Diagnosis of syndromes such as PTHS with genetic tests can be expensive, time consuming, and not available to some communities [11], and thus, not always performed when diagnosing. In a comparative study, only around 62% of diagnoses included genetic testing [7]. Thus, the identification of specific, quantifiable facial metrics extracted from photographs, which can be related to genotype and phenotype would be attractive as a pre-diagnosis step for individuals with PTHS, especially in those cases where the facial dysmorphism is not quite evident.
The recognition of a genetic syndrome based on a photograph is rather similar to the problem of classic facial recognition [12]. However, genetic syndrome detection is limited by the amount of data available and the imbalance against controls [13], the difficulty in differentiating between subtle facial patterns, gender, and ethnic differences [14], among other factors [15]. Facial characteristics have been correlated with other conditions, such as diabetes mellitus based on texture [16], block colors [17] and chromatic features [18].
Furthermore, the limited amount of data restricts the possibility of developing solutions based on deep learning approaches [15,19,20,21] as these typically require very large amounts of training labelled data. In addition, besides the requirement of training data and significant computational resources, deep learning and in general Artificial Intelligence (AT) have sometimes been criticized for being a “black box” [22,23] which requires significant work to make them interpretable and/or explainable. The research areas of Explainable AI [24,25,26] and Interpretable AI [27,28] are fast growing as a way to provide tractable explanations to humans. This is particularly important when clinical decisions are taken based on these complex computational methodologies. Thus, simpler methodologies that are immediately understood and related to facial characteristics [29] by clinicians remain attractive.
This paper describes a methodology that process frontal facial photographs. Whilst the photographs are not constrained, these are always frontal and not grabbed from videos “in the wild” [30,31], which is a much harder problem. The methodology extracts morphological measurements from geometric features through a series of image processing steps. For anonymity purposes, the faces of the document were obscured, and only the edges were retained to illustrate the process. The actual processing was applied to the photographs. These measurements showed statistical difference between individuals with PTHS and healthy controls. The methodology does not require user intervention nor the use of a large database. Although the methodology was tested with a relatively small database, the results are encouraging.

2. Materials and Methods

2.1. Database

A total of 126 photographs of Caucasian individuals aged between 2 years and 17 years divided in two groups were analyzed. The first group consisted of 71 cases of patients with a confirmed diagnosis of PTHS, which were selected from the Institute of Genomic Medicine, Gemelli Hospital Foundation, Rome, Italy. The second group consisted of 55 healthy controls acquired at the Department of Biomedical Science and Human Oncology, University of Bari, Italy. The database used in the study is not publicly available. The permission to use it for the study was obtained through written informed consent from the children’s parents or their legal guardians.
All the procedures were performed in agreement with the guidelines of the Helsinki Declaration on Human Experimentation (ethics approval: Institutional review board of DIPUSVSP, Protocol number 26-06-2147, Università Cattolica Sacro Cuore, Roma, Italy).
The photographs were acquired with a range of devices, mostly amateur cameras and smartphones, with variations of lighting, pose, distance from camera, background, orientation of device and resolution. The format for all of them was 8-bit RGB JPEG. The photographs used for our methodology were front or near-front facial images without occlusions or partial view of the face. Unfortunately, 34 photographs were excluded because of incorrect posture or blurred image. The best images were selected for the classification step of our methodology (61 images: 32 PTHS and 29 healthy), and other 31 acceptable images were selected for final validation of the classification step (6 PTHS and 25 healthy) for a total of 92 images.

2.2. Methodology

All processing was performed using Python 2.7 and the following libraries: sys, os, glob, Dlib, NumPy, Scikit-image and Scikit-learn [32,33,34,35,36].
The proposed pipeline consists of six stages, depicted in Figure 1:
(1) Pre-processing: Images were normalized using automatic histogram equalization and median filtering to obtain uniform brightness and contrast levels. The location of the face was extended by 30 pixels in all directions, and all pixels outside this region were discarded (Figure 2a). Images were re-sized to 600 × 600 pixels.
(2) Face detection: This step was performed using Histogram of Oriented Gradients (HOG) feature detectors [37,38]. The HOG face detector assumes that the shape and texture of a face can be described by the distribution of gradients of intensity or edge directions. As the first step of the HOG face detector, the gradient vector (magnitude and angle) of every pixel of the picture is computed. The final HOG descriptor size is the vector of all components of the normalized blocks in the detection window. For this implementation, the size of the detection window was 64 × 64 pixels, and 1400 histograms or training vectors were generated.
The combined vectors are the input of a linear kernel support vector machine (SVM) [39] classifier trained for face detection on public image datasets as described in previous works [37,38]. The classifiers returned the bounding box of the face.
(3) Face alignment and registration: In the face alignment step, the face is rotated, scaled, and translated considering the rigid and non-rigid face deformations. For this task, 5-point facial landmarks were used: 2 points for the corners of the left eye, 2 points for the corners of the right eye and 1 point for the tip of the nose. The horizontal and vertical (x, y) coordinates of the five points were detected with the pre-trained model based on the HOG descriptor and SVM classifier applied for face detection. First, the center of each eye (centroid) was calculated in accordance with the two corner points detected. Then, taking into account the differences in x and y coordinates of the centroids of both eyes and the arc-tangent function, the angle of rotation was calculated. Once the face is rotated, it is scaled back to the dimension of 600 × 600 pixels.
(4) Landmark localization: The localization of landmarks follows the method of Kazemi [40], which is able to precisely localize 68 landmarks. The implementation of this method, available in Dlib library, uses a training data set that consists of labeled facial landmarks on face images and the probability of distance between pairs of pixels near the facial landmarks. An ensemble of regression trees is trained to estimate the facial landmark positions using only pixel intensities. The configuration of 68 landmarks distributed around the eyes, nose, mouth, and contour of the face was selected and is illustrated in Figure 2b. The landmarks overlaid on the original facial image are shown in Figure 3a.
(5) Geometrical feature extraction: Patients with PTHS present special facial features related to the geometry of the face. Therefore, geometric features, specifically of distance, area and angles, as defined by the relationship among the landmarks previously located, were explored to support diagnosis of PTHS. Three distance features, R1, R2, and R3, were defined by the distances between the eyes (landmarks 39 and 40, in Figure 3b), width of the nose (landmarks 31 and 35) and width of the mouth (landmarks 48 and 54). These features were normalized by the respective baselines B1, B2, and B3 (the width of the face in three different locations) so that they are invariant to scale, rotation, and translation. For the first feature R1, the baseline B1 is the distance between the temples, for R2 the baseline B2 is the distance between the cheekbones, and for R3 the baseline B3 is the width of the jaw (Figure 3b). One angle feature called NoseAngle was extracted to describe the width of the noses of individuals in terms of angular extension (noted as alpha in Figure 3c). Starting from a triangle built using three landmarks of nose as apices, the Carnot’s theorem was used to calculate the top angle of the triangle.
Two area features were derived using polygons or circles defined on facial landmarks. First, the ratio of nose area over face area (RNose) and second, the ratio of mouth area over face area (RMouth) as illustrated in Figure 3d.
The area of the face was calculated as the area of the ellipse constructed based on the two axes, where the minor axis is the face width (landmark 1 to 15), and major axis is the height of the face (landmark 8, the chin, and centroid between landmarks 19 and 24) (Figure 2b).
(6) Classification: The classification module is the last stage of the proposed system. In this phase, the feature vector of the previous steps is the input of a classifier in order to discriminate between syndromic and healthy facial images.
Three different classifiers were chosen: support vector machines (SVMs) [39], random forest (RF) [41], and k-nearest neighbor (k-NN) [42]. The classifiers’ performance was evaluated using the K-fold cross validation strategy with K = 10. Since our dataset is unbalanced for the two populations (PTHS and healthy), in addition to the standard accuracy value (the proportion of correctly classified test samples), the weighted arithmetic mean of the F-score for both classes was chosen as the performance metric, which is the harmonic mean of precision and recall, and it takes into account true negative values and not only positive classes.

2.3. Statistical Analysis

In order to perform the classification step, six geometrical facial features (R1, R2, R3, RNose, RMouth, and NoseAngle) were used as input to the classifier algorithm. Prior to training an automatic classifier, a statistical comparative analysis of the extracted features was performed.
Particularly, the relationship between the syndrome and the extracted geometrical features was investigated, computing the Pearson correlation coefficient r and the p-value (two-tailed) of geometrical features of the two groups.
Furthermore, since the classification is a two-class problem, where each individual can be classified either as PTHS or healthy, Student’s t-test was used to evaluate whether the value of each feature for class PTHS is significantly different from the value of the same feature for the healthy class.

3. Results

An evaluation test of the proposed system was carried out on the collected database composed of 61 individuals (29 healthy and 32 PTHS patients). The evaluation aims at demonstrating the ability of the system to recognize facial features associated with PTHS and discriminate healthy and syndromic individuals with a high accuracy rate. The limited number of input files allowed a manual inspection of the data processing in the pipeline and a visual check of the performance in the image recognition steps. The system was able to run well in the first step of the proposed pipeline: acquisition and pre-processing. All images of the dataset were correctly acquired from a folder of the hard disk and they were pre-processed as expected. In the second step, the system also worked well. The face present in each picture of the database was detected and the bounding box correctly returned. Face alignment and registration is the third step of proposed pipeline. Furthermore, for this step, the system correctly executed the alignment of the face using the five landmarks approach based on the four corners of the eyes and on the nose tips. As the output of this step, all the face images detected were aligned, registered and finally scaled at 600 × 600 pixels. In the next step of the proposed pipeline, landmarks localization was implemented.
All the facial images were correctly described and contoured by landmarks. None of the face images of the collected dataset showed incorrectly localized landmarks. The main differences between PTHS patients and healthy controls were represented by the face features around the nose, the mouth and the face contours as is shown in Figure 4. The figure shows the mean face shape produced using the mean value of each facial landmark for both populations.
In order to perform the classification, six geometrical facial features (R1, R2, R3, RNose, RMouth, and NoseAngle) were used.
Support vector machines (SVMs), random forest (RF), and the k-nearest neighbor (k-NN) were applied on the dataset composed of 61 individuals, and evaluated by using 10-fold cross validation and computing weighted arithmetic average accuracy and F-score. The best classifier was k-NN, with an accuracy of 91.8% and an F-score of 0.918. SVM obtained an accuracy of 90% and an F-score of 0.9, and RF an accuracy of 87.19% and an F-score of 0.878.
Moreover, to prove the robustness of the proposed system, a comparison with human performance was performed. All 61 images of the dataset were shown to a pediatrician. The pediatrician assigned to each facial image a label: PTHS or not. The procedure determined the recognition rate of a human. The accuracy rate was 74% and F-score was 0.722. This result confirmed that in some cases, the facial dysmorphism is not quite evident and a computer-aided system would be useful for PTHS pre-diagnosis.
The k-NN classifier was validated on the dataset composed of 31 additional photographs (considered to be of acceptable quality), obtaining an accuracy of 94% and an F-score of 0.94.
In order to perform an additional evaluation of the geometrical features used as classification input, a statistical analysis of the extracted features was performed. A comparison between the two populations for each feature is reported in the tables and figures. The values Min, Q1, Median, Q3 and Max in the tables represent the quartile bounds of the boxplots in the figures. In Table 1, a statistical analysis of the three distance features is summarized. In Figure 5, the set of 183 values relatives to the three distance features is illustrated. Regarding the angle feature, in Table 2, the statistical comparison between PTHS and the control group is reported, and in Figure 6, the comparison in the form of box plot graphs is illustrated. In Table 3, the statistical comparison of the area features for both groups is described, and in Figure 7 the comparative analysis is shown.
Following the statistical comparative analysis on the geometrical features, the relationships between the syndrome and the extracted geometrical features were investigated. The Pearson correlation coefficient matrix of the features is reported in Table 4.
Pearson correlation coefficients r and the p-value (two-tailed) of geometrical features of the two groups for non-correlation testing were computed, and the details are reported in Table 5. Student’s t-test and the p-value were used to evaluate the significance levels of each feature in the classification of PTHS individuals.
The results reported suggest that moderate correlations (0.3 < r < 0.7) are present among almost all the features, and a strong correlation (r > 0.7) is observed between R3 and R3, R3 and RNose. Moreover, RMouth and NoseAngle are the most statistically significant features in the discrimination between the two experimental groups.

4. Discussion

Recent studies show that facial analysis technologies improve the capabilities of expert clinicians in syndrome identification [11]. This work presents a novel facial analysis system for the diagnosis of PTHS. The ability to clinically diagnose PTHS is quite limited as evidenced by the numerous patients described in the literature diagnosed with a different clinical syndrome.
The system proposed in this work exploits machine learning and computer vision technologies and learns facial representation from a dataset of faces. Previous studies have analyzed dysmorphic facial features of genetic syndromes [15], but only one ever focused on PTHS facial images using a dataset of five subjects [11]. Thus, it was not possible to achieve relevant results based on this study only. This is the first work which used a large dataset of children with molecular diagnosis of PTHS thanks to the collaboration with the Institute of Genomic Medicine, Gemelli Hospital Foundation, Rome, Italy. In this work, an automatic system for recognition of PTHS from facial images is proposed. The work focused on several items: picture acquisition and pre-processing, face detection, face registration and alignment, landmarks localization, features extraction, features selection, and classification. These methods were implemented and tested on a real and remarkable dataset of face images containing 61 subjects (32 PTHS patients and 29 healthy controls).
In this work, a model which focuses on the problem of identifying the correct facial phenotype related to PTHS is presented. Methods such as HOG feature descriptors and cascade of regressors and their improvements were applied to face detection and alignment. Face detection and face alignment tasks were performed by the system with excellent results. All faces in the dataset were correctly detected and aligned.
The landmark localization accuracy was dependent on image quality and pose variations. Non-frontal images can cause weak performance in the landmarks localization task. However, the method used and described in previous section works well for this task. The method was able to correctly localize facial landmark in PTHS and healthy controls. Starting from facial landmarks, features extraction and features selection tasks were based on geometric features. The geometric features are associated with specific dysmorphisms of the PTHS, such as the size of nose, and the size of the mouth. Geometrical features extracted in this work are suitable to depict the facial features of PTHS and healthy individuals, as indicated by the excellent accuracy rate reached in the last step of the system, the classification task. This is the first study to the author’s knowledge to investigate the efficiency of a system based on computer vision and machine learning algorithms in order to recognize PTHS subjects from face images. The k-nearest neighbor classifier reached the accuracy of 91.8% and an F-score of 0.918 on the dataset of 61 images. On the same dataset, a pediatrician obtained an accuracy of 74% and F-score of 0.722. The k-NN classifier was also validated with 31 additional photographs, obtaining an accuracy of 94% and an F-score of 0.94. The previous data on a small dataset of five patients reached an accuracy of 80% [11]. A statistical independent evaluation of each geometric feature revealed that RMouth and NoseAngle are the features most significantly correlated with PTHS phenotype.
This work demonstrated the possibility to describe phenotypes in a standardized manner, opening the door to the emerging field of precision medicine, as well as to the identification of new genetic syndromes by matching undiagnosed patients sharing a similar phenotype. Patient matching with such artificial intelligence technology will enhance the way that genetic syndromes and other genetically caused diseases are studied and explored.

5. Conclusions

A system for automated PTHS syndrome detection using frontal facial images of children is proposed in this work. Face detection was performed by a HOG object detector, whereas landmarks localization was based on a state-of-the-art method that used an ensemble of regression trees trained to estimate the facial landmark positions using pixel intensities.
Based on the detected facial landmarks, six geometric features were extracted and selected to discriminate PTHS and healthy individuals. Three classifiers were compared to reach high recognition rates between the two populations. The highest accuracy and F-score were achieved using the k-NN classifier.
On the basis of the results achieved, it can be stated that the system could also be able to identify PTHS in real environments and using face images taken in the wild. The proposed system could be a useful aid in early diagnosis of PTHS syndrome from a simple app for smartphones in an accurate and quick way. As a future development, it could be tested on images of patients with other dysmorphic syndromes, especially those requiring differentiation from PTHS.
In conclusion, it is possible that that the coupling of the phenotype analysis, done by computer vision algorithms, with the continuously growing genomic knowledge, will open new ways to rapidly reach an accurate molecular diagnosis for patients with genetic syndromes, and may become a key-factor in the field of precision medicine.
In future studies, it could be possible to combine the facial analysis described here with genome sequencing data. This will enable improved prioritization of gene-variant results.

Author Contributions

Conceptualization, E.D. and C.C.R.-A.; methodology, E.D.; software, E.D.; validation, E.D., C.C.R.-A. and A.C.; formal analysis, E.D. and C.C.R.-A.; investigation, E.D. and C.C.R.-A.; resources, M.Z.; writing—original draft preparation, E.D.; writing—review and editing, C.C.R.-A., G.D. and M.F.F.; visualization, A.C.; supervision, C.C.R.-A., G.D. and M.F.F. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

All the procedures were performed in agreement with the guidelines of the Helsinki Declaration on Human Experimentation (ethics approval: Institutional review board of DIPUSVSP, Protocol number 26-06-2147, Università Cattolica Sacro Cuore, Roma, Italy.

Informed Consent Statement

The permission to use the images for the study was obtained through written informed consent from the children’s parents or their legal guardians.

Data Availability Statement

The photographs of the children are available on request from the corresponding author for research purposes only.

Conflicts of Interest

The authors have no conflict of interest.

References

  1. Zollino, M.; Zweier, C.; Van Balkom, I.D.; Sweetser, D.A.; Alaimo, J.; Bijlsma, E.K.; Cody, J.; Elsea, S.H.; Giurgea, I.; Macchiaiolo, M.; et al. Diagnosis and management in Pitt-Hopkins syndrome: First international consensus statement. Clin. Genet. 2019, 95, 462–478. [Google Scholar] [CrossRef] [PubMed]
  2. Goodspeed, K.; Newsom, C.; Morris, M.A.; Powell, C.; Evans, P.; Golla, S. Pitt-Hopkins Syndrome: A Review of Current Literature, Clinical Approach, and 23-Patient Case Series. J. Child. Neurol. 2018, 33, 233–244. [Google Scholar] [CrossRef] [PubMed]
  3. Pitt, D.; Hopkins, I. A Syndrome of Mental Retardation, Wide Mouth and Intermittent Overbreathing. Aust. Paediatr. J. 1978, 14, 182–184. [Google Scholar] [CrossRef]
  4. Peippo, M.; Ignatius, J. Pitt-Hopkins Syndrome. Mol. Syndromol. 2012, 2, 171–180. [Google Scholar] [CrossRef]
  5. Amiel, J.; Rio, M.; de Pontual, L.; Redon, R.; Malan, V.; Boddaert, N.; Plouin, P.; Carter, N.P.; Lyonnet, S.; Munnich, A.; et al. Mutations in TCF4, Encoding a Class I Basic Helix-Loop-Helix Transcription Factor, Are Responsible for Pitt-Hopkins Syndrome, a Severe Epileptic Encephalopathy Associated with Autonomic Dysfunction. Am. J. Hum. Genet. 2007, 80, 988–993. [Google Scholar] [CrossRef] [Green Version]
  6. Brockschmidt, A.; Todt, U.; Ryu, S.; Hoischen, A.; Landwehr, C.; Birnbaum, S.; Frenck, W.; Radlwimmer, B.; Lichter, P.; Engels, H.; et al. Severe Mental Retardation with Breathing Abnormalities (Pitt-Hopkins Syndrome) Is Caused by Haploinsufficiency of the Neuronal BHLH Transcription Factor TCF4. Hum. Mol. Genet. 2007, 16, 1488–1494. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  7. de Winter, C.F.; Baas, M.; Bijlsma, E.K.; van Heukelingen, J.; Routledge, S.; Hennekam, R.C.M. Phenotype and Natural History in 101 Individuals with Pitt-Hopkins Syndrome through an Internet Questionnaire System. Orphanet J. Rare Dis. 2016, 11, 37. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  8. Whalen, S.; Héron, D.; Gaillon, T.; Moldovan, O.; Rossi, M.; Devillard, F.; Giuliano, F.; Soares, G.; Mathieu-Dramard, M.; Afenjar, A.; et al. Novel Comprehensive Diagnostic Strategy in Pitt–Hopkins Syndrome: Clinical Score and Further Delineation of the TCF4 Mutational Spectrum. Hum. Mutat. 2012, 33, 64–72. [Google Scholar] [CrossRef]
  9. Van Balkom, I.D.C.; Vuijk, P.J.; Franssens, M.; Hoek, H.W.; Hennekam, R.C.M. Development, Cognition, and Behaviour in Pitt-Hopkins Syndrome. Dev. Med. Child. Neurol. 2012, 54, 925–931. [Google Scholar] [CrossRef] [Green Version]
  10. Sweetser, D.A.; Elsharkawi, I.; Yonker, L.; Steeves, M.; Parkin, K.; Thibert, R. Pitt-Hopkins Syndrome. In GeneReviews®; Adam, M.P., Ardinger, H.H., Pagon, R.A., Wallace, S.E., Bean, L.J., Mirzaa, G., Amemiya, A., Eds.; University of Washington: Seattle, WA, USA, 1993. [Google Scholar]
  11. Kuru, K.; Niranjan, M.; Tunca, Y.; Osvank, E.; Azim, T. Biomedical Visual Data Analysis to Build an Intelligent Diagnostic Decision Support System in Medical Genetics. Artif. Intell. Med. 2014, 62, 105–118. [Google Scholar] [CrossRef]
  12. Samal, A.; Iyengar, P.A. Automatic Recognition and Analysis of Human Faces and Facial Expressions: A Survey. Pattern Recognit. 1992, 25, 65–77. [Google Scholar] [CrossRef]
  13. Schroff, F.; Kalenichenko, D.; Philbin, J. FaceNet: A Unified Embedding for Face Recognition and Clustering. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 815–823. [Google Scholar]
  14. Farnell, D.J.J.; Galloway, J.; Zhurov, A.; Richmond, S.; Perttiniemi, P.; Katic, V. Initial Results of Multilevel Principal Components Analysis of Facial Shape. In Proceedings of the Medical Image Understanding and Analysis, Oxford, UK, 12–14 July 2021; Valdés Hernández, M., González-Castro, V., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 674–685. [Google Scholar]
  15. Gurovich, Y.; Hanani, Y.; Bar, O.; Nadav, G.; Fleischer, N.; Gelbman, D.; Basel-Salmon, L.; Krawitz, P.M.; Kamphausen, S.B.; Zenker, M.; et al. Identifying Facial Phenotypes of Genetic Disorders Using Deep Learning. Nat. Med. 2019, 25, 60–64. [Google Scholar] [CrossRef] [PubMed]
  16. Shu, T.; Zhang, B.; Yan Tang, Y. An Extensive Analysis of Various Texture Feature Extractors to Detect Diabetes Mellitus Using Facial Specific Regions. Comput. Biol. Med. 2017, 83, 69–83. [Google Scholar] [CrossRef]
  17. Zhang, B.; Vijaya kumar, B.V.K.; Zhang, D. Noninvasive Diabetes Mellitus Detection Using Facial Block Color with a Sparse Representation Classifier. IEEE Trans. Biomed. Eng. 2014, 61, 1027–1033. [Google Scholar] [CrossRef] [PubMed]
  18. Wang, X.; Zhang, B.; Guo, Z.; Zhang, D. Facial Image Medical Analysis System Using Quantitative Chromatic Feature. Expert Syst. Appl. 2013, 40, 3738–3746. [Google Scholar] [CrossRef]
  19. Porras, A.R.; Rosenbaum, K.; Tor-Diez, C.; Summar, M.; Linguraru, M.G. Development and Evaluation of a Machine Learning-Based Point-of-Care Screening Tool for Genetic Syndromes in Children: A Multinational Retrospective Study. Lancet Digit. Health 2021, 3, e635–e643. [Google Scholar] [CrossRef]
  20. Meng, T.; Guo, X.; Lian, W.; Deng, K.; Gao, L.; Wang, Z.; Huang, J.; Wang, X.; Long, X.; Xing, B. Identifying Facial Features and Predicting Patients of Acromegaly Using Three-Dimensional Imaging Techniques and Machine Learning. Front. Endocrinol. 2020, 11, 492. [Google Scholar] [CrossRef]
  21. Silwal, R.; Alsadoon, A.; Prasad, P.W.C.; Alsadoon, O.H.; Al-Qaraghuli, A. A Novel Deep Learning System for Facial Feature Extraction by Fusing CNN and MB-LBP and Using Enhanced Loss Function. Multimed. Tools Appl. 2020, 79, 31027–31047. [Google Scholar] [CrossRef]
  22. Castelvecchi, D. Can We Open the Black Box of AI? Nat. News 2016, 538, 20. [Google Scholar] [CrossRef] [Green Version]
  23. Gaur, M.; Faldu, K.; Sheth, A. Semantics of the Black-Box: Can Knowledge Graphs Help Make Deep Learning Systems More Interpretable and Explainable? IEEE Internet Comput. 2021, 25, 51–59. [Google Scholar] [CrossRef]
  24. Tjoa, E.; Guan, C. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI. IEEE Trans. Neural Netw. Learn. Syst. 2021, 32, 4793–4813. [Google Scholar] [CrossRef]
  25. Samek, W.; Müller, K.-R. Towards Explainable Artificial Intelligence. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning; Samek, W., Montavon, G., Vedaldi, A., Hansen, L.K., Müller, K.-R., Eds.; Lecture Notes in Computer Science; Springer International Publishing: Cham, Switzerland, 2019; pp. 5–22. ISBN 978-3-030-28954-6. [Google Scholar]
  26. Holzinger, A. From Machine Learning to Explainable AI. In Proceedings of the 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA), Košice, Slovakia, 23–25 August 2018; pp. 55–66. [Google Scholar]
  27. Ahmad, M.A.; Eckert, C.; Teredesai, A. Interpretable Machine Learning in Healthcare. In Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, Association for Computing Machinery, New York, NY, USA, 15 August 2018; pp. 559–560. [Google Scholar]
  28. Elton, D.C. Self-Explaining AI as an Alternative to Interpretable AI. In International Conference on Artificial General Intelligence; Goertzel, B., Panov, A.I., Potapov, A., Yampolskiy, R., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 95–106. [Google Scholar]
  29. Zhang, B.; Zhao, Q.; Zhang, D. Facial Multi-Characteristics and Applications; World Scientific Publishing Company Pte Limited: Singapore, 2018. [Google Scholar]
  30. Kossaifi, J.; Walecki, R.; Panagakis, Y.; Shen, J.; Schmitt, M.; Ringeval, F.; Han, J.; Pandit, V.; Toisoul, A.; Schuller, B.; et al. SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 43, 1022–1040. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Asthana, A.; Zafeiriou, S.; Tzimiropoulos, G.; Cheng, S.; Pantic, M. From Pixels to Response Maps: Discriminative Image Filtering for Face Alignment in the Wild. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 1312–1320. [Google Scholar] [CrossRef] [Green Version]
  32. King, D.E. Dlib-Ml: A Machine Learning Toolkit. J. Mach. Learn. Res. 2009, 10, 1755–1758. [Google Scholar]
  33. PhD, T.E.O. Guide to NumPy: 2nd Edition, 2nd ed.; CreateSpace Independent Publishing Platform: Scotts Valley, CA, USA, 2015; ISBN 978-1-5173-0007-4. [Google Scholar]
  34. van der Walt, S.; Schönberger, J.L.; Nunez-Iglesias, J.; Boulogne, F.; Warner, J.D.; Yager, N.; Gouillart, E.; Yu, T. Scikit-Image: Image Processing in Python. PeerJ 2014, 2, e453. [Google Scholar] [CrossRef]
  35. Pedregosa, F.; Varoquaux, G.; Gramfort, A.; Michel, V.; Thirion, B.; Grisel, O.; Blondel, M.; Prettenhofer, P.; Weiss, R.; Dubourg, V.; et al. Scikit-Learn: Machine Learning in Python. J. Mach. Learn. Res. 2011, 12, 2825–2830. [Google Scholar]
  36. Viola, P.; Jones, M.J. Robust Real-Time Face Detection. Int. J. Comput. Vis. 2004, 57, 137–154. [Google Scholar] [CrossRef]
  37. Dalal, N.; Triggs, B. Histograms of Oriented Gradients for Human Detection. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Santa Barbara, CA, USA, 20–25 June 2005; Volume 1, pp. 886–893. [Google Scholar]
  38. Déniz, O.; Bueno, G.; Salido, J.; De la Torre, F. Face Recognition Using Histograms of Oriented Gradients. Pattern Recognit. Lett. 2011, 32, 1598–1603. [Google Scholar] [CrossRef]
  39. Cortes, C.; Vapnik, V. Support-Vector Networks. Mach. Learn. 1995, 20, 273–297. [Google Scholar] [CrossRef]
  40. Kazemi, V.; Sullivan, J. One Millisecond Face Alignment with an Ensemble of Regression Trees. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23–28 June 2014; pp. 1867–1874. [Google Scholar]
  41. Breiman, L. Random Forests, Machine Learning. Mach. Learn. 2001, 45, 5–32. [Google Scholar] [CrossRef] [Green Version]
  42. Cover, T.M.; Hart, P.E. Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 1967, 13, 21–27. [Google Scholar] [CrossRef]
Applsci 11 12086 g001 550
Figure 1. Pipeline of the proposed system.
Figure 1. Pipeline of the proposed system.
Applsci 11 12086 g001
Applsci 11 12086 g002 550
Figure 2. (a). One representative photograph of an individual with diagnosed PTHS. The face was obscured for anonymity purposes. Canny edge detection was applied to the blue channel, the regions without edges were set to black. Red bounding box denotes the region of interest that will be cropped. (b) Sixty-eight facial landmarks to be localized on the photographs.
Figure 2. (a). One representative photograph of an individual with diagnosed PTHS. The face was obscured for anonymity purposes. Canny edge detection was applied to the blue channel, the regions without edges were set to black. Red bounding box denotes the region of interest that will be cropped. (b) Sixty-eight facial landmarks to be localized on the photographs.
Applsci 11 12086 g002
Applsci 11 12086 g003 550
Figure 3. Illustration of the methodology and a subset of geometrical features. (a) Region of interest with the localized facial landmarks. (b) Three distance features illustrated with red lines: R1 (distance between eyes), R2 (distance between edges of the nostrils), and R3 (distance between the corners of the mouth). Three baselines: B1 (temples), B2 (cheekbones), and B3 (jaw). (c) Triangular region corresponding to the nose and the angle alpha. (d) Ellipsoidal region corresponding to the mouth.
Figure 3. Illustration of the methodology and a subset of geometrical features. (a) Region of interest with the localized facial landmarks. (b) Three distance features illustrated with red lines: R1 (distance between eyes), R2 (distance between edges of the nostrils), and R3 (distance between the corners of the mouth). Three baselines: B1 (temples), B2 (cheekbones), and B3 (jaw). (c) Triangular region corresponding to the nose and the angle alpha. (d) Ellipsoidal region corresponding to the mouth.
Applsci 11 12086 g003
Applsci 11 12086 g004 550
Figure 4. Mean face shape comparison between PTH syndrome and healthy individuals.
Figure 4. Mean face shape comparison between PTH syndrome and healthy individuals.
Applsci 11 12086 g004
Applsci 11 12086 g005 550
Figure 5. Box plots of distance features for both groups (PTHS and control). R1-PTHS = Group 1; R1-Control = Group 2; R2-PTHS = Group 3; R2-Control = Group 4; R3-PTHS = Group 5; R3-Control = Group 6.
Figure 5. Box plots of distance features for both groups (PTHS and control). R1-PTHS = Group 1; R1-Control = Group 2; R2-PTHS = Group 3; R2-Control = Group 4; R3-PTHS = Group 5; R3-Control = Group 6.
Applsci 11 12086 g005
Applsci 11 12086 g006 550
Figure 6. Box plots of angle features for both groups (PTHS and control). NoseAngle-PTHS = Group 1; NoseAngle-Control = Group 2.
Figure 6. Box plots of angle features for both groups (PTHS and control). NoseAngle-PTHS = Group 1; NoseAngle-Control = Group 2.
Applsci 11 12086 g006
Applsci 11 12086 g007 550
Figure 7. Box plots of area features for both groups (PTHS and control). RNose-PTHS = Group 1; RNose-Control = Group 2; RMouth-PTHS = Group 3; RMouth-Control = Group 4.
Figure 7. Box plots of area features for both groups (PTHS and control). RNose-PTHS = Group 1; RNose-Control = Group 2; RMouth-PTHS = Group 3; RMouth-Control = Group 4.
Applsci 11 12086 g007
Table
Table 1. Statistical comparison of distance features R1 (distances between the eyes), R2 (width of the nose), R3 (width of the mouth) for both groups (PTHS and controls).
Table 1. Statistical comparison of distance features R1 (distances between the eyes), R2 (width of the nose), R3 (width of the mouth) for both groups (PTHS and controls).
FeaturePopulationNMinQ1MedianQ3MaxMeanSD
R1 (Group 1)PTHS3233.433.683.834.413.670.25
R1 (Group 2)Control2933.583.663.854.233.720.26
R2 (Group 3)PTHS3244.284.454.756.234.530.47
R2 (Group 4)Control2944.354.564.845.314.570.36
R3 (Group 5)PTHS3221.921.972.192.762.060.26
R3 (Group 6)Control2921.992.172.272.522.140.21
Table
Table 2. Statistical comparison of angle features of the nose for both groups (PTHS and control).
Table 2. Statistical comparison of angle features of the nose for both groups (PTHS and control).
FeaturePopulationNMinQ1MedianQ3MaxMeanSD
NoseAngle (Group 1)PTHS323036.6938.2040.1345.7238.543.33
NoseAngle (Group 2)Control292931.6132.0834.1740.1232.782.63
Table
Table 3. Statistical comparison of area features (the ratio of nose area over face area (RNose) and second, the ratio of mouth area over face area (RMouth)) for both groups (PTHS and control).
Table 3. Statistical comparison of area features (the ratio of nose area over face area (RNose) and second, the ratio of mouth area over face area (RMouth)) for both groups (PTHS and control).
FeaturePopulationNMinQ1MedianQ3MaxMeanSD
RNose
(Group 1)		PTHS32152122243022.413.35
RNose
(Group 2)		Control29212325263024.792.13
RMouth
(Group 3)		PTHS3281112.51524144.47
RMouth
(Group 4)		Control2989910119.380.78
Table
Table 4. Pearson correlation coefficient matrix of the geometrical features.
Table 4. Pearson correlation coefficient matrix of the geometrical features.
R1R2R3NoseAngleRNoseRMouthC
R11.0000000.6318560.565448−0.1235010.4486900.234808−0.099263
R20.6318561.0000000.767697−0.1847650.7585480.277656−0.048716
R30.5654480.7676971.000000−0.1313570.6399320.438070−0.171853
RNose−0.123501−0.184765−0.1313571.0000000.329132−0.6019340.708482
RMouth0.4486900.7585480.6399320.3291321.000000−0.1507400.234934
NoseAngle0.2348080.2776560.438070−0.601934−0.1507401.000000−0.675969
C−0.099263−0.048716−0.1718530.7084820.234934−0.6759691.000000
The class to predict PTHS is C.
Table
Table 5. Pearson correlation coefficient matrix of the geometrical features between the two groups and Student’s t-test of each feature in classification of PTHS individuals.
Table 5. Pearson correlation coefficient matrix of the geometrical features between the two groups and Student’s t-test of each feature in classification of PTHS individuals.
FeaturePearson (r)t-Testp-ValueFDRSignificance
R10.09926−0.766240.446590.53591
R20.04872−0.374640.709270.70927
R3−0.17185−1.339970.185400.27809
RNose0.23493−1.856530.034190.06837
RMouth−0.57054−7.08696<0.00001<0.0001***
NoseAngle0.708487.44448<0.00001<0.0001***
Significance levels: *** for p < 0.001.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

D’Amato, E.; Reyes-Aldasoro, C.C.; Consiglio, A.; D’Amato, G.; Faienza, M.F.; Zollino, M. Detection of Pitt–Hopkins Syndrome Based on Morphological Facial Features. Appl. Sci. 2021, 11, 12086. https://doi.org/10.3390/app112412086

AMA Style

D’Amato E, Reyes-Aldasoro CC, Consiglio A, D’Amato G, Faienza MF, Zollino M. Detection of Pitt–Hopkins Syndrome Based on Morphological Facial Features. Applied Sciences. 2021; 11(24):12086. https://doi.org/10.3390/app112412086

Chicago/Turabian Style

D’Amato, Elena, Constantino Carlos Reyes-Aldasoro, Arianna Consiglio, Gabriele D’Amato, Maria Felicia Faienza, and Marcella Zollino. 2021. "Detection of Pitt–Hopkins Syndrome Based on Morphological Facial Features" Applied Sciences 11, no. 24: 12086. https://doi.org/10.3390/app112412086

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop