Tracking Long-Term Temperature Anomalies with Person Identification Using Thermal Cameras: An Initial Step towards Disease Recognition ^†

Abstract

An outbreak of infectious diseases has highlighted the importance of the early detection and prevention of high temperatures or fevers, which are some of the main symptoms of many diseases. Thermal cameras have become a promising tool for the detection of fever due to their non-invasive, non-contact and rapidly inspecting nature. By using person identification to analyze temperature data from the corner of the eye (inner canthus), the temperature of individuals can be tracked over time. This could provide information not only about their current temperature but also about long-term temperature anomalies, and therefore can urge people to visit a doctor. This paper is intended as an initial study for the long-term temperature measurement of individuals. The preliminary results show the feasibility of such approach. In the future, a similar procedure is to be used for the detection and recognition of individual diseases.

1. Introduction

In recent years, the outbreak of infectious diseases has posed a significant threat to global health. The early detection and prevention of such diseases is crucial for effective management and control. One of the symptoms of many infectious diseases is an increased temperature or fever, making it a useful indicator for disease detection. However, mass screening is a challenging problem. Thermographic cameras have the potential to be widely used for such purposes as they can quickly measure temperature without physical contact, thereby reducing the risk of cross-infection. Mainly due to the occurrence of the COVID-19 pandemic, extensive research has been carried out in this area, e.g., [1,2,3].

By using person identification (the recognition of individual people) to analyze temperature data, individuals’ temperatures can be tracked over time. This approach provides information not only on their current temperature but also on long-term temperature anomalies, which could help detect the early onset of a disease and alert individuals to visit a doctor.

In addition to detecting infectious diseases, long-term temperature measurement could also aid the early recognition of other diseases.

The idea is to provide an infrared camera, e.g., in a workplace or, in the future, at home, which would take facial measurements. The camera would measure users entering and leaving work (/home) every day. In this way, the system would know the normal temperature profile of the user’s face. If any anomaly appeared, the user would be alerted and advised of an appropriate action. Such an action could be, for example, a recommendation to see a doctor or go home when a fever occurs, i.e., when local minor changes (e.g., warmer spots around the lips, eyes, nose) occur, alerting the user to monitor themself and possibly visit a doctor. In the future, this system is intended to be able to recognize individual diseases, ideally before there are any significant health difficulties. This would enable early treatment and limit the impact of diseases on the health of individuals, thereby increasing the quality and length of individuals’ lives.

The system should consist of several parts. The infrared camera should fulfil certain technical parameters (more can be found in [2]). The facial region should be recognized. Individual people should be recognized (as we need to automatically determine which person was measured). Anomalies in the face map should be detected. And, finally, based on those anomalies, individual diseases can be recognized.

In this paper, only an initial evaluation of the data is shown. An example of a benefit which can be brought by person identification and long-term measurement is presented.

2. Materials and Methods

The study was focused on initial data collection and principal verification for the long-term monitoring of the facial temperature of individuals. Infrared cameras were installed at several locations (our workplace, a high school, and a town municipality). This paper deals only with the data from our workplace. The data were collected with an IR Face Scanner (an infrared camera created at our workspace based on a FLIR Lepton 3.5 module). The IR camera had the following parameters: wavelength range 8–14 μm; resolution 160 × 120 pixels; NETD < 50 mK. The device was equipped with an internal blackbody within its field of view to enhance measurement accuracy. The blackbody had a temperature of 37 °C (98.6 °F). This temperature was chosen because it is commonly regarded as an elevated body temperature and one of the goals was to indicate increased temperature.

2.1. Face Detection

In the initial phase, measurements were taken by having the person stand on a specified point. After collecting images, a supervised AI model for face, eye and glasses detection was created. The thermograms were manually annotated. The individual thermograms were transformed to greyscale images within a fixed temperature range from 12 to 42 °C. Network SqueezeNet-SSD [4] was used and trained using Caffe framework [5]. This network and framework were chosen because the detection must be performed in real time.

After this step, our measurement apparatus software was updated so that it recognized faces, eyes, and glasses.

In the second phase, individuals stepped in front of the IR camera. Their image was taken automatically. Additionally, the apparatus alerted the user whenever an issue arose (e.g., if a person is wearing glasses, and thus their temperature cannot be measured, it leads to a message asking them to please remove the glasses). This way, more relevant data were obtained.

2.2. Person Recognition

To reach a good scalability, the person identification should be performed in an unsupervised manner. Therefore, clustering was the method of choice. Nonetheless, clustering was not performed on whole images; instead, specific features were extracted from thermograms. Two approaches were compared. The first one is based on a histogram of oriented gradients (HOG). This is a general feature descriptor technique commonly used in computer vision. It works by computing and analyzing the distribution of gradient orientations in an image, providing a representation that captures local shape and texture information crucial for identifying individuals. The second one is a facial feature extraction method. This is a method developed for feature extraction from facial data used for VIS images. The solution was presented in the paper [6]. This method can also be essential for better understanding temperature changes in different facial regions for the detection of different diseases. Clustering itself for both feature extraction methods was done by density-based spatial clustering (DBSCAN).

An example of a facial thermogram, its HOG transformation and facial landmark detection is in Figure 1.

3. Results and Discussion

The method using facial landmarks was not able to distinguish key features in a significant amount of thermograms. It was able to detect the landmarks in only 22.77% of all cases. This is likely due to the lower resolution and quality compared to the training data in [6]. In this condition, clustering using facial landmarks was not suitable for the intended purpose. Nonetheless, it can be used for more complex analyses of a person’s temperature for certain images. For example, it can suggest an eye problem when one eye has a higher temperature than the other.

The clustering was performed using HOG features. Many thermograms were assigned to an unrecognized cluster. To be able to classify all images, Support Vector Machines were used for classification (the clusters were used for the SVM training). In total, 83% of the validation data were recognized correctly (if we exclude those which were assigned to an unknown cluster). This is quite a good result, considering that a relatively simple approach was used. Furthermore, if the requirements for prediction certainty are tightened, it would be possible to detect fewer frames, but be almost always correct.

The example in Figure 2 shows the long-term monitoring of one individual (September 2021–February 2023). Higher temperatures (maximum canthus temperature) were visible during summer months, while lower temperatures were observed during winter months. This is an expected result due the fact that individuals were measured immediately after they entered the building. However, the magnitude of the differences in the winter and summer months was surprising. This can pose a significant challenge for the detection of fever. It should be pointed out that the measurement was performed this way because one of the project’s objectives was to investigate the impact of outside temperature on the measurement. The person’s temperature measurement should be performed inside after about 20 min in standard room conditions. The results show a large variation of values due to significant changes in outside temperature. This can be an issue for detecting fever, but it may be beneficial for detecting some diseases; e.g., if parts of the face heats up/cools down at different rates.

4. Conclusions

This study aimed to investigate the feasibility of long-term temperature measurement of individuals using thermal cameras. The results showed that the method relying solely on facial landmarks was not effective in distinguishing key features in a significant number of thermograms, likely due to lower resolution and quality compared to the training data. However, it has the potential for more complex temperature analyses and detecting specific abnormalities.

The example of long-term monitoring revealed variations in canthus temperature throughout the year, with higher temperatures observed in summer months and lower temperatures in winter months. This pattern was attributed to the immediate measurement upon entering the building, with outside temperature influencing the results.

Considering the limitations and insights gained from this initial study, it is suggested that future research focuses on incorporating visible spectrum (VIS) cameras alongside thermal cameras to take advantage of their higher resolution and advancements in face identification algorithms. By combining the strengths of both spectra, more accurate and reliable temperature analysis, disease detection, and person recognition can be achieved.