Next Article in Journal
The Effect of Nitrogen Annealing on the Resistive Switching Characteristics of the W/TiO2/FTO Memory Device
Previous Article in Journal
Stochastic Modeling of Smartphones GNSS Observations Using LS-VCE and Application to Samsung S20
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Infrared Camera Geometric Calibration: A Review and a Precise Thermal Radiation Checkerboard Target

1
Department of Mechanical Engineering, American University of Sharjah, Sharjah 26666, United Arab Emirates
2
School of Engineering, Newcastle University, Newcastle upon Tyne NE1 7RU, UK
*
Author to whom correspondence should be addressed.
Sensors 2023, 23(7), 3479; https://doi.org/10.3390/s23073479
Submission received: 17 February 2023 / Revised: 18 March 2023 / Accepted: 23 March 2023 / Published: 26 March 2023
(This article belongs to the Section Optical Sensors)

Abstract

:
Different infrared (IR) planar geometric calibration targets have been developed over the years that exploit a well-established and flexible optical camera geometric calibration procedure following the pinhole approximation. This geometric calibration is typically neglected in IR cameras, due to the relatively low resolution of thermal images and the complex IR targets needed for the geometric calibration in comparison to the optical targets. In this study, a thorough literature review of numerous IR camera geometric calibration targets, along with their respective outcomes, were summarized and leveraged to deliver a practical checkerboard target for less experienced end users, while offering the lowest reprojection errors. It was concluded that the fabrication of high emissivity contrast and precise square points of intersection within a checkerboard pattern extends the accuracy of capturing these control points in a thermal image for an optimized IR camera geometric calibration. Accordingly, two simple planar checkerboard targets were fabricated using laser engraving and ultraviolet (UV) printing technologies on a polished stainless steel (SS304) plate. The UV-printed checkerboard target on a polished metallic alloy delivered the lowest mean reprojection error ( MRE ) of 0.057 pixels and the lowest root mean square error ( RMSE ) of reprojection of 0.063 pixels, with a standard deviation lower than 0.003 pixels. The UV-printed design offers better accuracy than any other checkerboard calibration target, and comparable results to the best prominent circular pattern results reported in the literature.

1. Introduction

Infrared thermography (IRT) is the practice of capturing thermal images using infrared (IR) cameras by measuring the infrared radiation emitted by objects. IR radiation is constantly emitted from objects at temperatures above absolute zero, where the intensity and wavelength of IR radiation mainly depend on the object’s surface temperature [1,2,3]. To guarantee the accurate temperature recording and measurement of the object’s surface, thermographic or IR sensors require proper thermal calibration [4]. The thermal calibration of IR cameras is typically based on blackbody simulators [1] or thermocouple systems [5], which produce controlled thermal conditions to calibrate and ensure accurate thermal recordings by the IR camera. Apart from providing information on the thermal condition of a particular scene or object, IR cameras bear the feature of providing geometrical information on the object being imaged [6]. The geometrical information could be utilized to size objects in a particular scene, measure distances in real-world coordinates, or in image reconstruction by aligning different images of the same scene [6]. Nondestructive testing and evaluation (NDT&E) of material, involving the use of IR cameras for crack size estimation using vibrothermography [7,8,9,10] or eddy current thermography [11,12] and thermal diffusivity estimation in metallic components [13], requires accuracy in the thermal mapping to the world coordinate geometry within the IR camera field of view (FOV). To accurately retrieve geometrical information of imaged objects, distortions introduced by the natural curvature of the IR camera’s lens should be removed through geometrical calibration. However, geometrical information is typically neglected in IRT due to two main reasons, namely, IR cameras provide relatively low-resolution images, and a complex geometrical calibration procedure is needed to extract geometrical information when compared to visible cameras [6]. Thus, IRT applications generally involve the thermal imaging and the analysis of a particular body, avoiding any geometrical information [14,15].
Geometric calibration is the process of imaging a calibration target with geometrically known features to determine the intrinsic and extrinsic parameters of a camera. Intrinsic parameters are values of the interior orientation elements of a camera, such as the principal points in image coordinate systems, focal length, skewness and, most importantly the radial and tangential distortions caused by the lens [16,17,18]. Radial and tangential distortions caused by the camera’s lens disfigure the captured images by introducing barrel or pincushion effects [19], as displayed in Figure 1. Extrinsic parameters, however, allow the transformation of a 3D geometrical scene to the camera’s 3D coordinate frame. This permits the representation of the spatial location of the camera in a 3D scene, allowing for measurements in world coordinates [6]. Thus, geometric calibration is crucial in photogrammetry to determine the relationship between 3D world coordinate points and 2D image coordinate points on the detector level of the camera [19]. Visible camera calibration is usually performed using the widely known checkerboard pattern that is printed using inexpensive off-the-shelf printers [20,21,22,23]. IR cameras do not display the visible color of an imaged object, since visible and IR cameras operate at different spectrum bands. Thus, directly employing the typical paper-printed checkerboard pattern used to calibrate visible cameras to IR cameras is not suitable, as the features of the checkerboard pattern are invisible at normal temperature conditions [24]. This causes the contrast produced by this pattern in thermal images to appear poor or dim [25]. Therefore, through varying materials, surface emissivity and thermal conductivities, calibration targets will have distinctive features in the IR spectrum, allowing for the geometric calibration of IR cameras [26].

2. Literature Review

Thermal camera geometric calibration targets are classified into two categories, namely passive and active calibration targets. Passive targets mainly rely on the difference in emissivity values of the pattern, and may employ an external heater to allow the pattern to be visible in the IR spectrum. External heaters could be sunlight, a flood lamp, or an electric fan heater that is independent of the calibration target. Contrarily, active targets rely on both the emissivity difference and the use of a built-in heater to magnify the contrast in a thermal image. This complicates the fabrication of the calibration target, as active targets require electrical work in addition to the selected pattern. Being an active research field, numerous developments have been presented in the literature, with different techniques and calibration targets to geometrically calibrate IR cameras.
One of the most popular approaches involves the typical ink/laser paper-printed square checkerboard pattern used to calibrate visible cameras. However, the pattern is heated using a flood lamp, allowing the pattern to be visible in the IR spectrum [27,28]. A similar study involved the use of a flood lamp to heat the square paper-printed checkerboard target, but here the target was placed on a ceramic tile to retain heating for a longer duration [29]. Similarly, a study geometrically calibrated an IR camera using the checkerboard pattern printed on matt paper and placed on a wall to ensure its flatness, while an IR-emitting lamp was used instead of the typical flood lamp [30]. Another study analyzed the performance of two calibration targets, a paper-printed checkerboard pattern heated using a flood lamp, and a cardboard mask of regularly cut squares placed on a monitor to intensify the contrast of the thermal image [31]. Likewise, a square grid was cut from a black painted K-line material, where a polyethylene foam board fixed between surface glossy papers was used to calibrate an IR camera without any source of heating [32]. Some studies involved the use of a paper-printed checkerboard pattern, along with aluminum foil tape placed on white squares, to provide a greater difference in emissivity and a higher contrast thermal image [33,34]. A few studies implemented the printed checkerboard technique, but not on paper; instead, laser-printed black ink squares on a thin aluminum sheet was implemented [35], or digitally printed thick black ink squares on a thin sandblasted aluminum plate exposed to a halogen lamp or sunlight for heating [25]. A recent study proposed the printing of the checkerboard pattern on a Dibond® panel, an aluminum composite material, but the printing was made using a flatbed ultraviolet (UV) industrial advertisements printer, and heated using an electric fan heater [6].
Apart from printing the square checkerboard pattern, few studies analyzed the effectiveness of two different taped calibration targets, an aluminum plate with high emissivity square tape, and cardboard with low emissivity square aluminum foil tape, with the targets being heated using a monitor display [36,37]. Similarly, a study involved the geometric calibration of thermal cameras, part of a target tracking project, and used a square checkerboard pattern made of a white highly emissive tape placed on an aluminum foil base [38]. Likewise, a recent study tested a square checkerboard pattern built from thick aluminum foil and thick thermocol (polystyrene) square pieces covered with white polyvinyl acetate (PVA) glue [19]. Geometric calibration was performed indoors, without external heating on long-wave infrared (LWIR) and mid-wave infrared (MWIR) cameras. Other recent targets involved painting the square checkerboard pattern on aluminum plates and using Peltier modules, also known as thermoelectric modules, fixed at the back of the calibration rig to either heat and cool the black and white painted squares, respectively [39], or to heat the entire calibration board [40]. Some studies made use of the high thermal conductivity and very low emissivity properties of copper, by milling square checkers on a printed circuit board (PCB) while using a hair dryer to heat the target [41], spraying highly emissive black ink squares on a copper plate and exposing the checkerboard target to sunlight [42]; alternately, polished copper squares were printed on an epoxy plate, and then the target was placed on a heated ceramic tile and an ice block to assess whether heating or cooling the pattern will result in a higher contrast thermal image and more accurate calibration results [43]. One study involved the production of a novel square calibration target by manufacturing a metallic net and placing it on top of a plastic board. With the use of a heat gun, the metallic net appeared brighter in a thermal image over the cooler plastic board [44].
Other than the simple square checker patterns, circular patterns of symmetrical and asymmetrical configurations have been developed in the literature to calibrate thermal cameras. Circular calibration targets are usually complicated in terms of manufacturing and recognition algorithms, where they require specifically tailored codes to recognize the circular pattern and estimate their center points. However, such circular calibration targets tend to deliver better calibration results over square or checker patterns [6], since the algorithm traces the circular patterns to estimate their center points and uses them as control points, rather than detects the corner points, which are not clearly defined in the square checkerboards. Being the most popular circular calibration target approach, burning lamps are aligned symmetrically on a wooden plank [45] or a black-coated plastic board [46]. When switched on, the lamps emit more IR radiation than the base plate, allowing it to be visible to the IR camera. This makes it easier to be recognized by the geometric calibration algorithm. A similar approach was developed [47], but IR-emitting electric resistors were used, fixed and aligned symmetrically on a metallic plate instead of the burning lamps on a wooden or a plastic board. Rather than manufacturing an active calibration target that is electrically operated, passive circular calibration patterns were developed by laser printing an asymmetric black ink circular pattern on a white glossy Dibond® panel and exposing it to sunlight [48], or UV printing a symmetrical circular pattern on a Dibond® plate and heating the plate using an electric fan heater [6] or by exposing it to sunlight [49]. A different approach resided in creating a symmetrical calibration target built purely from aluminum, heated using an electric heater, and divided into two pieces. The two pieces consisted of an anodized perforated circular grid to have a shiny silver appearance, and a black coated base, resulting in different emissivity values being visible to the IR camera [50]. Another method created a target by covering a wooden plank using thin aluminum foil, then perforating a symmetrical circular grid in a black card and placing it on top of the aluminum-covered wooden plank, allowing the aluminum sheet to be only visible from the perforated circular regions [51]. One technique involved creating an asymmetric grid by laser cutting the circular pattern from cardboard [52] or acrylic glass, and heating the target using a heat gun [53].
There are two main sources of error that affect the geometrical calibration of visible and IR cameras, namely the efficacy of the calibration target [54] and the ambiguity in detecting the control points of the images [55]. These issues tend to magnify when dealing with thermal cameras, as thermal images are typically of lower contrast and modest in quality than visible images [25]. With radial and tangential lens distortions affecting the images, imprecise calibration results in further errors that affect the certainty of the geometric calibration study [56,57]. While few studies described the detailed construction of the used geometric calibration targets [6,19,25,37,47,49,51], several studies shared little to no details on how the calibration rigs were constructed. Furthermore, many studies involved the manual construction of the targets, such as the manually cut square grids [31] or the manual aluminum foil and tape placements on the targets [19,33,34,36,38]. Additionally, with the circular patterns already being a more complicated calibration approach than square patterns, due to the need of specifically tailored algorithms, using burning lamps or resistors to construct an electrically operated active calibration board further complicates the procedure. One way to assess the accuracy of the calibration procedure is by computing a metric in pixels, which is usually measured by the adopted calibration toolbox, called the reprojection errors ( RE ) . Table 1 summarizes the different geometric calibration techniques performed on thermal cameras, showing how the calibration targets are constructed, along with the calibration conditions, the emissivity of the targets and the accuracy of the procedure, if reported. The table also shows the optical and thermal images of the geometric calibration targets, if reported in the literature. This active field of research, with many creative developments, all has a clear goal of fabricating a calibration target that provides the fewest reprojection errors possible. It can be seen that the reprojection errors for the square checkerboard pattern range from 0.100   to   1.274 pixels, with the lowest error of 0.100   pixels being reported in [25], while the reprojection errors for the circular pattern reported in the literature range from 0.052   to   0.588 pixels, with the lowest error of 0.052 pixels being reported in [6].
The thorough literature review discussed earlier indicates the importance of having high emissivity contrast within a checkerboard pattern to improve the consistency in detecting or estimating the position of the control points within a planar target. Other than the thorough literature review of the geometric calibration targets that was developed over the years, this study aims at proposing a practical checkerboard calibration target with a precise pattern that is easily constructed through controlled techniques, while providing maximum contrast in emissivity to reliably detect the control points of interest, that is, to attain the lowest RE possible, which makes the proposed target and procedure as practical as possible for different applications requiring geometrical information from thermal cameras. Thus, this research proposes two simple and passive square checkerboard calibration targets, as they lend themselves to commercially available calibration toolboxes. To improve the emissivity contrast within a checkerboard pattern, a polished stainless steel plate was selected as a base plate, while the square checkerboard pattern was laser-engraved and UV-printed on both sides of the plate to form two different calibration targets from one plate. An emissivity check following standard procedures was performed on the targets, in order to ensure that the squares had relatively high emissivity contrast, along with exposing the targets to direct sunlight before capturing the thermal images to guarantee high quality and high contrast thermal images needed for the geometric calibration of IR cameras. The UV-printed checkerboard pattern on a polished metallic alloy offered the lowest mean reprojection error ( MRE ) of 0.057 pixels and the lowest reprojection root mean square error ( RMSE ) of 0.063 pixels, with a standard deviation that was lower than 0.003 pixels. It delivered better accuracy than any other checkerboard calibration target, with comparable results to the best prominent circular pattern results reported in the literature.

3. Calibration Model and Reprojection Errors

3.1. Calibration Model

To recover and extract 3D information from a 2D image, camera calibration was required to assess the camera’s parameters in order to transpose data in the 2D image frame to the 3D world coordinate frame. The most popular and robust calibration procedure technique was developed by Zhengyou Zhang [58]. The method relies on imaging a planar pattern of well-known dimensions at a minimum of two different random orientations relative to the camera’s lens, as shown in the calibration schematic representation in Figure 2. The 3D world coordinate reference system is represented by o w   x w   y w   z w , the 3D camera coordinate reference system is represented by o c   x c   y c   z c , and the 2D image frame of reference is described by o u   x u   y u . Following the pinhole camera model approximation, a randomly positioned point in 3D world coordinate space projects a 2D point in the image frame, as implemented in [59,60], as follows:
s [ x u y u 1 ] = A ( I | 0 ) H w c [ x w y w z w 1 ]
where s is an arbitrary scaling coefficient, A is a 3 × 3 intrinsic matrix representing the camera’s internal parameters projecting a point p c = ( x c , y c , z c ) in the camera’s coordinate frame to a point p u = ( x u , y u ) in the 2D image frame, and H w c is a 4 × 4 extrinsic matrix consisting of an orthogonal rotational matrix and a translational vector linking the world coordinate system to the camera’s coordinate system. Finally, ( I | 0 ) is a 3 × 4 augmented transformational matrix used to transform points in the 3D camera coordinate frame to the 2D image frame, where I is a 3 × 3 identity matrix and 0 is a 3 × 1 zero matrix. One way to simplify Equation (1) is by splitting it into two equations. The first equation represents the extrinsic parameters of the camera, where it transforms a point in a 3D world coordinate frame to the 3D camera coordinate frame of reference, as follows:
[ x c y c z c 1 ] = [ R w c T w c 0 T 1 ] [ x w y w z w 1 ]
where R w c is the 3 × 3 orthogonal rotational matrix, T w c is the 3 × 1 translational column vector, while 0 T is a 1 × 3 zero row vector used to transform the 3 × 4 matrix into a 4 × 4 extrinsic matrix H w c for uniform multiplication in a homogeneous coordinate system, as shown below:
H w c = [ R w c T w c 0 T 1 ] = [ r 11 r 21 r 31 0 r 12 r 22 r 32 0 r 13 r 23 r 33 0 t x t y t z 1 ]
The second equation involves the projection of points from the 3D camera coordinate frame to the 2D undistorted image pixel frame. It involves the intrinsic parameters of the camera, as follows:
s [ x u y u 1 ] = A [ x c y c z c ]
where
A = [ α x c u 0 0 α y v 0 0 0 1 ] ,
α x and α y are the scaling factors function in the x and y axes of the camera’s focal length in pixels of the image, respectively; u 0 and v 0 are the principal point or the optical center in pixels of the image, and c is the skew coefficient measuring the skewness of the image frame axes. It should be noted that the skew coefficient c is zero when the image axes are perpendicular, which is the case in this model, as presented in Figure 2.
The camera matrices specified above do not account for distortions caused by the curvature of the lens, as the ideal pinhole model does not have a lens. To accurately calculate the projection, and thus correctly model the camera, the calibration model must account for radial and tangential lens distortions. Radial distortions arise when the light rays or IR radiation bend near the edges more than they bend at the camera’s optical center, while tangential distortions arise when the camera’s sensor detector or simple image plane is not parallel to the lens [61]. Let ( x , y ) be the ideal distortion-free normalized image frame coordinates, and ( x ˇ , y ˇ ) be the distorted normalized image frame coordinates. The lens distortion, radial and tangential, may be modeled using the following polynomial equation [61]:
x ˇ = x + x ( k 1 r 2 + k 2 r 4 + k 3 r 6 ) Radial + 2 p 1 x y + p 2 ( r 2 + 2 x 2 ) Tangential
and
y ˇ = y + y ( k 1 r 2 + k 2 r 4 + k 3 r 6 ) Radial + 2 p 2 x y + p 1 ( r 2 + 2 y 2 ) Tangential
where r 2 = x 2 + y 2 , ( k 1 , k 2 , k 3 ) are the radial distortion coefficients, and ( p 1 , p 2 ) are the tangential distortions coefficients of the lens.

3.2. Reprojection Errors

The metric used to assess the accuracy of camera calibration is the reprojection error. Reprojection errors are distances measured in pixels between the detected control points, or features extracted from the image with the corresponding reprojected control points, or features of the calibration checker pattern. The reprojected control points in the world coordinates of the calibration pattern could be the corners of squares, centers of squares, or the centers of circles, depending on the calibration pattern used. Through the use of the computed camera parameters, reprojection of the world coordinates control points to image coordinates is possible through a transformation procedure.
Reprojection errors are of two types, the MRE and RMSE of reprojection, which are defined as follows:
MRE = i = 1 I j = 1 J p i j p i j I J
RMSE = i = 1 I j = 1 J p i j p i j 2 I J
where p i j is the image point and p i j is the reprojected point through the calibration model. MRE is the average of the Euclidian distance between the detected control points of the image and the reprojected points from the pattern. The RMSE , however, gives an indication of the level of deviation from the detected control points of the image and the reprojected points from the pattern [39]. In the IR camera geometric calibration reported in the literature summarized and listed in Table 1, the MRE and RMSE are interchanged; some studies reported their MRE values, while others reported their RMSE of reprojection values. While both metrics have been used over the years to assess the performance and model errors on different applications, there is no consensus on which criteria to better model the reprojection errors [62]. Some studies argue that the commonly employed sums-of-squares-based errors, such as the RMSE , are often misleading criteria indicators for the average error, making the RMSE an inappropriate, ambiguous and misinterpreted measure of the mean error [63,64]. However, a study argues that the RMSE is not vague in its meaning, and is more appropriate as an indicator for model performance than the mean absolute error ( MAE ) , represented by the MRE in this study, when the error distribution is expected to follow a Gaussian trend [62]. Therefore, without leaning towards a certain evaluation criterion or indicator, a combination of metrics often prove practical in assessing model performance [62]. In order to remain conservative, while also being fair in comparison to the available literature, this research computed both the MRE and the RMSE of all trials, for both the laser-engraved and UV-printed calibration targets, using Equations (8) and (9). Thus, this study did not rely on just a single indicator for the accuracy of the calibration, but offered a verification of the obtained errors by computing the two indicators most widely used for assessing the accuracy of the geometric calibration process.

4. Experiment

4.1. Calibration Targets Design

As mentioned above in Section 2, the quality, precision and efficacy of the calibration target contribute massively to the accuracy of the geometric calibration performed on the thermographic camera. Additionally, the calibration model discussed in Section 3.1 and implemented in this study, relies on detecting the corners of a square checkerboard pattern, requiring precise alignment and seamless corners between the edges of the squares. Manually constructed checker patterns provide low geometric calibration accuracy, as the squares are not precisely aligned, resulting in high errors in control points recognition. Furthermore, active symmetrical and asymmetrical circular calibration targets are complicated in terms of construction and recognition, in the majority of the cases, as they mandate specially tailored algorithms that are not as user-friendly as the square checkerboard commercially available software toolboxes. The thorough literature review has been leveraged to deliver a practical checkerboard target, offering high geometric calibration accuracy that competes with that of circular calibration targets. Other than its competing accuracy, it offers a practical solution to less experienced end users to apply the IR camera geometric calibration with ease via commercially available software toolboxes tailored for square checkerboard patterns. Thus, the proposed calibration checkerboard targets should be well designed and manufactured to a perfectly flat and polished metallic alloy to offer low surface emissivity prior to adding the pattern. The added (engraved or printed) checkerboard squares should be of a relatively high emissivity value, and perfectly aligned with sharp and clear corners for better defined controlled points. This offers a sufficient emissivity contrast between the squares with well-defined corners in an IR camera field of view.
Accordingly, this research proposed the use of a polished stainless steel alloy (SS304) flat plate as the base material, with the square checker pattern laser-engraved on one side of the plate and UV-printed on the other side, as shown in Figure 3a,b, respectively. SS304 was selected due to its appealing physical and thermal properties, as detailed in a technical datasheet [65] and listed in Table 2, along with its wide availability in the market at a relatively low cost. It is also an extremely corrosion-resistant alloy, which allows for the preservation of the pattern and its corners over time. A 150   mm × 136   mm × 6   mm plate was used, and a checker pattern of 10 × 9 squares having a square size of 14   mm each was selected, leaving a 5   mm space margin between the pattern and the edges of the SS304 plate. The distinctive features or control points of both targets are the corners of the squares, while the difference in the number of rows (even) and columns (odd) is a distinctive feature that permits orientation detection by the calibration toolbox. The laser-engraved pattern was performed using SLTL’s Akshar Fiber Pro Ultra laser engraving machine, shown in Figure 4a. A mirror polishing machine was used to ensured that the plate had a mirror finish and was flat, while the laser engraving machine operated on a maximum plate size of 150   mm × 150   mm , and enforced an engraving of 0.3   mm in depth, leaving a brownish-black rough textured engraved pattern, as shown in Figure 3a. The printed pattern, shown in Figure 3b, was performed using Canon’s Arizona 1380 XT flatbed UV printer with a flatbed printer that used UV-curable inks, providing high-value graphic art and industrial printing. The printer, shown in Figure 4b, provides a printing resolution of 1440 dpi (dots per inch) or 56.7 dots per millimeter, delivering a highly accurate square checker calibration pattern with its state-of-the-art sharpness and resolution. It should be noted that the surface of the plate where the pattern was UV-printed was sanded using emery papers of different grades, in order to ensure a smooth surface without any scratches. Both the laser engraving machine and the UV printer required the plate to be flat, which was ensured using the 6   mm thick SS304 plate, as any slight bends or warpage to the plate leads to defective calibration results [66].

4.2. Experimental Setup

The experimental setup, equipped in this study and shown in Figure 5, consisted of placing the calibration targets on a plastic holder, a thermographic camera, and a laptop to control the acquisition of the thermal images. The IR camera model used was the VarioCAM HD Head-600 manufactured by InfraTec, having a resolution of 640 × 480 pixels. The thermal detector used was an uncooled micro-bolometer focal plane array (UFPA), operating in a photoconductivity system, where the sensor’s material alters its electrical resistance when heated by incident IR radiation of a certain wavelength [67]. Uncooled cameras referred to the uncooled detectors of the thermal camera, where the sensors operated in ambient conditions that required no cooling to cryogenic temperatures, as needed by the cooled quantum detector type. Uncooled IR cameras are usually less expensive, less complicated, and lower in weight compared to cooled IR cameras, which allowed for the introduction of thermal imaging to the mass market [67]. The thermographic camera used in this study was of the LWIR type, and its technical specifications are listed in Table 3.

4.3. Emissivity Coefficient

Accurately detecting the positions of the control points in thermal images of the calibration targets relied heavily on the ability to obtain high-quality thermal images. High-quality thermal images depend on the precision of the square checker pattern of the calibration target. For objects to be visible and distinct in a thermal image, the infrared radiation collected by the IR camera from the objects in the scene should be different [6]. The IR signature of an object depends on its temperature distribution and the emissivity of its material [68]. The emissivity coefficient ( ε ) of a material is a dimensionless coefficient ranging from 0 to 1 , which indicates the object’s ability to emit IR radiation in comparison to a blackbody at the same temperature. A blackbody is a theoretical material with ε = 1 , meaning that it absorbs and perfectly re-emits all incident IR radiation [69]. Therefore, an object with an emissivity value close to zero emits negligible thermal radiation in comparison to an object with an emissivity value close to unity. The emissivity of an object depends on many parameters; these include the material, structure and texture of an object’s surface, the surface temperature of the object, and the wavelength of the thermal radiation collected by the IR camera [70,71]. Thus, the emissivity coefficient is the ratio of the thermal radiation emitted by the imaged object M ( T ) to the thermal radiation emitted by the blackbody M 0 ( T ) under identical thermal conditions [26], as follows:
ε = M ( T ) M 0 ( T )
It is critical to note that the energy striking opaque surfaces is reflected and/or absorbed. Moreover, one particular case is where the emissivity of the surface does not influence the contactless measurement of the surface temperature, as emitted or reflected IR radiation becomes independent of the object’s surface emissivity. This is when the environmental/background temperature is identical to the object’s temperature [6], causing the target to be indistinguishable in a thermal image. The solution to this issue is to heat the object or place it in an environment that has a different background temperature. For calibration targets, this is usually performed in two ways, by either actively heating the calibration target using electrically powered heaters and performing the calibration imaging indoors, or by acquiring the thermal images outdoors where the sun’s or sky’s radiation reflects off the target and eliminates the need for external heating. This research proposed the acquisition of the calibration target images in an indoor controlled environment after placing the target under direct sunlight for a few minutes, excluding the need for any external heating devices, as well as avoiding any degradation in image quality caused by the reflection of IR radiation when performing outdoor calibration.
The American Society for Testing and Materials (ASTM E1933-14) defines two standard test methods for determining the emissivity of objects when measuring their surface temperatures with IR imaging radiometers [72]. The first method is called the contact thermometer method (CTM). It involves the use of a contact thermometer to measure the temperature of a point or area where the emissivity is to be defined. The emissivity on the infrared imaging radiometer or IR camera’s computer focused on the same spot is adjusted until both readings match. The second technique, called the noncontact thermometer method (NTM), employs a surface-modifying material (SMM) of known emissivity that adheres to the target’s surface. The IR camera’s emissivity is set to the SMM emissivity value, and the temperature value is recorded. Then, the IR camera is focused on a spot on the target adjacent to the SMM where emissivity is to be defined, and the IR camera’s emissivity is adjusted until the temperature reading of this spot matches the one recorded on the SMM. Both methods require a minimum of three repetitions, and the material’s emissivity is the average of these repetitions [72]. Several studies listed in Table 1 reported the emissivity of the calibration targets used. However, only a few studies discussed the method used to obtain the emissivity of the patterns. A recent study adopted the NTM to evaluate the emissivity of a square checker pattern printed on an aluminum composite material, with the SMM being the easily attainable electrical tape [6]. Apart from following the standard NTM while using commercially available electrical tape as the SMM, not knowing the tape’s exact emissivity may introduce errors in the emissivity check.
This research followed the NTM approach to estimate the emissivity of the laser-engraved and UV-printed checker patterns, using thermal emission tape as the SMM. The thermal emission tape, manufactured by Testo SE & Co., has an emissivity of ε = 0.95 . Two locations were selected to measure the emissivity of the patterns for both targets, as shown in Figure 6a,b for the laser-engraved target and Figure 6c,d for the UV-printed target, in order to assure certainty. After adhering the SMM, the targets were placed under direct sunlight for a few minutes for heating, then thermal imaging was performed indoors. The emissivity check was repeated four times, meeting the requirement set by the ASTM E1933-14 standard, with the average emissivity values of the patterns reported in Table 4. It was seen that UV printing the pattern resulted in a greater difference in emissivity between the base metal plate and the pattern, having ε m i n = 0.239 and ε m a x = 0.931 , while laser engraving the pattern resulted in ε m i n = 0.172 and ε m a x = 0.811 , respectively. The emissivity of the bare metal on the laser-engraved target was lower than that on the UV-printed one, as the earlier was polished by the supplier to attain a mirror surface finish, while the latter was only sanded using different emery paper grades to ensure a smooth surface. It should be noted that the proposed UV-printed checkerboard pattern on the stainless steel plate showed a greater difference in emissivity compared to a similarly UV-printed pattern on an aluminum composite board in a previous study, where ε m i n = 0.67 and ε m a x = 0.83 [6]. This improved difference in emissivity was mainly due to the selection of SS304 as a base plate and polishing its surface to obtain a mirror-like finish, resulting in a much lower ε m i n compared to the previously mentioned study. Furthermore, this improvement in emissivity difference resulted in higher quality thermal images, as the printed pattern was clearly distinguishable from the base material, with higher precision in the recognition of the corner control points.

4.4. Experimental Procedure

As mentioned in Section 4.3, to introduce high contrast between the calibration pattern and the background environment in indoor thermal imaging, the targets were exposed to direct sunlight for a few minutes, allowing the patterns to heat up. When observed by the IR camera, the temperature of the targets was in the range of 45–50 °C. It should be noted that the temperature of the targets when left under sunlight was not controlled, but was left for enough time to display good contrast in the thermal image for clear corner detection; the earlier mentioned range was just an observation made for the records. The targets were then brought to an indoor environment where the temperature was around 22 °C, placed on a plastic holder, and imaged using the thermal camera. To ensure accurate calibration results, multiple images of the calibration checkerboards at different 3D orientations were needed. Thus, by moving the calibration target holder in different positions and rotating the camera in different orientations, the thermal images of the checkerboard target were obtained. Due to the difference in temperature between the calibration checkerboard and the indoor background environment, the calibration targets cooled down in a few minutes, degrading the quality of the captured thermal images and their sharpness. Thus, the solution to this issue was to acquire 6 images at a time, which took around 5 min to adjust for the random orientations, followed by the process of placing the targets under sunlight and then acquiring the other 6 images 5 times until 30 thermal images of the calibration target at different orientations were obtained. This procedure was repeated 9 times for each checkerboard target.
The 30 thermal images, considered as 1 trial per checkerboard target, provided the thermal map as temperature readings for the entire FOV of the thermal camera lens. These thermal images were then converted to grayscale images and saved in the Tag Image File Format (TIFF), due to the format’s high quality and universal compatibility. The 30 images were then loaded into the single-camera calibrator toolbox using MATLAB [73], where the standard single-camera model was selected, the square size was specified as 14   mm , and the camera was calibrated for radial and tangential distortions. To improve the calibration results, it was recommended to remove the images that were blurry or showed high errors [73], and to repeat the calibration. Therefore, 5 images showing the highest errors were excluded from the 30 images, leaving 25 images per trial for the laser-engraved and UV-printed checkerboards, as shown in Figure 7 and Figure 8, respectively. This calibration process was repeated 9 times for both the laser-engraved and UV-printed patterns. It should be noted that at least 3 checkerboard calibration target images were required, but 10 to 20 thermal images of the target were captured, while the lowest performing images were eliminated to obtain the best checkerboard calibration results [73]. However, as more captured images would not harm but affirm the calibration results, 30 thermal images of the target were captured per trial, and the 5 images with the lowest calibration results were excluded.

5. Results and Discussion

To assess and compare the performance of the calibration targets, the internal parameters of the IR camera were evaluated. The calibration targets were observed at different orientations, and then the images were fed to the calibration toolbox for feature extraction, identification of control points, computing the intrinsic and extrinsic camera parameters, and finally for evaluating the reprojection errors to assess the accuracy of the calibration target. Figure 9a,b present RE values obtained between the detected and reprojected corners of the squares for one of the trials on the laser-engraved and UV-printed checkerboard targets, respectively. It was seen that the reprojection errors obtained by the laser-engraved pattern were more dispersed, with significant outliers over the more consistent and clustered reprojection errors obtained from the UV-printed pattern. This indicates that the UV-printed pattern was more precise, with clearer square corners than those of the laser-engraved pattern, resulting in a more accurate geometric calibration process.
In the visible camera calibration for computer vision applications, obtaining reprojection errors that are less than one pixel is considered suitable and widely acceptable [73]; only in exceedingly critical applications, reprojection errors that are less than 0.1 pixels are required [6]. Table 5 summarizes the reprojection errors, the MRE and RMSE , attained for the nine trials performed on each calibration target, along with their averages and standard deviations. Furthermore, a histogram of the calibration reprojection errors computed by one of the trials of each calibration target was plotted and shown in Figure 10. It was noticed that the MRE , represented by Figure 10a, and the RMSE , represented by Figure 10b, obtained by the UV-printed checker target was less than half of that computed by the laser-engraved target. This indicates that UV printing provided a more precise pattern over laser engraving.
Table 1 summarizes the available literature on this topic, where it also lists the reprojection errors computed by a few of the studies. It can be seen that the reprojection errors for the square checkerboard type range from 0.100   to   1.274 pixels, with the lowest reprojection error, as MRE , of 0.100   pixels attained by digitally printing a thick black ink square checker pattern on a sandblasted aluminum plate and performing the calibration outdoors by exposing the plate to direct sunlight [25]. Moreover, the circular calibration targets produced reprojection errors ranging from 0.052   to   0.588 pixels, displaying a more accurate but more complicated calibration technique over the square pattern method, due to its confined range and lower computed reprojection errors. The lowest reprojection error, an RMSE of 0.052 pixels, was attained using a circular pattern that was UV-printed on an aluminum composite plate, with the images acquired indoors after heating the target with an electrical fan [6]. This study proposed two square checkerboard targets, namely a laser-engraved pattern and a UV-printed pattern, on a polished stainless-steel alloy, then exposing the boards to direct sunlight for heating before capturing the thermal images indoors. From Table 5, it can be seen that the laser-engraved checker target produced an MRE in the range of 0.098 0.137 pixels and an RMSE in the range of 0.117 0.164 pixels over nine different calibration repetitions, with an average MRE of 0.119 pixels with a standard deviation of 0.013 pixels, and an average RMSE of 0.142 pixels with a standard deviation of 0.016 pixels. Similarly, the UV-printed target produced MRE values in the range of 0.053 0.059 pixels and RMSE values in the range of 0.059 0.066 pixels over the nine different calibration repetitions, with an average MRE of 0.057 pixels and a standard deviation of less than 0.003 pixels and an average RMSE of 0.063 pixels with a standard deviation of less than 0.003 pixels. This shows that the UV-printed target is more precise than the laser-engraved target, with lower reprojection errors and one order of magnitude lower standard deviations. With the experimental conditions and procedure all being fixed, such as the target material, checkerboard pattern, thermal camera along with the heating source, indoor conditions and number of thermal images captured per target, the superior calibration accuracy of the UV-printed target is mainly attributed to the precision of the printed square corners to be detected and extracted by the calibrator toolbox, where Figure 6c displays sharper corners over Figure 6a, along with improved emissivity differences between the polished base surface and the printed pattern. Nonetheless, the proposed calibration rigs, whether laser-engraved or UV-printed, provide better and more accurate results in terms of reprojection errors, when comparing them to the available literature that employed the square checker calibration patterns. Additionally, the UV-printed checker pattern produced better results than most circular grid calibration techniques, while producing almost similar results to the lowest reprojection error computed by a circular grid pattern. This proves that the less complicated well-known square checkerboard pattern with improved emissivity contrast delivers desirable reprojection error values. Finally, by repeating the calibration procedure nine times, an average MRE of 0.057 pixels along with an average RMSE of 0.063 pixels, with a standard deviation of less than 0.003 pixels in both was achieved, indicating the consistency of the UV-printed checkerboard target.

6. Conclusions

A thorough review of IR camera geometric calibration targets available in the literature, along with their respective results, was summarized and leveraged to deliver a recommended practice for establishing precision square checkerboard calibration targets with relatively high emissivity contrast. The main challenge to overcome was having a precise calibration target with well-defined control points that were clearly visible in the IR spectrum. The square checkerboard target should have a high emissivity contrast, along with clearly defined corners. Thus, this research presented two passive checkerboard targets used to accurately calibrate IR thermal cameras. A square checker pattern was laser-engraved on one side of a polished stainless-steel plate, while the same pattern was UV-printed on the plate’s opposing surface. A thick stainless steel alloy plate was used as the base material to ensure a flat calibration target while displaying a substantial difference in emissivity between its surface and the checker patterns. The targets were placed under direct sunlight for a few minutes, and then imaged using the IR camera in an indoor environment. Calibration was performed to eliminate radial and tangential distortions caused by the thermal camera’s lens, while the accuracy of the procedure was assessed on the basis of computed reprojection errors and compared to the available literature on this topic. The experimental procedure was repeated nine times on each calibration target, and the results indicated accuracy and consistency for both patterns. Average MREs of 0.119 and 0.057 pixels, along with standard deviations of 0.013 and 0.003 pixels, were attained for the laser-engraved and UV-printed patterns, respectively. Similarly, average RMSEs of 0.142 and 0.063 pixels, along with a standard deviation of 0.003 pixels, were attained for the laser-engraved and UV-printed patterns, respectively. This shows that the proposed calibration rigs, whether laser-engraved or UV-printed, provide better and more accurate results in terms of reprojection errors, when compared to those in the available literature using the square checker calibration patterns. Additionally, the UV-printed checker pattern produced better results than most circular grid calibration techniques, while producing almost comparable results to the lowest reprojection error computed by a circular grid pattern. This establishes that the practical square checkerboard pattern with improved emissivity contrast delivers desirable MRE / RMSE values, making it an attractive solution for less experienced end users, without the need to use a circular pattern and their tailored recognition algorithms, and relying only on commercially available tools for the geometric calibration of IR cameras using conventional square checker targets.

Author Contributions

Conceptualization, A.E., B.A.A.-N. and G.-Y.T.; methodology, A.E., B.A.A.-N. and M.O.H.; investigation, A.E.; writing—original draft preparation, A.E.; writing—review and editing, B.A.A.-N., M.O.H. and G.-Y.T.; visualization, A.E.; supervision, B.A.A.-N. and M.O.H.; project administration, B.A.A.-N.; funding acquisition, B.A.A.-N. and M.O.H. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the American University of Sharjah, grant number FRG21-M-E87.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

This research is financially supported by the Mechanical Engineering Department at the American University of Sharjah, and by the FRG21-M-E87 research grant. The work in this paper was supported, in part, by the Open Access Program from the American University of Sharjah. This study represents the opinions of the author(s) and does not mean to represent the position or opinions of the American University of Sharjah.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Maldague, X. Theory and Practice of Infrared Technology for Nondestructive Testing, 1st ed.; Wiley–Interscience: New York, NY, USA, 2001. [Google Scholar]
  2. Ocaña, S.M.; Guerrero, I.C.; Requena, I.G. Thermographic survey of two rural buildings in Spain. Energy Build. 2004, 36, 515–523. [Google Scholar] [CrossRef]
  3. Modest, M.F. Radiative Heat Transfer, 3rd ed.; Academic Press: Waltham, MA, USA, 2013. [Google Scholar]
  4. Usamentiaga, R.; Venegas, P.; Guerediaga, J.; Vega, L.; Molleda, J.; Bulnes, F.G. Infrared thermography for temperature measurement and non-destructive testing. Sensors 2014, 14, 12305–12348. [Google Scholar] [CrossRef] [Green Version]
  5. Ochs, M.; Horbach, T.; Schulz, A.; Koch, R.; Bauer, H.J. A novel calibration method for an infrared thermography system applied to heat transfer experiments. Meas. Sci. Technol. 2009, 20, 075103. [Google Scholar] [CrossRef]
  6. Usamentiaga, R.; Garcia, D.F.; Ibarra-Castanedo, C.; Maldague, X. Highly accurate geometric calibration for infrared cameras using inexpensive calibration targets. Measurement 2017, 112, 105–116. [Google Scholar] [CrossRef]
  7. Abu-Nabah, B.A.; Al-Said, S.A.M. Sonic IR crack size estimation using 2D heat diffusion model with arbitrary heat source function along the crack. Quant. Infrared Thermogr. J. 2018, 15, 271–290. [Google Scholar] [CrossRef]
  8. Abu-Nabah, B.A.; Al-Said, S.M.; Gouia-Zarrad, R. A simple heat diffusion model to avoid singularity in estimating a crack length using sonic infrared inspection technology. Sens. Actuators A Phys. 2019, 293, 77–86. [Google Scholar] [CrossRef]
  9. Al-Said, S.M.; Abu-Nabah, B.A. A simple algorithm based on a 2D heat diffusion mathematical model to extract a surface crack length from a sonic IR thermal image alone. J. Nondestruct. Eval. 2021, 40, 98. [Google Scholar] [CrossRef]
  10. Bhargava, V.; Abu-Nabah, B.A.; Alkhader, M. A theoretical approach towards the modeling of vibrothermography using finite element methods. Eur. J. Mech. A/Solids 2022, 91, 104389. [Google Scholar] [CrossRef]
  11. Chen, X.; Tian, G.; Wu, J.; Tang, C.; Li, K. Feature-based registration for 3D eddy current pulsed thermography. IEEE Sens. J. 2019, 19, 6998–7004. [Google Scholar] [CrossRef]
  12. Barakat, N.; Mortadha, J.; Khan, A.; Abu-Nabah, B.A.; Hamdan, M.O.; Al-Said, S.M. A one-dimensional approach towards edge crack detection and mapping using eddy current thermography. Sens. Actuators A Phys. 2020, 309, 111999. [Google Scholar] [CrossRef]
  13. ElSheikh, A.; Barakat, N.; Abu-Nabah, B.A.; Hamdan, M.O. Thermal diffusivity estimation in metallic alloys using a one-dimensional flux-based thermography. Infrared Phys. Technol. 2022, 127, 104411. [Google Scholar] [CrossRef]
  14. Avdelidis, N.P.; Moropoulou, A. Applications of infrared thermography for the investigation of historic structures. J. Cult. Herit. 2004, 5, 119–127. [Google Scholar] [CrossRef]
  15. Lerma, J.L.; Cabrelles, M.; Portalés, C. Multitemporal thermal analysis to detect moisture on a building faade. Constr. Build. Mater. 2011, 25, 2190–2197. [Google Scholar] [CrossRef]
  16. Luhmann, T.; Robson, S.; Kyle, S.; Harley, I.A. Close Range Photogrammetry: Principles, Techniques and Applications, 1st ed.; Whittles Publishing: Dunbeath, UK, 2006. [Google Scholar]
  17. Luhmann, T.; Piechel, J.; Roelfs, T. Geometric calibration of thermographic cameras. In Remote Sensing and Digital Image Processing; Kuenzer, C., Dech, S., Eds.; Springer: Dordrecht, The Netherlands, 2013; pp. 27–42. [Google Scholar]
  18. Fryskowska, A.; Kedzierski, M.; Grochala, A.; Braula, A. Calibration of low cost RGB and NIR UAV cameras. In Proceedings of the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Prague, Czech Republic, 12–19 July 2016; Volume XLI-B1, pp. 817–821. [Google Scholar] [CrossRef]
  19. Swamidoss, I.N.; Amro, A.B.; Sayadi, S. Systematic approach for thermal imaging camera calibration for machine vision applications. Optik 2021, 247, 168039. [Google Scholar] [CrossRef]
  20. Ricolfe-Viala, C.; Sánchez-Salmerón, A.J. Robust metric calibration of non-linear camera lens distortion. Pattern Recognit. 2010, 43, 1688–1699. [Google Scholar] [CrossRef]
  21. Ricolfe-Viala, C.; Sánchez-Salmerón, A.J. Using the camera pin-hole model restrictions to calibrate the lens distortion model. Opt. Laser Technol. 2011, 43, 996–1005. [Google Scholar] [CrossRef]
  22. Tan, L.; Wang, Y.; Yu, H.; Zhu, J. Automatic camera calibration using active displays of a virtual pattern. Sensors 2017, 17, 685. [Google Scholar] [CrossRef] [Green Version]
  23. Wang, K.; Liu, C.; Shen, S. Geometric calibration for cameras with inconsistent imaging capabilities. Sensors 2022, 22, 2739. [Google Scholar] [CrossRef]
  24. Komatsu, S.; Markman, A.; Mahalanobis, A.; Chen, K.; Javidi, B. Three-dimensional integral imaging and object detection using long-wave infrared imaging. Appl. Opt. 2017, 56, D120–D126. [Google Scholar] [CrossRef]
  25. St-Laurent, L.; Mikhnevich, M.; Bubel, A.; Prévost, D. Passive calibration board for alignment of VIS-NIR, SWIR and LWIR images. Quant. Infrared Thermogr. J. 2017, 14, 193–205. [Google Scholar] [CrossRef]
  26. Fryskowska-Skibniewska, A.; Delis, P.; Kedzierski, M.; Matusiak, D. The conception of test fields for fast geometric calibration of the FLIR VUE PRO thermal camera for low-cost UAV applications. Sensors 2022, 22, 2468. [Google Scholar] [CrossRef]
  27. Cheng, S.Y.; Park, S.; Trivedi, M.M. Multiperspective thermal IR and video arrays for 3D body tracking and driver activity analysis. In Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)–Workshops, San Diego, CA, USA, 21–23 September 2005. [Google Scholar] [CrossRef]
  28. Prakash, S.; Lee, P.Y.; Caelli, T.; Raupach, T. Robust thermal camera calibration and 3D mapping of object surface temperatures. In Proceedings of the SPIE—The International Society for Optical Engineering, Thermosense XXVIII, Orlando, FL, USA, 18 April 2006; Volume 6205, pp. 182–189. [Google Scholar] [CrossRef]
  29. Saponaro, P.; Sorensen, S.; Rhein, S.; Kambhamettu, C. Improving calibration of thermal stereo cameras using heated calibration board. In Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada, 27–30 September 2015; pp. 4718–4722. [Google Scholar] [CrossRef]
  30. Yahyanejad, S.; Misiorny, J.; Rinner, B. Lens distortion correction for thermal cameras to improve aerial imaging with small-scale UAVs. In Proceedings of the 2011 IEEE International Symposium on Robotic and Sensors Environments (ROSE), Montreal, QC, Canada, 17–18 September 2011; pp. 231–236. [Google Scholar] [CrossRef]
  31. Vidas, S.; Lakemond, R.; Denman, S.; Fookes, C.; Sridharan, S.; Wark, T. A mask-based approach for the geometric calibration of thermal-infrared cameras. IEEE Trans. Instrum. Meas. 2012, 61, 1625–1635. [Google Scholar] [CrossRef] [Green Version]
  32. Dias, A.; Bras, C.; Martins, A.; Almeida, J.; Silva, E. Thermographic and visible spectrum camera calibration for marine robotic target detection. In Proceedings of the 2013 OCEANS, San Diego, CA, USA, 23–27 September 2013; pp. 1–5. [Google Scholar]
  33. Rzeszotarski, D.; Wiecek, B. Calibration for 3D reconstruction of thermal images. In Proceedings of the 9th International Conference on Quantitative InfraRed Thermography (QIRT), Krakow, Poland, 2–5 July 2008; pp. 563–566. [Google Scholar] [CrossRef]
  34. Rzeszotarski, D.; Więcek, B. An integrated thermal and visual camera system for 3D reconstruction. In Proceedings of the 11th International Conference on Quantitative Infrared Thermography (QIRT), Naples, Italy, 11–14 June 2012. No. 1. [Google Scholar] [CrossRef]
  35. Campo, F.B.; Ruiz, F.L.; Sappa, A.D. Multimodal stereo vision system: 3D data extraction and algorithm evaluation. IEEE J. Sel. Top. Signal Process. 2012, 6, 437–446. [Google Scholar] [CrossRef]
  36. Engström, P.; Larsson, H.; Rydell, J. Geometric calibration of thermal cameras. In Proceedings of the Electro-Optical Remote Sensing, Photonic Technologies, and Applications VII; and Military Applications in Hyperspectral Imaging and High Spatial Resolution Sensing, Dresden, Germany, 15 October 2013; Volume 8897, pp. 104–111. [Google Scholar] [CrossRef]
  37. Peric, D.; Lukic, V.; Spanovic, M.; Sekulic, R.; Kocic, J. Geometric calibration of multi-sensor image fusion system with thermal infrared and low-light camera. In Electro-Optical Remote Sensing, Photonic Technologies, and Applications VIII; and Military Applications in Hyperspectral Imaging and High Spatial Resolution Sensing II; SPIE: Amsterdam, The Netherlands, 2014; Volume 9250, pp. 160–168. [Google Scholar] [CrossRef]
  38. Dehkharghani, S.; Pleshkova, S. Geometric thermal infrared camera calibration for target tracking by a mobile robot. Proc. Bulg. Acad. Sci. 2014, 67, 109–114. [Google Scholar]
  39. Wang, Z.; Liu, B.; Huang, F.; Chen, Y.; Zhang, S.; Cheng, Y. Corners positioning for binocular ultra-wide angle long-wave infrared camera calibration. Optik 2020, 206, 163441. [Google Scholar] [CrossRef]
  40. Zhang, S.; Huang, F.; Liu, B.; Zhong, H.; Li, G.; Chen, Y.; Wang, Z. Optimized calibration method for ultra-field dual bands cameras based on thermal radiation checkerboard. Infrared Phys. Technol. 2020, 108, 103346. [Google Scholar] [CrossRef]
  41. Hilsenstein, V. Surface reconstruction of water waves using thermographic stereo imaging. In Proceedings of the Image and Vision Computing New Zealand, University of Otago, Dunedin, New Zealand, 28–29 November 2005; pp. 102–107. [Google Scholar]
  42. Ursine, W.; Calado, F.; Teixeira, G.; Diniz, H.; Silvino, S.; De Andrade, R. Thermal/visible autonomous stereo visio system calibration methodology for non-controlled environments. In Proceedings of the 11th International Conference on Quantitative Infrared Thermography (QIRT), Naples, Italy, 11–14 June 2012; pp. 1–10. [Google Scholar] [CrossRef]
  43. Herrmann, T.; Migniot, C.; Aubreton, O. Thermal camera calibration with cooled down chessboard. In Proceedings of the 15th Quantitative InfraRed Thermography Conference (QIRT), Porto, Portugal, 21–30 September 2020; pp. 1–6. [Google Scholar] [CrossRef]
  44. Yiu-Ming; Ng, H.; Du, R. Acquisition of 3D surface temperature distribution of a car body. In Proceedings of the 2005 IEEE International Conference on Information Acquisition, Hong Kong, China, ; pp. 16–20. In Proceedings of the 2005 IEEE International Conference on Information Acquisition, Hong Kong, China, 27 June–3 July 2005; pp. 16–20. [Google Scholar] [CrossRef]
  45. Lagüela, S.; González-Jorge, H.; Armesto, J.; Arias, P. Calibration and verification of thermographic cameras for geometric measurements. Infrared Phys. Technol. 2011, 54, 92–99. [Google Scholar] [CrossRef]
  46. Yang, R.; Yang, W.; Chen, Y.; Wu, X. Geometric calibration of IR camera using trinocular vision. J. Light. Technol. 2011, 29, 3797–3803. [Google Scholar] [CrossRef]
  47. Gschwandtner, M.; Kwitt, R.; Uhl, A.; Pree, W. Infrared camera calibration for dense depth map construction. In Proceedings of the 2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany, 5–9 June 2011; pp. 857–862. [Google Scholar] [CrossRef]
  48. Harguess, J.; Strange, S. Infrared stereo calibration for unmanned ground vehicle navigation. In Proceedings of the Unmanned Systems Technology XVI, Baltimore, MD, USA, 3 June 2014; Volume 9084, pp. 276–283. [Google Scholar] [CrossRef]
  49. Usamentiaga, R.; Ibarra-Castanedo, C.; Maldague, X. Comparison and evaluation of geometric calibration methods for infrared cameras to perform metric measurements on a plane. Appl. Opt. 2018, 57, D1. [Google Scholar] [CrossRef] [PubMed]
  50. Kim, S.-U.; Choi, M.-Y.; Park, J.-H.; Kim, W.-T. Lens distortion calibration for mid-wave and long-wave infrared thermal cameras. In Proceedings of the 6th Pan American Conference for NDT, Cartagena, Columbia, 12–14 August 2015. [Google Scholar]
  51. Lagüela, S.; González-Jorge, H.; Armesto, J.; Herráez, J. High performance grid for the metric calibration of thermographic cameras. Meas. Sci. Technol. 2012, 23, 015402. [Google Scholar] [CrossRef]
  52. Rangel, J.; Soldan, S.; Kroll, A. 3D Thermal imaging: Fusion of thermography and depth cameras. In Proceedings of the 12th International Conference on Quantitative InfraRed Thermography (QIRT), Bordeaux, France, 7–11 July 2014. [Google Scholar] [CrossRef]
  53. Choi, J.D.; Kim, M.Y. A sensor fusion system with thermal infrared camera and LiDAR for autonomous vehicles and deep learning based object detection. ICT Express, 2022, in press. [CrossRef]
  54. Vo, M.; Wang, Z.; Luu, L.; Ma, J. Advanced geometric camera calibration for machine vision. Opt. Eng. 2011, 50, 110503. [Google Scholar] [CrossRef] [Green Version]
  55. Datta, A.; Kim, J.S.; Kanade, T. Accurate camera calibration using iterative refinement of control points. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, Kyoto, Japan, 27 September–4 October 2009; pp. 1201–1208. [Google Scholar] [CrossRef] [Green Version]
  56. Lin, J.; Chio, S. The accuracy influence of different camera calibration conditions to bundle adjsutment of closerange images. In Proceedings of the 33rd Asian Conference on Remote Sensing, ACRS, Pattaya, Thailand, 26–30 November 2012; pp. 1672–1681. [Google Scholar]
  57. Poulin-Girard, A.-S.; Thibault, S.; Laurendeau, D. Influence of camera calibration conditions on the accuracy of 3D reconstruction. Opt. Express 2016, 24, 2678–2686. [Google Scholar] [CrossRef] [PubMed]
  58. Zhang, Z. A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 2000, 22, 1330–1334. [Google Scholar] [CrossRef] [Green Version]
  59. Abu-Nabah, B.A.; ElSoussi, A.O.; Al Alami, A.E.R.K. Virtual laser vision sensor environment assessment for surface profiling applications. Measurement 2018, 113, 148–160. [Google Scholar] [CrossRef]
  60. Abu-Nabah, B.A.; ElSoussi, A.O.; Al Alami, A.E.R.K. Simple laser vision sensor calibration for surface profiling applications. Opt. Lasers Eng. 2016, 84, 51–61. [Google Scholar] [CrossRef]
  61. What Is Camera Calibration? MathWorks. 2022. Available online: https://www.mathworks.com/help/vision/ug/camera-calibration.html (accessed on 12 September 2022).
  62. Chai, T.; Draxler, R.R. Root mean square error (RMSE) or mean absolute error (MAE)?—Arguments against avoiding RMSE in the literature. Geosci. Model Dev. 2014, 7, 1247–1250. [Google Scholar] [CrossRef] [Green Version]
  63. Willmott, C.J.; Matsuura, K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim. Res. 2005, 30, 79–82. [Google Scholar] [CrossRef]
  64. Willmott, C.J.; Matsuura, K.; Robeson, S.M. Ambiguities inherent in sums-of-squares-based error statistics. Atmos. Environ. 2009, 43, 749–752. [Google Scholar] [CrossRef]
  65. Specification Sheet: Alloy 304/304L. Philadelphia. 2014. Available online: https://www.sandmeyersteel.com/304L.html#PhysicalProperties (accessed on 19 January 2022).
  66. Lavest, J.-M.; Viala, M.; Dhome, M. Do we really need an accurate calibration pattern to achieve a reliable camera calibration? In Proceedings of the Computer Vision—ECCV’98, Freiburg, Germany, 2–6 June 1998; pp. 158–174. [Google Scholar] [CrossRef]
  67. Deane, S.; Avdelidis, N.P.; Ibarra-Castanedo, C.; Zhang, H.; Nezhad, H.Y.; Williamson, A.A.; Mackley, T.; Maldague, X.; Tsourdos, A.; Nooralishahi, P. Comparison of cooled and uncooled ir sensors by means of signal-to-noise ratio for ndt diagnostics of aerospace grade composites. Sensors 2020, 20, 3381. [Google Scholar] [CrossRef]
  68. Livada, B.; Perić, D. The influence of the water on scene IR signature. In Proceedings of the Target and Background Signatures V, Strasbourg, France, 17 October 2019; Volume 11158, pp. 129–138. [Google Scholar] [CrossRef]
  69. Handcock, R.N.; Torgersen, C.E.; Cherkauer, K.A.; Gillespie, A.R.; Tockner, K.; Faux, R.N.; Tan, J. Thermal infrared remote sensing of water temperature in riverine landscapes. In Fluvial Remote Sensing for Science and Management; Carbonneau, P.H., Piégay, H., Eds.; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2012; pp. 85–113. [Google Scholar]
  70. Chen, H.-Y.; Chen, C. Determining the emissivity and temperature of building materials by infrared thermometer. Constr. Build. Mater. 2016, 126, 130–137. [Google Scholar] [CrossRef]
  71. Lisiecka, E. Reduction of the impact of emissivity on high temperature measurements in non-contact thermometric devices. Opt. Appl. 2017, 47, 373–381. [Google Scholar] [CrossRef]
  72. ASTM E1933-14 (2018); Standard Practice for Measuring and Compensating for Emissivity Using Infrared Imaging Radiometers. American Society for Testing and Materials: West Conshohocken, PA, USA, 2018.
  73. Bouguet, J.-Y. Camera Calibration Toolbox for Matlab. Computational Vision at the California Institute of Technology. 2008. Available online: https://www.mathworks.com/help/vision/ug/using-the-single-camera-calibrator-app.html (accessed on 14 February 2023).
Figure 1. A schematic representation of a typical checkerboard with (a) no distortion, (b) barrel distortion and (c) pincushion distortion.
Figure 1. A schematic representation of a typical checkerboard with (a) no distortion, (b) barrel distortion and (c) pincushion distortion.
Sensors 23 03479 g001
Figure 2. A schematic representation of the camera calibration setup following Zhang’s model [58] by mapping the 3D world scene into an image plane using a randomly oriented planar checkerboard target.
Figure 2. A schematic representation of the camera calibration setup following Zhang’s model [58] by mapping the 3D world scene into an image plane using a randomly oriented planar checkerboard target.
Sensors 23 03479 g002
Figure 3. Optical images of (a) the laser-engraved and (b) the UV-printed checkerboard patterns.
Figure 3. Optical images of (a) the laser-engraved and (b) the UV-printed checkerboard patterns.
Sensors 23 03479 g003
Figure 4. Checkerboard targets fabricated using (a) SLTL’s Akshar Fiber Pro Ultra laser engraving machine and (b) Canon’s Arizona 1380 XT flatbed UV printer.
Figure 4. Checkerboard targets fabricated using (a) SLTL’s Akshar Fiber Pro Ultra laser engraving machine and (b) Canon’s Arizona 1380 XT flatbed UV printer.
Sensors 23 03479 g004
Figure 5. A presentation of the experimental setup used in this study.
Figure 5. A presentation of the experimental setup used in this study.
Sensors 23 03479 g005
Figure 6. Laser-engraved pattern (a) optical and (b) thermal images and UV-printed pattern (c) optical and (d) thermal images.
Figure 6. Laser-engraved pattern (a) optical and (b) thermal images and UV-printed pattern (c) optical and (d) thermal images.
Sensors 23 03479 g006
Figure 7. Thermal images of the laser-engraved checkerboard pattern viewed at random orientations for the IR camera geometric calibration.
Figure 7. Thermal images of the laser-engraved checkerboard pattern viewed at random orientations for the IR camera geometric calibration.
Sensors 23 03479 g007
Figure 8. Thermal images of the UV-printed checkerboard pattern viewed at random orientations for the IR camera geometric calibration.
Figure 8. Thermal images of the UV-printed checkerboard pattern viewed at random orientations for the IR camera geometric calibration.
Sensors 23 03479 g008
Figure 9. Distribution of the reprojection errors ( RE ) of all detected image points, using one calibration trial per selected checkerboards. (a) Laser-engraved checkerboard; (b) UV-printed checkerboard.
Figure 9. Distribution of the reprojection errors ( RE ) of all detected image points, using one calibration trial per selected checkerboards. (a) Laser-engraved checkerboard; (b) UV-printed checkerboard.
Sensors 23 03479 g009
Figure 10. Histogram of (a) mean reprojection errors ( MRE ) and (b) root mean square errors ( RMSE ) of reprojection, using thermal images of calibration trial number 9 per selected checkerboards.
Figure 10. Histogram of (a) mean reprojection errors ( MRE ) and (b) root mean square errors ( RMSE ) of reprojection, using thermal images of calibration trial number 9 per selected checkerboards.
Sensors 23 03479 g010
Table 1. Summary of reported thermal camera geometric calibration targets. Pattern: S (square) and C (circular); condition: I (indoor) and O (outdoor); εminmax: (lowest/highest emissivity coefficients of pattern); heating: N/A (not available) and NI (no information); RE (reprojection error); RMSE (root mean square error of reprojection); MRE (mean reprojection error).
Table 1. Summary of reported thermal camera geometric calibration targets. Pattern: S (square) and C (circular); condition: I (indoor) and O (outdoor); εminmax: (lowest/highest emissivity coefficients of pattern); heating: N/A (not available) and NI (no information); RE (reprojection error); RMSE (root mean square error of reprojection); MRE (mean reprojection error).
PatternMakeCondition—HeatingεminmaxRE [Pixels]Optical ImageThermal Image
[27]—SPaper-printed checkerboard patternI—Flood lampSensors 23 03479 i001Sensors 23 03479 i002
[28]—SPaper-printed checkerboard patternI—Flood lampSensors 23 03479 i003Sensors 23 03479 i004
[29]—SPaper-printed checkerboard pattern taped on a ceramic tileI—250 W Flood lampRMSE: 0.410–0.480Sensors 23 03479 i005Sensors 23 03479 i006
[30]—SCheckerboard pattern printed on matt paper and taped to a wallI—IR emitting lampRMSE: 0.600–0.714Sensors 23 03479 i007
[31]—SPaper-printed checkerboard patternI—500 W Flood lampMRE: 0.804–1.274Sensors 23 03479 i008
Cardboard mask of regularly cut squaresI—Monitor displayMRE: 0.284–0.324Sensors 23 03479 i009
[32]—SBlack K-Line board of regularly cut squaresI—N/ASensors 23 03479 i010Sensors 23 03479 i011
[33]—SPaper-printed checkerboard pattern with aluminum foil taped on the white squaresI—N/ASensors 23 03479 i012
[34]—SPaper-printed checkerboard pattern with aluminum foil taped on the white squaresI—RadiatorSensors 23 03479 i013Sensors 23 03479 i014
[35]—SLaser-printed checkerboard pattern on a thin aluminum sheetI—NISensors 23 03479 i015Sensors 23 03479 i016
[25]—SDigitally printed checkerboard pattern on a thin sandblasted aluminum plateI—Halogen lamp
O—Sunlight
0.21/-RMSE: 0.200–0.370
RMSE: 0.100–0.220
Sensors 23 03479 i017Sensors 23 03479 i018
[6]—SUV-printed checkerboard pattern on a Dibond (aluminum composite) panelI—Electric fan heater0.67/0.83RMSE: 0.233Sensors 23 03479 i019Sensors 23 03479 i020
[36]—SHigh emissivity squares tape placed on a thin aluminum plate
Low emissivity aluminum foil squares tape placed on cardboard
I—N/A
I—Monitor display
O—Clear cold sky
Sensors 23 03479 i021Sensors 23 03479 i022
[37]—SHigh emissivity insulating squares tape placed on a polished aluminum plateI—External heaterSensors 23 03479 i023
Low emissivity aluminum foil squares tape placed on a cardboardI—Monitor displaySensors 23 03479 i024
[38]—SHigh emissivity squares tape placed on an aluminum foil baseI—NISensors 23 03479 i025Sensors 23 03479 i026
[19]—SPrecisely cut thick thermocol squares coated with white glue along with thick aluminum foil squares adhered to a thick thermocol boardI—N/A0.03/0.99MRE: 0.310–0.400Sensors 23 03479 i027Sensors 23 03479 i028
[39]—SPainted black and white aluminum alloy squares forming a checkerboard pattern and fixed to a wallI—Peltier moduleMRE: 0.230Sensors 23 03479 i029Sensors 23 03479 i030
[40]—SPainted black and white aluminum alloy squares forming a checkerboard pattern and fixed to a wallI—Peltier moduleRMSE: 0.280Sensors 23 03479 i031Sensors 23 03479 i032
[41]—SMilled copper squares on a printed circuit board (PCB) baseI—Hair dryerSensors 23 03479 i033Sensors 23 03479 i034
[42]—SHigh emissivity black squares painted on a low emissivity copper base plateO—Sunlight0.09/0.98MRE: 0.480–0.650Sensors 23 03479 i035Sensors 23 03479 i036
Low emissivity copper squares on a high emissivity black base plateO—Sunlight0.09/0.98MRE: 0.330–0.450Sensors 23 03479 i037Sensors 23 03479 i038
[43]—SPrinted polished copper checkerboard pattern on an epoxy plateI—N/A
I—Hot ceramic plate
I—Ice block
Sensors 23 03479 i039
[44]—SMetal square net placed on a plastic boardI—Heat gunMRE: 0.175–0.177Sensors 23 03479 i040Sensors 23 03479 i041
[45]—CCircular burning lamps fixed on a wooden plankI—Burning lampsSensors 23 03479 i042
[46]—CMiniature bulbs attached to a black plastic boardI—Light bulbsMRE: 0.588Sensors 23 03479 i043Sensors 23 03479 i044
[47]—CPrinted checkerboard pattern mounted on a metallic plate with IR-emitting resistors attached at the centers of the squares, forming a circular calibration targetI—Electric resistorsO—Electric resistorsSensors 23 03479 i045Sensors 23 03479 i046
[48]—CPrinted black asymmetrical circular pattern on a white Dibond (aluminum composite) boardO—SunlightRMSE: 0.348Sensors 23 03479 i047
[6]—CUV-printed symmetrical circular pattern on a Dibond (aluminum composite) panelI—Electric fan heater0.67/0.83RMSE: 0.052Sensors 23 03479 i048Sensors 23 03479 i049
[49]—CUV-printed symmetrical circular pattern on a Dibond (aluminum composite) panelO—Sunlight0.67/0.83Sensors 23 03479 i050Sensors 23 03479 i051
[50]—CAnodized perforated Al-6061 symmetrical circular grid placed on a black coated baseI—Electric heater0.04/0.95Sensors 23 03479 i052Sensors 23 03479 i053
[51]—CBlack perforated symmetrical circular cardboard grid placed on top of a wooden plank wrapped in aluminum foilI—N/A-/0.94Sensors 23 03479 i054Sensors 23 03479 i055
[52]—CLaser-cut asymmetrical circular cardboard patternI—Heat gun-/0.81MRE: 0.500Sensors 23 03479 i056Sensors 23 03479 i057
[53]—CLaser-cut asymmetrical circular acrylic patternI—Heat gunSensors 23 03479 i058Sensors 23 03479 i059
Table 2. Physical and thermal properties of the SS304 calibration plate.
Table 2. Physical and thermal properties of the SS304 calibration plate.
PropertiesValues [65]
Physical propertiesdensity ( ρ ) 7900   kg / m 3
Thermal propertiesthermal conductivity ( k ) 16.3   W / m · K
heat capacity ( c p ) 500   J / kg · K
linear thermal expansion ( α L ) 16.6 × 10 6   mm / mm · K   @   20 100   K
Target dimensionslength 150   mm
width 136   mm
thickness 6   mm
Table 3. Technical specifications of the thermographic camera used in this research.
Table 3. Technical specifications of the thermographic camera used in this research.
Infrared Camera Specifications
ModelInfraTec VarioCAM HD Head-600
Spectral range7.5–14 μm
Temperature measuring range 40 °C to + 1200 °C
Temperature sensitivity 0.03 °C at + 30 °C
Measurement accuracy ± 1.5 °C
Detector 640 × 480 UFPA
Image framerate 60   Hz
Table 4. Emissivity coefficients ( ε ) of the calibration targets.
Table 4. Emissivity coefficients ( ε ) of the calibration targets.
Laser-EngravedUV-Printed
Black squares0.8110.931
Bare metal squares0.1720.239
Table 5. List of reprojection errors ( MRE   and   RMSE ) for the two calibration targets.
Table 5. List of reprojection errors ( MRE   and   RMSE ) for the two calibration targets.
Laser-EngravedUV-Printed
Trials
MRE   ( Pixels )
RMSE   ( Pixels )
MRE   ( Pixels )
RMSE   ( Pixels )
10.13660.16230.05380.0602
20.13730.16380.05440.0605
30.12540.15000.05320.0591
40.12160.14530.05790.0646
50.12180.14560.05580.0622
60.11530.13730.05530.0617
70.10660.12650.05920.0661
80.10740.12770.05930.0660
90.09820.11720.05920.0662
Mean0.11890.14170.05650.0630
Standard deviation0.01340.01600.00250.0028
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

ElSheikh, A.; Abu-Nabah, B.A.; Hamdan, M.O.; Tian, G.-Y. Infrared Camera Geometric Calibration: A Review and a Precise Thermal Radiation Checkerboard Target. Sensors 2023, 23, 3479. https://doi.org/10.3390/s23073479

AMA Style

ElSheikh A, Abu-Nabah BA, Hamdan MO, Tian G-Y. Infrared Camera Geometric Calibration: A Review and a Precise Thermal Radiation Checkerboard Target. Sensors. 2023; 23(7):3479. https://doi.org/10.3390/s23073479

Chicago/Turabian Style

ElSheikh, Ahmed, Bassam A. Abu-Nabah, Mohammad O. Hamdan, and Gui-Yun Tian. 2023. "Infrared Camera Geometric Calibration: A Review and a Precise Thermal Radiation Checkerboard Target" Sensors 23, no. 7: 3479. https://doi.org/10.3390/s23073479

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop