Deep Quantified Visibility Estimation for Traffic Image

Abstract

Image-based quantified visibility estimation is an important task for both atmospheric science and computer vision. Traditional methods rely largely on meteorological observation or manual camera calibration, which restricts its performance and generality. In this paper, we propose a new end-to-end pipeline for single image-based quantified visibility estimation by an elaborate integration between meteorological physical constraint and deep learning architecture design. Specifically, the proposed Deep Quantified Visibility Estimation Network (abbreviated as DQVENet) consists of three modules, i.e., the Transmission Estimation Module (TEM), the Depth Estimation Module (DEM), and the Extinction coEfficient Estimation Module (E3M). Casting on these modules, the meteorological prior constraint can be combined with deep learning. To validate the performance of DQVENet, this paper also constructs a traffic image dataset (named QVEData) with accurate visibility calibration. Experimental results compared with many state-of-the-art methods on QVEData demonstrate the effectiveness and superiority of DQVENet.

1. Introduction

Quantified visibility estimation is of great significance in applications such as air safety [1], ground transport control [2], as well as air quality assessment [3]. Recognizing the current visibility state has been an urgent need for many areas [4,5] and plays a key role in machine learning [6].

Typical visibility estimation methods rely largely on the professional meteorological stations with expensive sensors and human observations. However, limited by the manufacturing costs, these stations are often distributed in a non-uniform way, which in turn, restricts the capability of accurate visibility estimation. Recently, with the widespread use of mobile cameras, thousands of images have been collected under different weather conditions, which might provide an alternative solution for quantified visibility estimation.

In fact, many researchers have tried to explore the possibility of estimating visibility from single images in the past decades [7,8,9,10,11,12]. Hautiére et al. [7] propose a probabilistic model-based approach, which takes into account the distribution of contrasts in the scene. Thus, the proposed model is more robust to illumination variations. Though helpful, this method is still limited in ability to deal with cameras captured in daylight. In order to improve the performance during night, Varjo et al. [8] propose a method based on feature vectors that are projections of the scene images with lighting normalization. Li et al. [9] estimate the extinction coefficient in a clear atmosphere by assuming it to be approximately a constant. Combined with the dark channel prior [13], the ratio of two extinction coefficients in the current and clear atmosphere can be calculated, and the quantified visibility can then be obtained. It is further improved by employing an edge collapse-based transmission refinement [10].

With the rapid development of visibility estimating methods, one notable issue is that the precise visibility label is difficult to obtain, and the not necessarily accurate visibility label has agreement on image-based visibility estimation. On one hand, Song et al. [11] solve this by associating a label distribution with each image. The label distribution contains all of the possible visibilites with corresponding probabilities. After that, typical machine learning-based methods can be employed. On the other hand, Xun et al. [12] use the extra ordinal information and relative relation of images for visibility estimation. They pre-train a model using classified outdoor foggy images and then fine-tune the model via indoor synthetic continuous annotation.

Though effective, most of the former mentioned methods confront the following problems. Under the influence of camera angle, previous methods need large efforts on parameter calibration, which (a) restricts the model generalization and increases the model costs. Moreover, (b) the image scale is also sensitive. For instance, some cameras focus on local road details (small scale), while others concentrate on global information (large scale). Apart from that, the precedent visibility estimation approaches exploit (c) either data-driven or physical-constrained techniques. We believe the physical constraint plays a key role in visibility estimation and the data-driven strategy is also important in future work.

Consequently, to solve these issues and achieve quantified visibility estimation from images, this paper proposes a new network architecture, named Deep Quantified Visibility Estimation Network (abbreviated DQVENet), especially for traffic images. The contributions of this paper are summarized as follows.

• By integrating physical constraint with deep learning network, a novel framework named DQVENet is proposed for single image visibility estimation.
• Within this framework, a Transmission Estimation Module (TEM), a Depth Estimation Module (DEM), and an Extinction Coefficient Estimation Module (E3M) is unified as a whole according to meteorological theory.
• A new benchmark dataset, which is especially designed for traffic image-based quantified visibility estimation, is constructed.

2. Related Work

From the perspective of machine learning, image-based quantified visibility estimation is equal to learning a regression model. This paper focuses on deep learning-based visibility estimation. As a result, this section first presents a short investigation of deep learning methods followed by discussing single image visibility estimation. Considering the great similarity between visibility estimation and weather recognition, we give a brief introduction to it. Furthermore, taking the fact that visibility can be estimated via dehazing techniques into consideration, single image-based haze removal is also discussed.

2.1. Deep Learning

Since the emergence of AlexNet [14] in computer vision, deep learning has attracted attention among areas such as speech recognition [15], natural language processing [16], video understanding [17], etc. Recently, it also shows great potential in atmospheric science. For example, by a thorough combination of materials such as satellite, radar, and ground station, Sønderby et al. [18] propose MetNet for precipitation forecasting. Results show that it is superior to pure numerical weather prediction. For getting precise and accurate prediction, Yu et al. [19] proposes an auxiliary guided spatial distortion network. However, few existing methods take both a deep learning architecture design and meteorological theory into account, which might be a possible avenue for future research. To achieve that, Kuang et al. [20] introduce three branches corresponding to the temperature variation equation in deep learning and propose a new method for temperature forecasting.

2.2. Image-Based Visibility Estimation

Visibility-related studies have been a hot research topic for decades. Basically, both the parameter settings of the camera and the environmental conditions highly affect image quality, which influences estimation accuracy. To solve this issue, Li et al. [21] employ a pre-trained convolutional neural netwok to extract the visibility features automatically instead of manual extraction. Giyenko et al. [22] also implement a simple but useful three-layer network for visibility estimation and trained it on a dataset collected in South Korea. Palvanov et al. [23] propose a new approach based on deep integrated convolutional neural networks for the estimation of visibility distances from camera imagery. This network uses three paralleled streams of deep integrated convolutional neural networks.

2.3. Weather Recognition from Images

Apart from quantified visibility estimation, recognizing the weather conditions from images is also of great significance. Under the framework of pattern recognition, a possible solution is treating weather recognition as image classification [6]. Casting on this mechanism, there are methods concentrating on extracting discriminative features such as global histogram [24] as input for classification methods such as support vector machine [25]. With the overwhelming successes of deep learning, many brilliant methods have also been proposed [24,26,27].

2.4. Single Image Haze Removal

There is a deterministic correlation between the fog image and clear image, and the visibility can be mathematically calculated through an investigation of these two images. Consequently, there are also methods estimating image visibility via dehazing techniques. Specifically, Zhou et al. [28] propose a visibility estimation method based on dark channel prior and image entropy. The dark channel prior is used to estimate atmospheric transmittance and optimized via guided filter. The road region is extracted based on a region growing algorithm. After that, the depth map is calculated using lane information and the haze visibility is obtained based on the minimum image entropy of road images. Bae et al. [29] propose a visibility distance estimation method using dark channel prior-based single image haze removal. Hence, the dark channel for an input sea-fog image is first calculated. The binary transmission image is obtained by applying a threshold to the estimated transmission from the dark channel. Then, the sum of the distance values of pixels, corresponding to the sea-fog boundary, is averaged, in order to derive the visibility distance. Furthermore, image dehazing is a highly ill-posed problem [30] and various methods have been developed to tackle this problem. Specifically, the key ingredient for haze removal is the estimation of the transmission map. Except for dark priors [13], Cai et al. [31] introduce an end-to-end network for estimating it with a novel activation unit. Ren et al. [32] and Li et al. [33] also provide effective frameworks for transmission map estimation.

3. Materials

3.1. Motivation

Quantified visibility estimation is quite important to traffic control. Nevertheless, there is few datasets focusing on this, which limits the developing of estimation methods. To illustrate this key issue and demonstrate the effectiveness of the proposed method, this paper provides a new dataset named QVEData (abbreviation for Quantified Visibility Estimation Dataset).

3.2. Dataset Construction

Images are first obtained via real high-speed road cameras and 24,031 traffic images are collected during this stage. The corresponding quantified visibility observation are acquired using the nearest meteorological station. Basically, five cameras (denoted as C1, C2, C3, C4, and C5) are employed and the distances between the camera and the nearest visibility station are listed as following

Table 1. Distances between the camera and the nearest visibility station.

Camera Id	C1	C2	C3	C4	C5
Distance	450 m	402 m	268 m	501 m	171 m

.

Unfortunately, most of these images have high similarity and the corresponding visibility observations tend to be consistent. After similarity eliminating and manual quality control, 3236 images are finally preserved for constructing the final dataset. We denote this dataset Quantified Visibility Estimation Dataset (abbreviated as QVEData). Considering the fact that this paper is devoted to visibility estimation of traffic images, the selected cameras are along a high-speed road. Therefore, the road is the main content of the image. Nevertheless, due to the variation of traffic cameras, camera angles, weather conditions, illuminations, moving vehicles, etc., the finally preserved 3326 images are quite different, which increases the difficulty of visibility estimation. Figure 1 presents the visibility distribution of QVEData. Specifically, most of the corresponding visibility lies in the interval of 0–20 km, with a focus on 0–10 km.

3.3. Comparison with Other Datasets

Generally, there are many significant fog-related datasets.

Table 2. Comparison of different fog-related datasets.

Task	Name	Year	Reference	Property
Image Dehazing	D-Haze	2016	[34]	indoor, synthetic images
	I-Haze	2018	[35]	indoor, real images
	O-Haze	2018	[36]	outdoor, real images
	RESIDE	2018	[37]	indoor and outdoor, synthetic and real images
	Dense-Haze	2019	[38]	outdoor, synthetic images
	Haze4K	2021	[39]	synthetic images, with clean image, transmission map, and atmospheric light as groundtruth
	NH-Haze	2020	[40]	outdoor, real, non-homogeneous images
	REVIDE	2021	[41]	real, videos
Weather Recognition	MWI	2015	[42]	Sunny, Rainy, Snowy, Haze
	Image2Weather	2017	[43]	Sunny, Cloudy, Snowy, Rainy, Foggy, Others
	MWD	2017	[44]	Cloudy, Rainy, Snowy, Haze, Thunder, Sunny
	RFS	2018	[27]	Rain, Fog, Snow
	Five-class	2019	[45]	Sunny, Cloudy, Foggy, Rainy, Snowy
	TWData	2020	[6]	Sunny, Fog50, Fog200, Fog500, RoadSnow, RoadWet, RoadIce
Visibility Estimation	WILD	2002	[46]	real images
	QVEData	2022	ours	real traffic images, with professional meteorological calibration

gives an intuitive comparison among these datasets and the proposed QVEData. Specifically, we divide these datasets into three categories according the corresponding tasks, i.e., image dehazing, weather recognition, and visibility estimation.

4. Methods

4.1. Preliminary

For a given hazy image I, it is degraded due to the presence of haze, which can be mathematically formulated as

$$\begin{array}{c}\hfill I\left(x\right)=J\left(x\right)t\left(x\right)+A\left(x\right)(1-t(x\left)\right)\end{array}$$

(1)

where J is the groundtruth image (without haze), t is the transmission map, A is the global atmospheric light, and x is the pixel position. Specifically, the transmission map t can be expressed as

$$\begin{array}{c}\hfill t\left(x\right)=e^{-\beta d\left(x\right)}\end{array}$$

(2)

where $\beta$ represents the atmospheric extinction coefficient and d is the scene depth.

According to Koschmieder’s law, the visibility can be expressed as a function of the atmospheric extinction coefficient, i.e.,

$$\begin{array}{c}\hfill C=e^{-\beta V}\end{array}$$

(3)

where C is a threshold contrast and V is the visibility in metres. In other words, the visibility can be quantified as

$$\begin{array}{c}\hfill V=-\frac{lnC}{\beta }\end{array}$$

(4)

Typically, the threshold C is set to be 0.05. Consequently, Equation (4) can be approximately formulated as

$$\begin{array}{c}\hfill V=\frac{3}{\beta }\end{array}$$

(5)

For Equation (2), we can rewrite it as

$$\begin{array}{c}\hfill \beta =\frac{-lnt\left(x\right)}{d\left(x\right)}\end{array}$$

(6)

Consequently, for a given position, the corresponding visibility can be roughly estimated as

$$\begin{array}{c}\hfill V=-\frac{3d\left(x\right)}{lnt\left(x\right)}\end{array}$$

(7)

by integrating Equation (6) to Equation (5).

4.2. Overall Architecture

From Equation (7), the visibility V is highly correlated with transmission map t and scene depth d. This motivates us to design a network that is physically constrained by Equation (7). As a result, this paper proposes Deep Quantified Visibility Estimation network (abbreviated as DQVENet), an elaborately designed convolutional architecture, especially for single image-based quantified visibility estimation. The overall architecture can be found in Figure 2.

Following the working mechanism of Equation (5), DQVENet mainly consists of three basic modules, i.e., the Transmission Estimation Module (TEM), the Depth Estimation Module (DEM), and the Extinction coEfficient Estimation Module (E3M). We then present the details of each module.

4.3. Transmission Estimation Module

To estimate the transmission map from a single image, previous methods [30,32,47] often use the multi-level features. DQVENet employs the densely connected pyramid network [30] as the Transmission Estimation Module (TEM). Specifically, TEM adopts a densely connected UNet [30,48] structure for feature encoder-decoder.

Denote the input image as $X\in {\mathbb{R}}^{w\times h\times c}$, where w, h, and c represent the width, height, and the number of channels of X. It is first processed by a $3\times 3$ convolution with stride 2, followed by batch normalization, ReLU activation, and Max-pooling, which can be mathematically formulated as

$$\begin{array}{c}\hfill X_{TEM}^0=\mathrm{MaxPool}\left(\mathrm{ReLU}\left(\mathrm{BN}\left({\mathrm{Conv}}_{3\times 3}\left(X\right)\right)\right)\right)\end{array}$$

(8)

Here, $X_{TEM}^0\in {\mathbb{R}}^{\frac{w}{4}\times \frac{h}{4}\times c0}$ is the pre-processed feature for the following cascaded encoder-decoder. Without loss of generality and for simplicity, we still use X to represent the input feature in the following.

The encoder comprises of both a dense block and a transition block. For dense block, it contains two sequential batch normalization—ReLU activation—convolution layers. Mathematically, it can be defined as

$$\begin{array}{c}\hfill X_{dense}^{E1}={\mathrm{Conv}}_{1\times 1}\left(\mathrm{ReLU}\left(\mathrm{BN}\left(X\right)\right)\right)\\ \hfill X_{dense}^{E2}={\mathrm{Conv}}_{3\times 3}\left(\mathrm{ReLU}\left(\mathrm{BN}\left(X\right)\right)\right)\end{array}$$

(9)

For the transition block, it contains an additional Max-pooling layer for enlarging receptive fields compared with the dense block, i.e.,

$$\begin{array}{c}\hfill X_{trans}^{E1}=\mathrm{MaxPool}\left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{ReLU}\left(\mathrm{BN}\left(X\right)\right)\right)\right)\end{array}$$

(10)

Similar to the encoder, the decoder also involves a dense block and a transition block as

$$\begin{array}{cc}\hfill & X_{dense}^{D1}={\mathrm{Conv}}_{1\times 1}\left(\mathrm{ReLU}\left(\mathrm{BN}\left(X\right)\right)\right)\hfill \\ \hfill & X_{dense}^{D2}={\mathrm{Conv}}_{3\times 3}\left(\mathrm{ReLU}\left(\mathrm{BN}\left(X\right)\right)\right)\hfill \\ \hfill & X_{trans}^{D1}=\mathrm{Upsample}\left({\mathrm{TransConv}}_{1\times 1}\left(\mathrm{ReLU}\left(\mathrm{BN}\left(X\right)\right)\right)\right)\hfill \end{array}$$

(11)

We should note that the stride of the transpose convolution is set to 1, and the up-sampling operation is indeed achieved by the nearest interpolation. Moreover, the encoder feature at different levels are also fed to the corresponding decoder layer for precise feature extraction.

For getting global structural information at different scales, the multi-level pyramid pooling block is employed. Specifically, four Max-pooling operations with size $\frac{1}{4}$, $\frac{1}{8}$, $\frac{1}{16}$, and $\frac{1}{32}$ are used, i.e.,

$$\left\{\begin{array}{c}{X}_{pool}^1=\mathrm{Upsample}\left(\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}\left({\mathrm{MaxPool}}_4\left(X\right)\right)\right)\right)\hfill \\ X_{pool}^2=\mathrm{Upsample}\left(\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}\left({\mathrm{MaxPool}}_8\left(X\right)\right)\right)\right)\hfill \\ X_{pool}^3=\mathrm{Upsample}\left(\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}\left({\mathrm{MaxPool}}_{16}\left(X\right)\right)\right)\right)\hfill \\ X_{pool}^4=\mathrm{Upsample}\left(\mathrm{ReLU}\left({\mathrm{Conv}}_{1\times 1}\left({\mathrm{MaxPool}}_{32}\left(X\right)\right)\right)\right)\hfill \end{array}\right.$$

(12)

and they are concatenated for estimating the final transmission map $X_t$ using Tanh activation as

$$\begin{array}{cc}\hfill & X_t=\mathrm{Tan}\mathrm{h}\left({\mathrm{Conv}}_{1\times 1}\left(\mathrm{concat}\left([X_{pool}^1,X_{pool}^2,X_{pool}^3,X_{pool}^4]\right)\right)\right)\hfill \end{array}$$

(13)

4.4. Depth Estimation Module

Depth estimation from a single image has been explored through various approaches such as non-parametric scene sampling [49], supervised end-to-end learning [50], semi-supervised estimation [51], and generative adversarial prediction [52].

Taking the fact that obtaining ground-truth scene depth from unconstrained images is relatively hard into consideration, DQVENet focuses on seeking the approximated scene depth in an unsupervised or self-supervised manner. Specifically, the pre-trained self-supervised scene estimation method proposed by Godard et al. [53] is employed as the Depth Estimation Module (DEM).

Basically, the DEM consists of several ResNet [54]-based encoder-decoder blocks, i.e., Res-UNet [55]. Compared with vanilla UNet [48] or other backbones, taking Res-UNet as our quantified visibility-oriented depth estimation, i.e., DEM, has the following advantages.

• It contains more layers with more parameters. Consequently, it can describe the more abstract high-level features accurately.
• The structure is more sophisticated. In application, the images can be obtained at any place, any time, and any view, which increases the difficulty of image recognition. This architecture promotes the corresponding generalization.
• Benefiting from the residual connection, training such a network is supposed to be more easy with less risk of gradient vanishing.

4.5. Joint Atmospheric Extinction Coefficient and Quantified Visibility Estimation

Denoting the estimated transmission map and the depth map as $X_t\in {\mathbb{R}}^{w\times h\times 3}$ and $X_d\in {\mathbb{R}}^{w\times h\times 1}$, respectively, the atmospheric extinction coefficient can be obtained according to Equation (6) and the quantified visibility estimation can be further achieved via Equation (5).

However, imposing $X_t$ and $X_d$ to Equation (6) directly confronts the following two main inconveniences.

• Due to the lack of labeled transmission and depth maps, we cannot fine-tune the TEM and DEM on our own data or other weather images. Consequently, we only use the pre-trained model and the pre-obtained transmission map $X_t$ and depth map $X_d$ are relatively coarse, which will restrict the performance of quantified visibility estimation.
• The two modules, i.e., TEM and DEM, work separately. Limited by this strategy, the obtained transmission map and depth map are relatively independent. Nevertheless, for a given image, these items should be highly correlated. Moreover, bringing $X_t$ and $X_d$ into Equation (6) makes the network not end-to-end, which in turns constraints the efficiency.

To overcome the former illustrated problems, this paper attempts to integrate the TEM and DEM together for joint atmospheric extinction coefficient and quantified visibility estimation. As a result, the DenseNet [56]-based Extinction coEfficient Estimation Module (E3M) with joint quantified visibility estimation is introduced.

For E3M, the estimated transmission map $X_t$ and depth map $X_d$ are first concatenated as the input, and then processed via a pre-trained 121-layer DenseNet by replacing the last classification layer with a regression layer. The number of channel inputs for the first convolutional layer is also set to 4 instead of the original 3. This makes the whole network end-to-end trainable, and we denote this architecture DQVENet.

4.6. Iterative Training

For getting better performance, this paper employs an iterative training strategy. To be specific, both TEM and DEM are first pre-trained using the corresponding dehaze dataset [30] and depth estimation dataset [53]. After that, these two modules are frozen and combined to the DQVENet for training E3M. Finally, the whole DQVENet is fine-tuned with a relatively small learning rate.

DQVENet is implemented using Pytorch (https://pytorch.org/, accessed on 28 November 2022). During pre-training of TEM and DEM, we use the same setting as the original paper. For E3M, the stochastic gradient descent optimizer with learning rate $1\times {10}^{-4}$ is employed. The learning rate drops by 0.1 during epoch 150 and 225. The early stopping strategy is also implemented, i.e., training stops after 300 epochs or the validation loss does not decrease within 10 epochs. As for fine-tuning DQVENet, a small learning rate $1\times {10}^{-5}$ is employed.

5. Results

To validate the performance of the proposed DQVENet, this section conducts experiments on QVEData. Specifically, there are a total of 3236 images for QVEData. We split these images randomly into training, validation, and testing with ratio 60%, 10%, and 30%, respectively. Casting on this, these experiments can be mainly divided into two categories, i.e., qualitative results and quantitative results.

5.1. Qualitative Results

Figure 3 first presents an intuitive illustration of the learned depth map and transmission map with their corresponding original image. For depth map, the green area represents short depth and the blue area denotes long depth. While for transmission map, the dark area indicates low transmissivity and the light area stands for high transmissivity.

From this figure, both the estimated depth map of DEM and the estimated transmission map of TEM can describe the scene depth and transmission map to some extent. This is mainly due to the reason that these two modules are pre-trained using the corresponding dataset.

5.2. Quantitative Results

Figure 4 then shows the quantified visibility estimation of DQVENet and the ground truth visibility observation. To be specific, the first two rows present examples that DQVENet is underestimated. The third row gives results of DQVENet being overestimated; the last row shows the instances that DQVENet fits groundtruth observations well.

Figure 5 presents the DQVENet results and the groundtruth visibility observation on the testing set of QVEData. Specifically, the 971 testing images are first sorted from small to large according to their groundtruth visibility, i.e., the orange line. Then, the corresponding predictions are plotted along with the groundtruth, i.e., the blue line. Finally, we also present the soft gap, i.e., the red area, for better demonstrating the effectiveness of DQVENet. Except for some overestimated cases, DQVENet can fit the visibility well in general.

Furthermore, Figure 6 shows the correlation analysis of DQVENet. Within this scatter figure, the DQVENet Prediction is treated as the x-axis, and the groundtruth observation is the y-axis. The more dots converge at the middle red line, the better the performance of the method is. Nevertheless, we cannot restrict the scatters to lie in a fixed gap shown in the following Figure 6a. Instead, a proportional gap, as in the following Figure 6b, might be more feasible. From this figure, the DQVENet is capable of describing the tendency of visibility estimations.

Generally, for meteorological forecasting or machine learning-based regression, the Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), Bias Error, and Correlation are supposed to be significant indicators. Nevertheless, for quantified visibility estimation, the former three criterions, i.e., MAE, RMSE, and Bias, are not suitable. MAE, RMSE, or Bias for different visibility groundtruths are not comparable. In other words, if the groundtruth is 20,000 m, the estimated results within (19,000 m, 21,000 m) are acceptable for a traffic manager. However, if the groundtruth is 1000 m, the desired results should be in the range (900 m, 1100 m). As a result, we put these criterions aside and apply Correlation as a basic criterion due to the fact that correlations are overall statistical results instead of point-to-point comparisons. Suppose the predicted visibility is $Y_p$ and the groundtruth visibility is Y, then it is defined as

$$\begin{array}{c}\hfill Corr(Y,Y_p)=\frac{E\left[(Y-{\mu }_Y)(y_p-{\mu }_{Y_p})\right]}{{\sigma }_Y{\sigma }_{Y_p}}\end{array}$$

(14)

The larger the correlation coefficient, the better the model performance. The correlation coefficient of DQVENet is 0.7237, demonstrating that DQVENet is competent to quantify visibility estimation.

6. Discussion

For further illustrating the effectiveness of DQVENet, this section compares DQVENet with several state-of-the-art methods, including the ResNet family [54] and the EfficientNet family [57]. The classification accuracy is discussed.

To demonstrate the performance of the estimation method, we define the classification accuracy as

$$\begin{array}{c}\hfill Ac{c}_t=\frac{\mathcal{N}\left(\right|Y_p-Y|<Y\times t)}{\mathcal{N}\left(Y\right)}\times 100\%\end{array}$$

(15)

Here, $Y_p$ and Y denote the predicted and groundtruth visibility, respectively. $|·|$ denotes the absolute value, $\mathcal{N}(·)$ represents the number. $0<t<1$ is a threshold that can be defined arbitrarily, and $Ac{c}_t$ is the accuracy according to t.

In other words, the classification accuracy under threshold t is the ratio between the number of samples that lie in the interval $[Y-Y\times t,Y+Y\times t]$ and the total number of samples. We denote this accuracy as a soft gap. This soft gap differs from the hard gap, which splits visibilities hardly into intervals such as [0–100 m], [100–500 m], etc., due to the fact that this paper focuses on quantified visibility estimation instead of classification. More specifically, if the groundtruth visibility is 200 m, the estimated 100 m will be regarded as better than 200.1 m for the hard gap. Nevertheless, within this soft partition, 200.1 m will be treated as a preferred estimation than 100 m. Detailed results can be found in the following Figure 7.

From this figure, the classification accuracy of all compared methods increases with the threshold being larger. In general, DQVENet outperforms the ResNet family (including ResNet-50, ResNet-101, and ResNet-152) and the EfficientNet family (including EfficientNet-B0, EfficientNet-B1, EfficientNet-B2, and EfficientNet-B4), illustrating that DQVENet is a feasible method for quantified visibility estimation.

7. Conclusions

In this paper, we propose a framework (DQVENet) for quantified visibility estimation based on deep learning. Specifically, the physical constraint is employed as the guideline of network design. Under this framework, a transmission estimation module, a depth estimation module, and an extinction coefficient estimation module are introduced. Furthermore, a new dataset named QVEData, that is especially collected for traffic image-based quantified visibility estimation, is proposed. Experimental results on this dataset demonstrate the effectiveness of DQVENet. This framework is flexible so that the three modules can be replaced by any other backbones. We should also note that DQVENet tries to integrate meteorological priors into deep learning network design, which might be a promising avenue for future research. Our future work will focus on digging deeper into interdisciplinary theories between them.