Introduction of Robustness

The robustness of a model refers to its ability to maintain its performance when confronted with various conditions, including different types of anomalies, noise, interference, changes in data distribution, and adversarial attacks. Mathematically, robustness can be described in terms of the stability of the loss function. Specifically, for a given input $X$ and true label $Y$, the model's prediction $f_{\theta }(X)$ and the loss $L(f_{\theta }(X),Y)$ between the prediction and the true label should remain within an acceptable limit even in the presence of input perturbation $\mathrm{\Delta }X$, i.e., $L(f_{\theta }(X+\mathrm{\Delta }X),Y)\le ϵ$, where $ϵ$ represents the upper limit of the loss. If the loss exceeds this limit, it can potentially lead to incorrect model outputs. In the context of adversarial robustness, the model should maintain its robustness even in the presence of the worst-case adversarial perturbation, which can be expressed as $\underset{\mathrm{\Delta }X}{max}L(f_{\theta }(X+\mathrm{\Delta }X),Y)\le ϵ$. Enhancing robustness is crucial to ensure that the model can reliably perform in complex and variable real-world environments. Additionally, the construction of perturbation noise should be inconspicuous to humans. In other words, when generating test examples with image noise, the evaluation of the created test samples should be designed in such a way that $\mathrm{\Delta }X$ makes $X+\mathrm{\Delta }X$ differ minimally in human perception from the original $X$, yet it can easily lead to errors in the model's output.

We assess the robustness of the model by applying two types of perturbations, adversarial perturbations, and natural noise perturbations, to input images across various computer vision tasks. Specifically, we introduce perturbations to image datasets, encompassing two primary categories of interference. First, we consider common natural noise present in the physical world, such as Gaussian noise caused by lighting and sensor noise, as well as atmospheric conditions like fog, rain, and snow. The second category involves targeted adversarial perturbations where we utilize proxy models to generate adversarial perturbations with the aim of causing the model to produce erroneous results when exposed to these perturbations. By applying the perturbation methods described above, we create datasets with varying degrees of perturbations and calculate the model's robustness metrics by evaluating its performance on these perturbed datasets. This evaluation method enables us to gain a better understanding of the model's performance when confronted with various image interferences, thereby enhancing its robustness in practical computer vision applications.

Datasets

COCO-Stuff Robustness Dataset

The construction of the robustness dataset involves two aspects. One is the individual addition of various types of natural noise to the original images to create perturbations for generating test images. This primarily includes Gaussian noise, impulse noise, and granular noise. The other aspect involves the use of adversarial perturbation algorithms, perturbing the results through a surrogate model. The goal is to disrupt the surrogate model's results as much as possible within a certain number of iterations.

COCO-Stuff_gaussian

Gaussian Noise

Gaussian noise is a common type of natural noise in image processing, typically caused by variations in lighting, random fluctuations in electronic sensors, or interference during the transmission process. It is characterized by the randomness of pixel values and follows a Gaussian distribution. The mathematical expression for Gaussian noise is as follows:

$$I(x,y)=I(x,y)+N(0,{\sigma }^2)$$

Sample of Perturbation Dataset

COCO-Stuff_impulse

Impulse Noise

Impulse noise, also known as salt and pepper noise, is a common type of natural noise in image processing. It often manifests as sudden bright or dark pixels in an image, simulating abrupt errors in the image sensor or during the transmission process. Its mathematical expression can be represented in the following form:

$$I(x,y)=\{\begin{array}{ll}A,& \text{ with probability }{P}_a\\ B,& \text{ with probability }{P}_b\\ I(x,y),& \text{ with probability }1-P_a-P_b\end{array}$$

Sample of Perturbation Dataset

COCO-Stuff_shot

Shot Noise

Shot noise is a common type of natural noise in image processing, typically caused by malfunctioning pixels in the image sensor or sensor defects. It manifests as individual pixels in the image appearing unusually bright or dark, often due to some pixels in the sensor having exceptionally high or low values. The mathematical expression for shot noise can be represented in the following form:

$$I(x,y)=I(x,y)+S(x,y)$$

Sample of Perturbation Dataset

COCO-Stuff_pgd

PGD Algorithm

The PGD (Projected Gradient Descent) algorithm, used in adversarial noise, is a commonly employed optimization algorithm for generating adversarial samples. Its objective is to find the smallest perturbation that causes a deep learning model to produce incorrect outputs on input data. The goal of the PGD algorithm is to minimize the loss function while constraining the perturbation within a predetermined bound. It iterates multiple times, taking small steps in each iteration and projecting the perturbation within the specified range. Selecting ResNet-50 as the surrogate model, the mathematical expression can be represented in the following form:

$$\delta (t+1)=Clipϵ(\delta (t)+\alpha \cdot sign(\nabla \delta J(\theta ,x+\delta (t),y)))$$

Sample of Perturbation Dataset

Robustness Metrics（RB-index）

For the original data set and different perturbation data sets we have $Ac{c}_{org}$，$Ac{c}_{dist1}$，$Ac{c}_{dist2}$，$Ac{c}_{dist3}$，$...$，$Ac{c}_{distT}$ ( $Acc$ refers to the evaluation index of the model under this data set, $org$ refers to the original data set, and $dis1...T$ refers to different perturbation data sets).

The calculation formula of the robustness index on this data set is:

$Robustness=\frac{1}{T\ast Ac{c}_{org}}{\mathrm{\Sigma }}_{i=1}^T(Ac{c}_{org}-Ac{c}_{disti})\ast Flag$

If the higher value of the $Acc$ metric indicates that the model is more effective, then the $Flag$ value is 1, otherwise the $Flag$ value is -1.

Smaller values of the robustness metric indicate better model robustness and can be negative (mostly found in NLP)