Noise-cuts-Noise Approach for Mitigating the JPEG Distortions in Deep Learning

Published in IEEE International Conference on Artificial Intelligence in Information and Communication (ICAIIC), 2023

Abstract

Lossy image compression provides an efficient solution to the exchange and storage of large volumes of image data for various applications. The main design principle of a lossy compression algorithm is to discard visually insignificant information as much as possible while keeping the resulted visible artifacts at a minimum. However, these unperceivable defects significantly degrade the performance of a trained deep learning (DL) model. Therefore, to improve the classification performance of the models on noisy images, we propose a noise-based data augmentation technique called noise-cuts-noise (as the noise is used to mitigate the noise impact) approach. The simulation analysis have shown that the proposed method efficiently mitigates the performance gap on highly compressed images for example, the accuracy difference is reduced from 11% to 2% for classification of natural images. For uncompressed images, the model performance is either preserved or improved. In addition, to validate the usefulness of the proposed method, we considered a case study of multi-label classification task in chest X-ray (CXR) images. The model accuracy on highly compressed images with the proposed augmentation method increased 2% on higher resolution images while the accuracy difference reduced from 6% to 1% on smaller resolution images.

Contributions

  • Noise-based image augmentation method to make DL robust against the JPEG distortions.
  • Optimal level of distortion for noise-based augmentation method.
  • As an application of our proposed method, we designed a case study of multi-label classification task for medical image analysis.

Proposed Method

For an image \(I\), its JPEG compressed image \(I_{c}\) without the chroma-subsampling function can obtained as

\[I_{c}=\left[Q\left(D\left(C_{YCbCr}\left(I\right)\right),Qf\right)\right],\]

and \(I_{c}\) can be decompressed to obtain \(I_{d}\) as

\[I_{d}=\left[C_{RGB}\left(\left[D^{-1}\left(Q^{-1}\left(I_{c},Qf\right)\right)\right]\right)\right]\rvert_0^{255}\]

where, \(\left[.\right]\) and \(.\rvert_0^{255}\) rounds and truncates the values to a valid range in the spatial domain, respectively.

In the JPEG algorithm, the information loss is resulted from the quantization forward function \(Q\) as in Equation (\(3\)) and inverse function \(Q^{-1}\) as in Equation (\(4\)). The two transformation functions that is, the colorspace conversion function \(C\) and discrete cosine transform function \(D\) are lossless in nature; however, when their values are rounded or truncated then certain information are lost.

\[\widehat{F}_{(u,v)}=round\left(\frac{F_{(u,v)}}{QT_{(u,v)}}\right),\]

where \(F_{\left(u,v\right)}\) is the DCT matrix and \(QT_{\left(u,v\right)}\) is the quantization table.

\[\widetilde{F}_{\left(u,v\right)}=\widehat{F}_{\left(u,v\right)}\times QT_{\left(u,v\right)}.\]

Conclusion

We proposed a noise-based image augmentation method to make DL models robust against compression distortions – noise-cuts-noise. The analysis have shown that choosing varying level of noise helps the model to perform well on the future noisy images. Besides improving the model performance on noisy images, the main advantage of the proposed method is that it does not require any preprocessing for the artifacts correction.


Full Article

Assessment of the JPEG compression artifacts due to chroma subsampling, impact on a deep learning model performance