Conventional social media platforms usually downscale high-resolution (HR) images to restrict their resolution to a specific size for saving transmission/storage cost, which makes those visual details inaccessible to other users. To bypass this obstacle, recent invertible image downscaling methods jointly model the downscaling/upscaling problems and achieve impressive performance. However, they only consider fixed integer scale factors and may be inapplicable to generic downscaling tasks towards resolution restriction as posed by social media platforms. In this paper, we propose an effective and universal Scale-Arbitrary I mage Downscaling Network (AIDN), to downscale HR images with arbitrary scale factors in an invertible manner. Particularly, the HR information is embedded in the downscaled low-resolution (LR) counterparts in a nearly imperceptible form such that our AIDN can further restore the original HR images solely from the LR images. The key to supporting arbitrary scale factors is our proposed Conditional Resampling Module (CRM) that conditions the downscaling/upscaling kernels and sampling locations on both scale factors and image content. Extensive experimental results demonstrate that our AIDN achieves top performance for invertible downscaling with both arbitrary integer and non-integer scale factors. Also, both quantitative and qualitative evaluations show our AIDN is robust to the lossy image compression standard.
The video shows the visual comparisons of the restored HR images with scale factors varying from 1.1 to 4.0. We considered three solutions:
1). Downscaling the input images with the Nearest-neighbor interpolation to produce the downscaled image and upscaling it with the Nearest-neighbor interpolation for generating the final restored HR image;
2). Downscaling the input images with the Bicubic interpolation to produce the downscaled image and upscaling it with the Bicubic interpolation for generating the final restored HR image; and
3). Downscaling the input images with the encoder network from our AIDN to produce the downscaled image and upscaling it with the decoder network from our AIDN for generating the final restored HR image.
Given an HR image to be distributed and the required arbitrary scale factor s, the embedding network can downscale the HR image to an LR image, and meanwhile, the restoration network can restore the original HR image solely from the LR counterpart, whenever users want to explore the details of the original version. The CRM denotes our proposed conditional resampling module to resample feature maps with arbitrary scale factors. The JPEG simulator is an extension module to AIDN, making it robust to JPEG compression.
Given the input feature Fin and required arbitrary scale factor
Visual comparisons of the reconstructed HR images produced by the Bicubic interpolation (BI+BI), IRN, and our AIDN with various non-integer scale factors (from ×1.5 to ×3.9). The test images are sampled from Set14 and Urban100 datasets.
@article{xing2023scale,
title={Scale-arbitrary invertible image downscaling},
author={Xing, Jinbo and Hu, Wenbo and Xia, Menghan and Wong, Tien-Tsin},
journal={IEEE Transactions on Image Processing},
year={2023},
publisher={IEEE}
}