Multi-Reference Nonlinear Transform Source-Channel Coding for Wireless Image Semantic Transmission

We propose a multi-reference nonlinear transform source-channel coding (MR-NTSCC) system to improve the rate-distortion (RD) performance by introducing multi-dimensional contexts into the entropy model of the state-of-the-art (SOTA) NTSCC system. Improvements in RD performance of the proposed MR-NTSCC system are particularly significant in high-resolution wireless image transmission under low bandwidth constraints, compared with the existing NTSCC+ system. The proposed multi-reference entropy model leverages correlations within the latent representation in both spatial and channel dimensions. In the spatial dimension, the latent representation is divided into anchors and non-anchors in a checkerboard pattern, where anchors serve as reference to estimate the mutual information between anchors and non-anchors. In the channel dimension, the latent representation is partitioned into multiple groups, and features in previous groups are analyzed to estimate the mutual information between features in previous and current groups. Taking mutual information into account, the entropy model provides an accurate estimation on the entropy, which enables efficient bandwidth allocation and enhances RD performance. Comprehensive experiments are conducted to verify the peak signal-to-noise ratio (PSNR) performance of the proposed MR-NTSCC model superior to SOTA methods over both additive white Gaussian noise (AWGN) channel and Rayleigh fading channel.