Improving Super-Resolution Performance using Meta-Attention Layers. (arXiv:2110.14638v1 [eess.IV])

Convolutional Neural Networks (CNNs) have achieved impressive results across
many super-resolution (SR) and image restoration tasks. While many such
networks can upscale low-resolution (LR) images using just the raw pixel-level
information, the ill-posed nature of SR can make it difficult to accurately
super-resolve an image which has undergone multiple different degradations.
Additional information (metadata) describing the degradation process (such as
the blur kernel applied, compression level, etc.) can guide networks to
super-resolve LR images with higher fidelity to the original source. Previous
attempts at informing SR networks with degradation parameters have indeed been
able to improve performance in a number of scenarios. However, due to the
fully-convolutional nature of many SR networks, most of these metadata fusion
methods either require a complete architectural change, or necessitate the
addition of significant extra complexity. Thus, these approaches are difficult
to introduce into arbitrary SR networks without considerable design
alterations. In this paper, we introduce meta-attention, a simple mechanism
which allows any SR CNN to exploit the information available in relevant
degradation parameters. The mechanism functions by translating the metadata
into a channel attention vector, which in turn selectively modulates the
network’s feature maps. Incorporating meta-attention into SR networks is
straightforward, as it requires no specific type of architecture to function
correctly. Extensive testing has shown that meta-attention can consistently
improve the pixel-level accuracy of state-of-the-art (SOTA) networks when
provided with relevant degradation metadata. For PSNR, the gain on
blurred/downsampled (X4) images is of 0.2969 dB (on average) and 0.3320 dB for
SOTA general and face SR models, respectively.



Related post