A Comprehensive Survey of Deep Learning Super-Sampling and Neural Rendering Techniques for Real-Time AI-Powered Image Upscaling

The high level of realism required in modern video games and interactive applications goes beyond the possibilities provided by traditional rasterization pipelines in terms of computational power necessary for achieving both photorealism and decent frame rates. This survey is aimed at providing an overview of more than 30 relevant papers on deep learning approaches such as DLSS, neural super-resolution, temporal anti-aliasing, generative adversarial networks upsampling and real-time denoising. In total, the reviewed studies can be organized into six main themes, namely, convolutional network-based image upscaling, GAN-based perceptual upscaling, temporal feedback and anti-aliasing, real-time image denoising, upscaling in specific domains and virtual reality, and Transformer networks-based image restoration. An analysis of the reviewed literature proves that the evolutionary path leads from purely spatial solutions for image scaling based on SRCNN to the advanced temporal models like DLSS 4 and SwinIR with attention mechanisms. In this regard, it is concluded that hybrid solutions involving denoising and upscaling should be considered among the most interesting areas of future research together with transformer models and optimized hardware implementation.