INFRARED IMAGE COLORIZATION AND ENHANCEMENT FOR IMPROVED OBJECT INTERPRETATION USING SWINIR

This paper turned a prototype infrared pipeline into an effective enhancement, colorization and interpretation system and evaluated it on held-out data. A genuine SwinIR, converted exactly from colour to thermal weights and fine-tuned on FLIR ADAS v2, improves PSNR by 1.60 dB over the existing pipeline at σ = 10 and wins on every test video at every noise level tested. Combined with a thermal-domain detector, it raises detection [email protected] on degraded frames from 0.021 to 0.154. The experiments also show that colorization should serve the human operator, not the detector. The study has limitations. The degradations are synthetic, and training was much shorter than in the cited studies. Colorization quality could not be scored without aligned colour references. In the deployed web pipeline, the existing 512-pixel letterbox step resamples small inputs before enhancement, which reduces detections on low-resolution uploads. Future work will train on real sensor degradations, train the enhancement network and the detector jointly with a detection-aware loss, use the 16-bit radiometric data and temporal information from video, learn colorization from aligned thermal–visible pairs, and optimise the models for edge devices.

1.  R. Gade and T. B. Moeslund, "Thermal cameras and applications: a survey," Machine Vision and Applications, vol. 25, no. 1, pp. 245–262, 2014. 2.  T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, "Microsoft COCO: Common objects in context," in Proc. European Conference on Computer Vision (ECCV), 2014, pp. 740–755. 3.  C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295–307, 2016. 4.  B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "Enhanced deep residual networks for single image super-resolution," in Proc. IEEE CVPR Workshops, 2017, pp. 136–144. 5.  J. Liang, J. Cao, G. Sun, K. Zhang, L. Van Gool, and R. Timofte, "SwinIR: Image restoration using Swin Transformer," in Proc. IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, pp. 1833–1844. 6.  X. Chen, X. Wang, J. Zhou, Y. Qiao, and C. Dong, "Activating more pixels in image super-resolution transformer," in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 22367–22377. 7.  Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, "Swin Transformer: Hierarchical vision transformer using shifted windows," in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 10012–10022. 8.  E. Agustsson and R. Timofte, "NTIRE 2017 challenge on single image super-resolution: Dataset and study," in Proc. IEEE CVPR Workshops, 2017, pp. 126–135.