Abstract:Image coloring is an inherently uncertain and multimodal problem. By inputting a grayscale image into a coloring network, visually plausible colored photos can be generated. Conventional methods primarily rely on semantic information for image colorization. Although effective in coloring images with clear semantic information, these methods still suffer from color contamination and semantic confusion. This is largely due to the limited capacity of convolutional neural networks to effectively learn deep semantic information inherent in images.In this paper, we propose a network structure that addresses these limitations by leveraging multi-level semantic information classification and fusion. Additionally, we introduce a global semantic fusion network to combat the issues of color contamination. The proposed coloring encoder accurately extracts object-level semantic information from images.To further enhance visual plausibility, we employ a self-supervised adversarial training method. We train the network structure on various datasets with varying amounts of data and evaluate its performance using the ImageNet validation set and COCO validation set. Experimental results demonstrate that our proposed RepColor can generate more realistic images compared to previous approaches, showcasing its high generalization ability.