RGB Color Model Aware Computational Color Naming and Its Application to Data Augmentation

TY - CHAP

T1 - RGB Color Model Aware Computational Color Naming and Its Application to Data Augmentation

AU - Yan, Zipei

AU - Xu, Linchuan

AU - Suzuki, Atsushi

AU - Wang, Jing

AU - Cao, Jiannong

AU - Huang, Jun

N1 - Funding Information: This work is partially supported by the Research Institute for Artificial Intelligence of Things, The Hong Kong Polytechnic University, HK RGC CRF No. C5026-18G and HK PolyU ZUVE P0035268 Funding Information: This work is partially supported by the Research Institute for Artificial Intelligence of Things, The Hong Kong Polytechnic University, HK RGC CRF No. C5026-18G and HK PolyU ZUVE P0035268. Publisher Copyright: © 2022 IEEE.

PY - 2023/1/26

Y1 - 2023/1/26

N2 - Computational color naming (CCN) aims to learn a mapping from pixels into semantic color names, e.g., red, green and blue. CCN has wide applications including color vision deficiency assistance and color image retrieval. Existing research on CCN mainly studies pixels collected under laboratory settings or studies images collected from the web. However, laboratory pixels are very limited such that the learned mapping may not generalize well on unseen pixels, and the mapping discovered from images is usually data-specific. In this paper, we aim to learn a universal mapping by studying pixels collected from the web. To this end, we formulate a novel classification problem that incorporates both the pixels and the RGB color model. The RGB color model is beneficial for learning the mapping because it characterizes the production of colors, e.g., the addition of red and green produces yellow. However, the characterization is rather qualitative. To solve this problem, we propose ColorMLP, which is a multilayer perceptron (MLP) embedded with graph attention networks (GATs). Here, the GATs are designed to capture color relations that we construct by referring to the RGB color model. In this way, the parameters of the MLP can be regularized to comply with the RGB model. We conduct comprehensive experiments to demonstrate the superiority of ColorMLP to alternative methods.To expand the application of CCN, we design a novel data augmentation method named partial color jitter (PCJ), which performs color jitter (CJ) on a subset of pixels belonging to the same color of an image. In this way, PCJ partially changes the color properties of images, thereby significantly increasing images’ diversity. We conduct extensive experiments on CIFAR10/100 and ImageNet datasets, showing that PCJ can consistently improve the classification performance. Our data and software can be found at https://https://github.com/yanzipei/CCN_and_ItsApp.

AB - Computational color naming (CCN) aims to learn a mapping from pixels into semantic color names, e.g., red, green and blue. CCN has wide applications including color vision deficiency assistance and color image retrieval. Existing research on CCN mainly studies pixels collected under laboratory settings or studies images collected from the web. However, laboratory pixels are very limited such that the learned mapping may not generalize well on unseen pixels, and the mapping discovered from images is usually data-specific. In this paper, we aim to learn a universal mapping by studying pixels collected from the web. To this end, we formulate a novel classification problem that incorporates both the pixels and the RGB color model. The RGB color model is beneficial for learning the mapping because it characterizes the production of colors, e.g., the addition of red and green produces yellow. However, the characterization is rather qualitative. To solve this problem, we propose ColorMLP, which is a multilayer perceptron (MLP) embedded with graph attention networks (GATs). Here, the GATs are designed to capture color relations that we construct by referring to the RGB color model. In this way, the parameters of the MLP can be regularized to comply with the RGB model. We conduct comprehensive experiments to demonstrate the superiority of ColorMLP to alternative methods.To expand the application of CCN, we design a novel data augmentation method named partial color jitter (PCJ), which performs color jitter (CJ) on a subset of pixels belonging to the same color of an image. In this way, PCJ partially changes the color properties of images, thereby significantly increasing images’ diversity. We conduct extensive experiments on CIFAR10/100 and ImageNet datasets, showing that PCJ can consistently improve the classification performance. Our data and software can be found at https://https://github.com/yanzipei/CCN_and_ItsApp.

UR - http://www.scopus.com/inward/record.url?scp=85147937365&partnerID=8YFLogxK

U2 - 10.1109/BigData55660.2022.10020750

DO - 10.1109/BigData55660.2022.10020750

M3 - Conference paper

T3 - Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022

SP - 1172

EP - 1181

BT - Proceedings - 2022 IEEE International Conference on Big Data, Big Data 2022

A2 - Tsumoto, Shusaku

A2 - Ohsawa, Yukio

A2 - Chen, Lei

A2 - Van den Poel, Dirk

A2 - Hu, Xiaohua

A2 - Motomura, Yoichi

A2 - Takagi, Takuya

A2 - Wu, Lingfei

A2 - Xie, Ying

A2 - Abe, Akihiro

A2 - Raghavan, Vijay

PB - IEEE

ER -