How to use colorization model for 1 channel input

[nebebe]

Hello,
I want to apply the colorization_minimal_example to a 2D CT image using a tumor mask as a label in order to generate a CT image that contains the label. Since both images are gray, I want to modify the network that is built for 3 channel images to work for 1 channel image. But the gray images are already one channel, so they can not be split across the channel dimensions furthermore. Do you have any suggestions on how I could deal with this problem?
Thank you!

[ardizzone]

Hi!

Very good question! The easiest way would be to have a Haar wavelet downsampling or i-RevNet downsampling operation as the very first operation after the input. This way, you have 4 channels to work with.

Alternatively, you could also try a different splitting scheme apart from channel-wise, such as checkerboard splitting.
However, note that e.g. checkerboard splitting is equivalent to first having a i-RevNet dowsampling, and then a channel-wise splitting, so I don't think it will make much of a difference.

Let me know whether it works as expected!