This was my trial for toy project with CycleGAN.
The project was inspired by an animation called Pokémon. We selected five types(fire, water, grass, ground, and electric) among lots of types in Pokémon and try to apply such types to people. We experimented with CycleGAN as our first model, attracted by the availability of unparalleled image datasets.
When applying CycleGAN, our attempts were divided into two. The first was to use natural pictures of real fire, water, grass, electricity, and ground for training. The second was to use characters of Pokémon with similar designs among target types for training.
However, the results were not good. They reacted sensitively to the overall background and color because they did not use paired data. Therefore, in the first experiment, there was a problem of changing the color of the entire photo, including the person. It was simply the color change.
Brief results are as follows, and you can see it in CycleGAN_result1_2types.zip. https://drive.google.com/file/d/1sv1ONHk8rkgCixXu3YZgpJevpb_kgeWs/view?usp=sharing
Experiments were conducted only on fire and electric (lighting) because the results were not good.
In the second experiment, we learned with Pokémon pictures. The character has an image with the background. Therefore, it was found that the characteristics inside the Pokémon boundary were reflected only to humans.
The results are shown in the picture below, and you can see more results in CycleGAN_result2_4types.zip. https://drive.google.com/file/d/1wY6wKVPaWmTreYhi_0unTtRA4Fnr1tXg/view?usp=sharing
Experiments were conducted only in four types: water, fire, grass, and electricity.
However, our team thought that the goal was not to simply reflect colors on people. Therefore, after two experiments using CycleGAN, we decided to try a different model.