Feat: GAN-Classifier

[TheLemonPig]

Idea:

The current classifier doesn't train well (without a generator).
The current discriminator doesn't classify well.
The idea here would be to take a combined approach: train a classifier during normal GAN training
In short: training the classifier at the same time as we train the discriminator, both using the generator.
This would involve only a few extra lines of code. If the simplest version doesn't work, there are several variations worth considering:

1. replace the classifier with an original discriminator (removing the sigmoid layer)
2. vary how the generator is trained on the classifier loss (none/some/all, constant/increasing)
3. incorporate real data into training the classifier (none/some/all, constant/increasing)
4. one-hot labels
5. make 2 generators/discriminators each only train on one condition -- computationally expensive
6. train on wavelets -- unvalidated approach

[TheLemonPig]

Extra details:

I realize I didn't mention that the classifier importantly isn't trained on any real data. It is entirely trained by the generator. Consequently, the classification loss serves a different purpose to what it usually does in classical classification training. I believe this creates a learning scenario that is uniquely different to both GAN training and Classifier training. The classifier training is two-sided, except it is playing a Max-max game with the generator. This by itself would be unstable. But combined with the Min-max game of the generator with the discriminator, it becomes stable. The generator hence has to determine a cost-benefit analysis, between making classes easy to differentiate for the classifier while still making it hard to discriminate from real data.

This should create an optimal situation for the classifier, where the generator gradually teaches the classifier how to differentiate in increasingly difficult and more realistic circumstances. At some point in the middle it will be giving the classifier exemplary examples of realistic looking data of each class to learn, before giving increasingly borderline cases. (edited)

My concern is that this system would be even more unstable compared to a GAN, and meta-parameters would become even more important.