GAN-tensorflow | Implement Generative Adversarial Nets by Tensorflow | Machine Learning library

Unfortunately, like you've said for GANs the losses are very non-intuitive. Mostly it happens down to the fact that generator and discriminator are competing against each other, hence improvement on the one means the higher loss on the other, until this other learns better on the received loss, which screws up its competitor, etc.

Now one thing that should happen often enough (depending on your data and initialisation) is that both discriminator and generator losses are converging to some permanent numbers, like this: (it's ok for loss to bounce around a bit - it's just the evidence of the model trying to improve itself)

This loss convergence would normally signify that the GAN model found some optimum, where it can't improve more, which also should mean that it has learned well enough. (Also note, that the numbers themselves usually aren't very informative.)

Here are a few side notes, that I hope would be of help:

• if loss haven't converged very well, it doesn't necessarily mean that the model hasn't learned anything - check the generated examples, sometimes they come out good enough. Alternatively, can try changing learning rate and other parameters.
• if the model converged well, still check the generated examples - sometimes the generator finds one/few examples that discriminator can't distinguish from the genuine data. The trouble is it always gives out these few, not creating anything new, this is called mode collapse. Usually introducing some diversity to your data helps.
• as vanilla GANs are rather unstable, I'd suggest to use some version of the DCGAN models, as they contain some features like convolutional layers and batch normalisation, that are supposed to help with the stability of the convergence. (the picture above is a result of the DCGAN rather than vanilla GAN)
• This is some common sense but still: like with most neural net structures tweaking the model, i.e. changing its parameters or/and architecture to fit your certain needs/data can improve the model or screw it.

Answering myself. According to the answer they do share the same weights, but the tensors on the graph are separate:

To summarise this topic - the generic advice would be:

• try playing with model parameters (like learning rates, for instance)
• try adding more variety to the input data
• try adjusting the architecture of both generator and discriminator networks.

However, in my case the issue was the data scaling: I've changed the format of the input data from the initial .jpg to .npy and lost the rescaling on the way. Please note that this DCGAN-tensorflow code rescales the input data to [-1,1] range, and the model is tuned to work with this range.

Can be done by the following steps:

Define variable scopes for discriminator and generator:

It appears Tensorflow has most of the necessary operations that are available in numpy, so here is the tf version of my numpy code above:

In your ops.py file

your problem come from the striding size in your deconv filter, modify the header for conv2d and deconv2d to:

In DCGAN.__init__, change input_fname_pattern='*.jpg' to input_fname_pattern='*.npy':

I realised what was the problem with this - just needed an updated version of the Tensorflow (had 0.9.0 versus required 0.12.1), and pip install tensorflow --upgrade solved my problem.

There is currently no definitive guide. The basic idea would be to replace all occurrences of native Python file operations with equivalents in the file_io module, most notably:

• open() -> file_io.FileIO()
• os.path.exists() -> file_io.file_exists()
• glob.glob() -> file_io.get_matching_files()

These functions will work locally and on GCS (as well as any registered file system). Note, however, that there are some slight differences in file_io and the standard file operations (e.g., a different set of 'modes' are supported).

Fortunately, checkpoint and summary writing do work out of the box, just be sure to pass a GCS path to tf.train.Saver.save and tf.summary.FileWriter.

In the sample you sent, that looks potentially painful. Consider monkey patching the Python functions to map to the TensorFlow equivalents when the program starts to only have to do it once (demonstrated here).

As a side note, all of the samples on this page show reading files from GCS.