wgan | Tensorflow Implementation of Wasserstein GAN | Machine Learning library

I'm trying to develop a GAN using FastAi. When converting the Tensor to an Image I get this error.

I want to use WGAN-GP, and when I run the code, it gives me an error:

On this GitHub page of "Improved WGAN Training", particularly in their Python-Cifar-file, the authors use the Python-module tflib. I installed it via pip (pip install tflib), but when I import it, I still get the error message No module named tflib.

Any hints would be appreciated.

I have trained a WGAN on the CelebA dataset in PyTorch following this youtube video. Since I do this on Google Cloud Platform where TensorBoard is not availabe, I save one figure of generated images by the GAN every epoch to see how the GAN is actually doing.

Now, the saved pdf files look sth like this: generated images. Unfortunately, this is not really readable, and I suspect this has to do with the preprocessing I do:

I am trying to implement a WGAN-GP model using tensorflow and keras (for credit card fraud data from kaggle).

I mostly followed the sample code that is provided in keras website and several other sample codes on the internet (but changed them from image to my data), and it is pretty straightforward.

But when I want to update the critic, the gradient of loss w.r.t critic's weights becomes all nan after a few batches. And this causes the critic's weights to become nan and after that the generator's weights become nan,... So everything become nan!

I used tf.debugging.enable_check_numerics and found that the problem arises because a -Inf appears in the gradient after some iterations.

This is directly related to the gradient-penalty term in the loss, because when I remove that the problem goes away.

Please note that the gp itself is not nan, but when I get the gradient of the loss w.r.t critic's weights (c_grads in the code below) it contains -Inf and then somehow becomes all nan.

I checked the math and network architecture for possible mistakes (like probability of gradient vanishing, etc.), and I checked my code for possible bugs for hours and hours. But I'm stuck.

I would very much appreciate it if anyone can find the root of the problem

Note: Bear in mind that the critic's output and loss function is slightly different from the original paper (because I'm trying to make it conditional) but that has nothing to do with the problem because as I said before, the whole problem goes away when I just remove the gradient penalty term

This is my critic:

I am working on a project with Wasserstein GANs and more specifically with an implementation of the improved version of Wasserstein GANs. I have two theoretical questions about wGANs regarding their stability and training process. Firstly, the result of the loss function notoriously is correlated with the quality of the result of the generated samples (that is stated here). Is there some extra bibliography that supports that argument?

Secondly, during my experimental phase, I noticed that training my architecture using wGANs is much faster than using a simple version of GANs. Is that a common behavior? Is there also some literature analysis about that?

Furthermore, one question about the continuous functions that are guaranteed by using Wasserstein loss. I am having some issues understanding this concept in practice, what it means that the normal GANs loss is not continuous function?

Good morning,

I am trying to implement the improved WGAN for 1D data as described on this paper: https://arxiv.org/pdf/1704.00028.pdf

It has been implemented as an example in the keras-contrib github: https://github.com/keras-team/keras-contrib/blob/master/examples/improved_wgan.py Nevertheless, this implementation of the gradient penalty loss is not working anymore with tf2. K.gradients() returns [None].

I am trying to implement WGAN with GP in TensorFlow 2.0. To calculate the gradient penalty it requires you to calculate the gradients of the predictions with respect to input images.

Now, to make it a bit more tractable, instead of computing the gradients of the predictions with respect to all the input images, it computes interpolated data points along the lines of original and fake data points and uses these as the inputs.

To implement this, I am first developing the compute_gradients function which would take some predictions and return the gradients of those with respect to some input images. First, I thought of doing this with tf.keras.backend.gradients but it won't work in eager mode. So, I am trying to do this now using GradientTape.

Here's the code I am using to test things out:

As I understand what of the diff between regular GAN to WGAN is that we train the discriminator/critic with more examples in each epoch. If in the regular gan we have in each epoch one batch for both modules, in WGAN we will have 5 batches (or more..) for the discriminator and one for the generator.

So basically we have another inner loop for the discriminator :