Cntk: Problem when running the GAN example

Created on 3 Aug 2017  路  2Comments  路  Source: microsoft/CNTK

I try to run the example from "CNTK 206 Part A: Basic GAN with MNIST data", although it can run normally, but the following note appear
[Note:] Trainer ctor: 4 of the model parameters are not covered by any of the specified Learners; these parameters will not be learned
[Note:] Trainer ctor: 4 of the model parameters are not covered by any of the specified Learners; these parameters will not be learned

Most helpful comment

Good question.

What happens with GANs is that you train the Generator and Discriminators independently. Which means that while training the Generator, the discriminator parameters are not updated. Similarly when training the discriminator you donot update the generators parameters.

If you notice the D_loss = -(C.log(D_real) + C.log(1.0 - D_fake)) and
D_fake = D_real.clone(
method = 'share',
substitutions = {X_real_scaled.output: X_fake.output}
)

Where the input from the Generator is substituted. Hence, the parameters of the Generator gets tied up with the D_loss.

However, if you notice the there are two separate trainer for Discriminator and Generator which independently train the their respective parameters. Hence, the warnings and in this case, the warnings are expected and by design.

All 2 comments

Good question.

What happens with GANs is that you train the Generator and Discriminators independently. Which means that while training the Generator, the discriminator parameters are not updated. Similarly when training the discriminator you donot update the generators parameters.

If you notice the D_loss = -(C.log(D_real) + C.log(1.0 - D_fake)) and
D_fake = D_real.clone(
method = 'share',
substitutions = {X_real_scaled.output: X_fake.output}
)

Where the input from the Generator is substituted. Hence, the parameters of the Generator gets tied up with the D_loss.

However, if you notice the there are two separate trainer for Discriminator and Generator which independently train the their respective parameters. Hence, the warnings and in this case, the warnings are expected and by design.

I see a thumbs up. Assuming you are happy with the answer and closing this issue. If you need help feel free to open an new one. Thanks.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

jykim picture jykim  路  18Comments

Madgeeno picture Madgeeno  路  31Comments

jsundh picture jsundh  路  16Comments

haryngod picture haryngod  路  17Comments

fchollet picture fchollet  路  16Comments