I try to run the example from "CNTK 206 Part A: Basic GAN with MNIST data", although it can run normally, but the following note appear
[Note:] Trainer ctor: 4 of the model parameters are not covered by any of the specified Learners; these parameters will not be learned
[Note:] Trainer ctor: 4 of the model parameters are not covered by any of the specified Learners; these parameters will not be learned
Good question.
What happens with GANs is that you train the Generator and Discriminators independently. Which means that while training the Generator, the discriminator parameters are not updated. Similarly when training the discriminator you donot update the generators parameters.
If you notice the D_loss = -(C.log(D_real) + C.log(1.0 - D_fake)) and
D_fake = D_real.clone(
method = 'share',
substitutions = {X_real_scaled.output: X_fake.output}
)
Where the input from the Generator is substituted. Hence, the parameters of the Generator gets tied up with the D_loss.
However, if you notice the there are two separate trainer for Discriminator and Generator which independently train the their respective parameters. Hence, the warnings and in this case, the warnings are expected and by design.
I see a thumbs up. Assuming you are happy with the answer and closing this issue. If you need help feel free to open an new one. Thanks.
Most helpful comment
Good question.
What happens with GANs is that you train the Generator and Discriminators independently. Which means that while training the Generator, the discriminator parameters are not updated. Similarly when training the discriminator you donot update the generators parameters.
If you notice the D_loss = -(C.log(D_real) + C.log(1.0 - D_fake)) and
D_fake = D_real.clone(
method = 'share',
substitutions = {X_real_scaled.output: X_fake.output}
)
Where the input from the Generator is substituted. Hence, the parameters of the Generator gets tied up with the D_loss.
However, if you notice the there are two separate trainer for Discriminator and Generator which independently train the their respective parameters. Hence, the warnings and in this case, the warnings are expected and by design.