Pytorch-cyclegan-and-pix2pix: intuition behind the loss functions loss_idt_A and loss_idt_B

[devraj89]

Hi

Thank you for posting this wonderful code but I am wondering what is the intuition behind the two losses loss_idt_A and loss_idt_B mentioned in the cycle_gan_model.py file? By reading through the implementation it seems like the loss is supposed to discourage the generator to translate the image in case it is already in the correct domain. Like if the image is in domain B then G_A should act as identity and not try to translate it?

Though I understand the intuition behind this loss, I have several questions pertaining to it
[1] why exactly is the loss relevant? since it is a controlled training setup where we know the images are coming from which domain, why would we send domain B images through G_A?
[2] Is this loss relevant to the testing time when the domain of the image is unknown?
[3] Is the loss mentioned anywhere in the paper?
[4] Is the loss helpful in generating the images? has any benchmarking been done for this?

Thanks again for the code! Hoping to get the doubts cleared soon!
Devraj

[phamnam95]

I think the identity loss is used to preserve the color and prevent reverse color in the result.

[junyanz]

@devraj89 This is a great question. For your questions:

1. You are right. This loss can regularize the generator to be near an identity mapping when real samples of the target domain are provided. If something already looks like from the target domain, you should not map it into a different image.
2. Yes. The model will be more conservative for unknown content.
3. It was described in Sec 5.2 "Photo generation from paintings (Figure 12) " in the CycleGAN paper. The idea was first proposed by Taigman et al's paper. See Eqn (6) in their paper.
4. It depends on your goal and it is quite subjecive. We don't have a benchmark yet. But Fig
   9 in the paper illustrate the difference. In general, it can help bette preserve the content if that is your priority.