Pytorch-cyclegan-and-pix2pix: Which model should I use cyclegan or pix2pix for closed paired data?

[John1231983]

Thanks for great project. I have a dataset A and datasetB. They are unpaired data. I applied some preprocessing to align them together. Hence, i have a paired data (x,y). Due to the algin method, the result is not so perfect (but reasonable) (the error of align in 2 or 3 pixels in the boundary). What method should I use for my paired data?

[junyanz]

Pix2pix might work better for you.

[John1231983]

Thanks. But the problem is that the two images aligned not so well. Do you think pix2pix still work?

[junyanz]

2 or 3 pixels should be fine. You can also downsample both input and output images by 4x when you calculating the L1 loss.

[John1231983]

Thanks @junyanz . I confirmed that pix2pix worked much better than cycleGAN in case of paired data, although the paired data is not aligned well. Hope it can help other people

However, the synthetic image is still blurred. Do you think the reason comes from L1 loss? I am using L1 loss for Generation (with lambda=10) and MSE for Discrimination.

Synthetic image

Real image

[junyanz]

A smaller L1 might help (lambda=1 or 2.5)

[John1231983]

Thanks. I am following your suggestion by changing the generator. I used Unet generator network. I want to get the intermediate output (likes yellow arrow). However, your coding style is sequence model.

https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix/blob/18a40e606eb5ef5214db84b1bb24b9f0e3641371/models/networks.py#L459

How can I get intermedidate output if I still want to use your code above?

[junyanz]

It's hard to get intermediate output using the current code. One possible way is to give different names to different modules.

self.unet_block1 =  UnetSkipConnectionBlock(ngf * 4, ngf * 8, input_nc=None, submodule=unet_block, norm_layer=norm_layer)
self.unet_block2 =UnetSkipConnectionBlock(ngf * 2, ngf * 4, input_nc=None, submodule=self.unet_block1, norm_layer=norm_layer)

You need to change the forward function as well.

[wadesunyang]

I have a question about the pix2pix, with the default set, the preprocess is resize_and_crop with param of 286 and 256. So after crop the paired images is not matched , is some thing wrong with my understanding? Thank you very much.

[junyanz]

We apply the same cropping (x, y, height, width) to the input and output images. They are still matched.