Pytorch-cyclegan-and-pix2pix: Generate HD images

[flpgrz]

Hello,

Is it possible to use this architecture to generate HD images?
The default settings give you low-definition squared images. Did someone try with rectangular HD pics?

What I basically would like to do a training on Cityscapes images with their real shape.

Thanks in advance.
Cheers,
Filippo

[phillipi]

Sure, you can apply the architecture to any size images, as long as they fit in memory. You'll probably hit memory limits pretty quickly though. You can modify loadSize and fineSize to change the image resolution. Upon loading, images are first scaled to loadSize then cropped to fineSize.

You might also want to try some of the newer methods for making really HD images, like pix2pixHD or progressive GANs.

[flpgrz]

Thanks a lot, Phillip.

I am struggling with an unpaired samples problem. I'd like to be able to translate a datasat A into a dataset B. I guess the pix2pixHD framework won't help me. Do you know a "HD version" of the Jun-Yan's unpaired-sample GAN?

[phillipi]

I think you could apply the pix2pixHD architecture to unpaired translation, but yeah, it may take some modification to make it work well. I haven't yet seen an HD CycleGAN.

In general, I would try the coarse-to-fine approach where you first learn the translation between low frequency images, then add additional layers to learn the residual high frequencies. This idea, applied to GANs, first appeared in LAPGAN, and it underlies a lot of recent success, including pix2pixHD, progressive GANs, and StackedGAN++.

[robbiebarrat]

@FilippoGrazioli you could also try training using small patches of images, but then when you're testing it generate an HD image. That's been working for me so far.

[javl]

@robbiebarrat Do you have an example of the workflow you use for the second part? I can imagine the first part using some random crops from the source images, but what would be a good way to generate an HD image after?

[robbiebarrat]

@javl just add the arguments --loadSize 1024 --fineSize 1024 to the command you use to generate images. (replacing 1024 with your desired resolution, of course)

I got mediocre results this way, though.

[javl]

@robbiebarrat Sorry, thought you might have a different way (like how some use tiling). Thanks.

[robbiebarrat]

@javl wait - sorry if I wasn't clear; I have used tiling in combination with changing the loadSize and fineSize in the past to get alright results; just train the network to generate 256x256 unresized chunks of your images, and then after training ask it to generate a larger image (with loadSize/fineSize), and since it's learned larger features from the chunks, it should be able to generalize to a much higher resolution image.

[javl]

@robbiebarrat Thank you for your clarification. I've tried all kinds of data sets and though some have given interesting results, I keep struggling to fine tune and direct them towards the output I have in mind. Might be time to start working on some sort of cycleGAN/pix2pix tips and tricks cheatsheet.

Quick related question: I understand using fineSize during training crops your image. Does this always results in the same crop or is the position randomized? If I have an image that is 512x512 for instance, and I use loadSize=512 fineSize=128 does the network always only gets to see the center part? (meaning I should split the image into something like 4 256x256 images beforehand if I want the network to see more parts of the image).

[robbiebarrat]

@javl I'm actually not sure - I've never used loadSize and fineSize while training; only while generating the images. I just wrote a super quick python/cv2 script to chunk up the images into random 256/256 crops for me beforehand.

[junyanz]

Also see pix2pixHD

[edgardeivis]

Also see pix2pixHD
can it be used for windows?

[junyanz]

I think so. See this issue for more details.