Pytorch-cyclegan-and-pix2pix: Visdom: add support for multiple environments

Created on 25 Jun 2018  路  3Comments  路  Source: junyanz/pytorch-CycleGAN-and-pix2pix

Hello.

We run all our trainings on single GPU and are therefore able to run multiple trainings at the same time. By setting a display_id all the images and plots gets messed up in Visdom. Is there anyway to set a visdom environment for each run?

Most helpful comment

@davidwessman

The quickest and easiest solution i found is set different display_port for each train so each job's trainign result can be displayed on different visdom sever.

For example when you want to train 4 jobs at once. At first start 4 instances of visdom server at 4 different ports. Then for each job, set the display_port to corresponding port so the training result would appear on separated visdom server

  1. start 4 different visdom server instances
python -m visdom.server -port 8096
python -m visdom.server -port 8097
python -m visdom.server -port 8098
python -m visdom.server -port 8099
  1. start training jobs and mount on desired visdom server
job1: python train.py ..... --display_port 8096
job2: python train.py ..... --display_port 8097
job3: python train.py ..... --display_port 8098
job4: python train.py ..... --display_port 8099

Hope that help

All 3 comments

Great idea! That would make things look much nicer. I'll look into this.

@davidwessman

The quickest and easiest solution i found is set different display_port for each train so each job's trainign result can be displayed on different visdom sever.

For example when you want to train 4 jobs at once. At first start 4 instances of visdom server at 4 different ports. Then for each job, set the display_port to corresponding port so the training result would appear on separated visdom server

  1. start 4 different visdom server instances
python -m visdom.server -port 8096
python -m visdom.server -port 8097
python -m visdom.server -port 8098
python -m visdom.server -port 8099
  1. start training jobs and mount on desired visdom server
job1: python train.py ..... --display_port 8096
job2: python train.py ..... --display_port 8097
job3: python train.py ..... --display_port 8098
job4: python train.py ..... --display_port 8099

Hope that help

Was this page helpful?
0 / 5 - 0 ratings