Models: model_main.py freezes on execution

[aanagn]

When I try to run model_main.py, it generates the following errors, and then freezes.
https://pastebin.com/0f8mvDWS

---

System information

• What is the top-level directory of the model you are using:
• * this is the output of ls -R *: https://pastebin.com/6yDdX3tM
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
  No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  Linux Ubuntu 18.04LTS
• TensorFlow installed from (source or binary):
  Binary
• TensorFlow version (use command below):
  v1.11.0-0-gc19e29306c 1.11.0
• Bazel version (if compiling from source):
• CUDA/cuDNN version:
  Cuda compilation tools, release 9.0, V9.0.176
  cuDNN 7.2.1
• GPU model and memory:
  03:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)
  04:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1)
  RAM 64GB
• Exact command to reproduce:
  python3 model_main.py --logtostderr --train_dir=/home/tanasi/Documents/models/research/object_detection/training/ --pipeline_config_path=/home/tanasi/Documents/models/research/object_detection/training/ssd_mobilenet_v1_pets.config

Thanks in advance!

[flamato]

I'm facing the same freeze using ubuntu 18.04, tf 1.9, cuda 9.0 and cudnn 7.1.2
An idea ?

[varun19299]

Are there any scalars written to tensorboard?

[aanagn]

@varun19299 I don't know, I was just trying to run a custom object detection following tutorials.

[dori-reichmann]

It doesn't freezes it's a bug with logging to stdout.
add:
tf.logging.set_verbosity(tf.logging.INFO)
to: 'model_main.py' (after the imports)

[AlanNaoto]

It doesn't freezes it's a bug with logging to stdout.
add:
tf.logging.set_verbosity(tf.logging.INFO)
to: 'model_main.py' (after the imports)

Thanks! Worked like a charm :)

[rringham]

It doesn't freezes it's a bug with logging to stdout.
add:
tf.logging.set_verbosity(tf.logging.INFO)
to: 'model_main.py' (after the imports)

This is still broken.

[tensorflowbutler]

Hi There,
We are checking to see if you still need help on this, as this seems to be considerably old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing.
If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.

[amahendrakar]

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!