Models: Can't run ImageNet download and preprocess script

[bmzhao]

Model: Inception
The download_and_preprocess_imagenet.sh script does not work using the instructions given. Currently using Mac OS X El Capitan with a tensorflow installation built from source described here: https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html#installation-for-mac-os-x

After setting the DATA_DIR variable, and executing:
bazel build inception/download_and_preprocess_imagenet
in the models/inception directory, this is the output:

INFO: Found 1 target...
Target //inception:download_and_preprocess_imagenet up-to-date:
bazel-bin/inception/download_and_preprocess_imagenet
INFO: Elapsed time: 0.499s, Critical Path: 0.00s

But, the next command:
bazel-bin/inception/download_and_preprocess_imagenet "${DATA_DIR}$"
fails with the following output:
bazel-bin/inception/download_and_preprocess_imagenet: line 66: bazel-bin/inception/download_and_preprocess_imagenet.runfiles/inception/data/download_imagenet.sh: No such file or directory

[xuehui1991]

Same issue.

And there might have another issue:

BUILD_SCRIPT="${WORK_DIR}/build_imagenet_data"

this path seems wrong?

build_imagenet_data
or
build_imagenet_data.py?

[prb12]

@bmzhao I'm not sure what instructions you are following ... was the extra $ in your message deliberate?

13:45:51 :~/tensorflow$ DATA_DIR=~/data
13:50:56 :~/tensorflow$ echo "${DATA_DIR}$"
/usr/local/google/home/pbar/data$

vs.

13:51:05 :~/tensorflow$ echo "${DATA_DIR}"
/usr/local/google/home/pbar/data

[bmzhao]

Hello prb12,

The extra dollar sign came from the instructions here: https://github.com/tensorflow/models/blob/master/inception/README.md#getting-started

I tried executing both with and without the extra "$", but got the same error.

[prb12]

@shlens Jon could you please take a quick look at this one and see if it's anything obvious?

[shlens]

@bmzhao, in order to help me diagnose what is going on, would you be able to try the following to commands?

bazel build inception/download_and_preprocess_imagenet
ls -l -R bazel-bin/inception/download_and_preprocess_imagenet.runfiles/

[bmzhao]

Hello shlens,

The result of the command 'bazel build inception/download_and_preprocess_imagenet' is:
INFO: Found 1 target...
Target //inception:download_and_preprocess_imagenet up-to-date:
bazel-bin/inception/download_and_preprocess_imagenet
INFO: Elapsed time: 0.099s, Critical Path: 0.00s

The output of the command 'ls -l -R bazel-bin/inception/download_and_preprocess_imagenet.runfiles/' is:
total 8
-r-xr-xr-x 1 bzhao wheel 1.5K Jun 14 11:15 MANIFEST
drwxr-xr-x 3 bzhao wheel 102B Jun 14 11:15 main
drwxr-xr-x 3 bzhao wheel 102B Jun 14 11:52 inception

bazel-bin/inception/download_and_preprocess_imagenet.runfiles//main:
total 0
drwxr-xr-x 6 bzhao wheel 204B Jun 14 11:15 inception

bazel-bin/inception/download_and_preprocess_imagenet.runfiles//main/inception:
total 16
-r-xr-xr-x 1 bzhao wheel 0B Jun 14 11:15 init.py
lrwxr-xr-x 1 bzhao wheel 132B Jun 14 11:15 build_imagenet_data -> /private/var/tmp/_bazel_bzhao/67ba93ddd7c33c63def5f0c9997bb082/inception/bazel-out/local-fastbuild/bin/inception/build_imagenet_data
drwxr-xr-x 11 bzhao wheel 374B Jun 14 11:15 data
lrwxr-xr-x 1 bzhao wheel 145B Jun 14 11:15 download_and_preprocess_imagenet -> /private/var/tmp/_bazel_bzhao/67ba93ddd7c33c63def5f0c9997bb082/inception/bazel-out/local-fastbuild/bin/inception/download_and_preprocess_imagenet

bazel-bin/inception/download_and_preprocess_imagenet.runfiles//main/inception/data:
total 64
-r-xr-xr-x 1 bzhao wheel 0B Jun 14 11:15 init.py
lrwxr-xr-x 1 bzhao wheel 81B Jun 14 11:15 build_imagenet_data.py -> /Users/bzhao/Documents/JPL/models/inception/inception/data/build_imagenet_data.py
lrwxr-xr-x 1 bzhao wheel 94B Jun 14 11:15 download_and_preprocess_imagenet.sh -> /Users/bzhao/Documents/JPL/models/inception/inception/data/download_and_preprocess_imagenet.sh
lrwxr-xr-x 1 bzhao wheel 79B Jun 14 11:15 download_imagenet.sh -> /Users/bzhao/Documents/JPL/models/inception/inception/data/download_imagenet.sh
lrwxr-xr-x 1 bzhao wheel 101B Jun 14 11:15 imagenet_2012_validation_synset_labels.txt -> /Users/bzhao/Documents/JPL/models/inception/inception/data/imagenet_2012_validation_synset_labels.txt
lrwxr-xr-x 1 bzhao wheel 90B Jun 14 11:15 imagenet_lsvrc_2015_synsets.txt -> /Users/bzhao/Documents/JPL/models/inception/inception/data/imagenet_lsvrc_2015_synsets.txt
lrwxr-xr-x 1 bzhao wheel 80B Jun 14 11:15 imagenet_metadata.txt -> /Users/bzhao/Documents/JPL/models/inception/inception/data/imagenet_metadata.txt
lrwxr-xr-x 1 bzhao wheel 97B Jun 14 11:15 preprocess_imagenet_validation_data.py -> /Users/bzhao/Documents/JPL/models/inception/inception/data/preprocess_imagenet_validation_data.py
lrwxr-xr-x 1 bzhao wheel 84B Jun 14 11:15 process_bounding_boxes.py -> /Users/bzhao/Documents/JPL/models/inception/inception/data/process_bounding_boxes.py

bazel-bin/inception/download_and_preprocess_imagenet.runfiles//inception:
total 0
drwxr-xr-x 2 bzhao wheel 68B Jun 14 11:52 data

bazel-bin/inception/download_and_preprocess_imagenet.runfiles//inception/data:

[danielschonfeld]

WORK_DIR is missing __main__ before the /inception

[shlens]

On my end we have been able to git clone a fresh copy of tensorflow 0.8 as well as tensorflow/models, compile and run:

bazel build -c opt --config cuda inception/download_and_preprocess_imagenet
bazel-bin/inception/download_and_preprocess_imagenet 

I am running though on a Ubuntu machine.

I suspect that bazel generates a different directory structure for data entries in a BUILD rule on Macintosh then on Unix. In particular, the Mac version of bazel inserts a main/ sub-directory. @martinwicke, are you aware of any differences of how bazel handles data entries in BUILD rules when compiled on a Mac?

In the mean time, I might suggest that you try cloning a fresh copy of TensorFlow and the models repo, compiling and running again to see if the problem goes away.

The larger picture item to be aware of is that ImageNet is a very large data set and unless you have a custom GPU in your Mac, it is generally not practical to train such a large model/data set on a default Macintosh setup.

[bmzhao]

I've re-cloned the both models and tensorflow, but I'm still seeing the same problem after building; the expected files are indeed inside the "main" directory.

Thank you for the advice! I've managed to use the docker image provided here: https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html#docker-installation on a more powerful server to successfully start the training process.

Thank you again for explaining how bazel behaves differently on OSX, and your advice to use a different machine!

[zhangtemplar]

I got extractly the same problem as @bmzhao

[nhe150]

I got same problem.
Solved by Looking at daniel's comment
WORK_DIR is missing main before the /inception

edit bazel-bin/inception/download_and_preprocess_imagenet and correct WORK_DIR.

[sguada]

@shlens should we close this issue?

[guybd]

Yes.
Thank you

On Sun, Aug 28, 2016, 02:36 Sergio Guadarrama [email protected]
wrote:

@shlens https://github.com/shlens should we close this issue?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/models/issues/202#issuecomment-242946839,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQ69qkHw76dkUrLElLIybeRh39S6i1D2ks5qkMoGgaJpZM4I1p5d
.

[pankajkgupta]

Hi all,
This seems to have been solved by @bmzhao but I am still getting the error.
I tried solution by @danielschonfeld but that does not seem to work either.

I am using bazel on a docker container.

After setting the DATA_DIR-
root@0047d9668d34:/data/workspace/models/inception# DATA_DIR=/data/imagenet-data

I run the following command-
root@0047d9668d34:/data/workspace/models/inception# bazel-bin/inception/download_and_preprocess_imagenet "${DATA_DIR}"

It asks me imagenet credentials and downloads them fine but soon fails with following error-

bazel-bin/inception/download_and_preprocess_imagenet.runfiles/inception/inception/data/download_imagenet.sh: line 105: bazel-bin/inception/download_and_preprocess_imagenet.runfiles/inception/inception/data/imagenet_lsvrc_2015_synsets.txt: No such file or directory

bazel is complaining even though I can list and open the file at that path.

Anybody has any clue why this would be happening?

Much appreciated!

[justinej]

I got the same error, and I fixed it by replacing line 105 in inception/data/download_imagenet.sh:
done < "${SYNSETS_FILE}"
with the absolute path:
done < "$HOME/Documents/...YOUR PATH HERE.../bazel-bin/inception/download_and_preproce ss_imagenet.runfiles/inception/inception/data/imagenet_lsvrc_2015_synsets.txt"
(and rebuilding & rerunning). Not sure why that worked, but hopefully that's helpful.

[jvlmdr]

I had the same problem. It seems to occur because $0 is a relative path. I made the following change to download_and_preprocess_imagenet and this seems to have fixed it.

# old:
WORK_DIR="$0.runfiles/inception/inception"
# new:
WORK_DIR="$(realpath -s "$0").runfiles/inception/inception"

If you can't depend on realpath, then something like this might work (I can't use readlink because you don't want to actually follow the link)

WORK_DIR="$(cd "$(dirname "$0")" && pwd)/$(basename "$0").runfiles/inception/inception"

[shlens]

The inception library is deprecated. Please use the corresponding scripts in the slim repository.

[scotthuang1989]

I have similar issues with slim module. I think the tutorial is not up to date, miss some instructions.

[ellamunde]

is there any update on this? I still get no such file ... on line "${DOWNLOAD_SCRIPT}" "${SCRATCH_DIR}" "${LABELS_FILE}"

Please kindly help or is there any workaround for this?
Thanks!

[scotthuang1989]

yes. This script is still broken. But I guess it is not easy to fix given the time it needs to run the scripts.

[lebinhe]

Thanks @danielschonfeld.
vim bazel-bin/inception/download_and_preprocess_imagenet
Find WORK_DIR and fix it to a right path.