Models: ImportError: cannot import name preprocessor_pb2 when trying to run job on google cloud

[kosintop]

i'm trying run my own custom object detection training job by following 'pet training quickstart' and guideline on 'running on cloud' but has the following error

The replica master 0 exited with a non-zero status of 1. Termination reason: Error. Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/root/.local/lib/python2.7/site-packages/object_detection/train.py", line 49, in from object_detection import trainer File "/root/.local/lib/python2.7/site-packages/object_detection/trainer.py", line 27, in from object_detection.builders import preprocessor_builder File "/root/.local/lib/python2.7/site-packages/object_detection/builders/preprocessor_builder.py", line 21, in from object_detection.protos import preprocessor_pb2 ImportError: cannot import name preprocessor_pb2

and a few more error for each worker with the same pattern

The replica worker 0 .... ImportError: cannot import name preprocessor_pb2
The replica worker 1 .... ImportError: cannot import name preprocessor_pb2
The replica ps 0 .... ImportError: cannot import name preprocessor_pb2
The replica ps 1 .... ImportError: cannot import name preprocessor_pb2

this is the input show in Google console

{
"scaleTier": "CUSTOM",
"masterType": "standard_gpu",
"workerType": "standard_gpu",
"parameterServerType": "standard",
"workerCount": "5",
"parameterServerCount": "3",
"packageUris": [
"gs://image-detection-kos/checkpoints/packages/885544b8b58fca05c7af7daec46b5a33a9f3cb4a598093eb59983e1c2485c597/object_detection-0.1.tar.gz",
"gs://image-detection-kos/checkpoints/packages/885544b8b58fca05c7af7daec46b5a33a9f3cb4a598093eb59983e1c2485c597/slim-0.1.tar.gz"
],
"pythonModule": "object_detection.train",
"args": [
"--train_dir=gs://image-detection-kos/checkpoints",
"--pipeline_config_path=gs://image-detection-kos/cloud.config"
],
"region": "us-east1",
"runtimeVersion": "1.0",
"jobDir": "gs://image-detection-kos/checkpoints"
}

I'm not sure if this is a bug or if I do something wrong? Do i need to build proto files by myself? or did the script not building properly?

[pjl1995]

Hi, I have the same problem with you. Did you solve it? Thank you~

[kosintop]

@pjl1995 sadly, I have no idea how to fix this yet. I will update if I found a solution

[pjl1995]

@kosintop Thanks, if I found the solution, I will also tell you~

[pjl1995]

Maybe you forget this?
protoc object_detection/protos/*.proto --python_out=.
In detail, you can look the link:
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/installation.md

Now I have another problem......
ImportError: No module named matplotlib.pyplot

[kosintop]

@pjl1995 thanks, I checked and it turn out my problem is i didn't properly config PYTHONPATH.
I already compile proto files but the build script couldn't find those because there's no PYTHONPATH

and now i'm having another error "unexpected keyword argument 'file'" though

i will close this issue for now

[evolu8]

This is driving me crazy. I see the preprocessor_pb2.py file. I know the path to research is there. I don't see any other versions it could be looking at. Why is protobuf such pain?

If someone has an answer please post.

@kosintop what did you put in your path? I've added as described in the documentation but still failing :/

[Denny143]

Has anybody found a solution to this problem yet?

[Adityaupadhyay20]

C:\Users\user\Desktop\new_project\models\research>python object_detection/builders/model_builder_test.py
Traceback (most recent call last):
File "object_detection/builders/model_builder_test.py", line 21, in
from object_detection.builders import model_builder
File "C:\Python36\lib\site-packages\object_detection-0.1-py3.6.egg\object_detection\builders\model_builder.py", line 17, in
from object_detection.builders import anchor_generator_builder
File "C:\Python36\lib\site-packages\object_detection-0.1-py3.6.egg\object_detection\builders\anchor_generator_builder.py", line 21, in
from object_detection.protos import anchor_generator_pb2
ImportError: cannot import name 'anchor_generator_pb2'

please help

hey guys i found the solution
https://stackoverflow.com/questions/1828379/why-do-i-see-cannot-import-name-descriptor-pb2-error-when-using-google-protoco

use this link

[shray-yes]

@Denny143 @evolu8
There isn't an __init__.py in your protos folder. Just create an empty file and rename it __init__.py

Do let me know if it works.

[Rajamohanreddyai]

hi,
check this one, run the below command from research path
you can do manually

From tensorflow/models/research/
export PYTHONPATH=$PYTHONPATH:pwd:pwd/slim

protoc --python_out=. .\object_detection\protos\calibration.proto
protoc --python_out=. .\object_detection\protos\preprocessor.proto
protoc --python_out=. .\object_detection\protos\anchor_generator.proto

and verify below video
https://youtu.be/nZUxoHPFf4

[mmaasstteerr]

@Rajamohanreddyai The wideo is unawailable. Can you reupload?