Protobuf: How to import .proto file in different package correctly?

Created on 17 Jan 2018  路  2Comments  路  Source: protocolbuffers/protobuf

I have two packages like this

com.abc.
         protobuf.
                    share.proto
         depart.
                    detect.proto 

and the conent of share.proto like this:

syntax = "proto3";
package com.adc.protobuf;
message Test{}

and the content of detect.proto like this:

syntax = "proto3";
package com.adc.depart;
import "com/abc/protobuf/share.proto"

and compile share.proto in it's dir like this:

protoc -I=. --python_out=. share.proto

then compile detect.proto in it's dir like this:

protoc -I=/pathToSrcDir/ -I=. --python_out=. detect.proto

and

pathToSrcDir has been added to PYTHONPATH,all compilations work fine,but when run a python script which

from com.abc.depart import detect_pb2

got this error

TypeError: Couldn't build proto file into descriptor pool!
Invalid proto descriptor for file "detect.proto":
  detect.proto: Import "com/abc/protobuf/share.proto" has not been loaded.
  com.abc.depert.XClass.ymethod: "com.abc.protobuf.Test" seems to be defined in "share.proto", which is not imported by "detect.proto".  To use it here, please add the necessary import.

How to solve this import problem?

question

Most helpful comment

(Note: I'm not a protobuf developer, I just came across your report.)

I think your inconsistent use of includes is the root of the problem. You have two choices, and you need to pick one and stick to it:

  1. Are the files in no particular package? In that case they are ./share.proto and ./detect.proto (as when using protoc -I. from within com/abc/depart/), and from Python use import share_pb2 and import detect_pb2.
  2. Or are they com.abc.<something> packages? In that case they are com/abc/protobuf/share.proto and com/abc/depart/detect.proto (as when using protoc -I/pathToSrcDir and in "com/abc/protobuf/share.proto"), and from Python use import com.abc.protobuf.share_pb2 and import com.abc.depart.detect_pb2.

I suspect you probably want the second one, so I'll just cover that. Then the import in detect.proto is fine, but you need to change your protoc commands like this:

cd /pathToSrcDir/
protoc -I. --python_out=. com/abc/protobuf/share.proto 
protoc -I. --python_out=. com/abc/depart/detect.proto

You could even merge the two commands:

cd /pathToSrcDir/
protoc -I. --python_out=. com/abc/protobuf/share.proto com/abc/depart/detect.proto

Make sure you have added /pathToSrc/ to your PYTHONPATH, NOT /pathToSrc/com/abc/protobuf.

By the way, you can always use a different directory for your protos and Python files. For example, if you have a proto/ directory and a src/ directory, use this command (and add /pathToProjectDir/src to your PYTHONPATH):

cd /pathToProjectDir/
protoc -Iproto --python_out=src proto/com/abc/protobuf/share.proto proto/com/abc/depart/detect.proto

All 2 comments

(Note: I'm not a protobuf developer, I just came across your report.)

I think your inconsistent use of includes is the root of the problem. You have two choices, and you need to pick one and stick to it:

  1. Are the files in no particular package? In that case they are ./share.proto and ./detect.proto (as when using protoc -I. from within com/abc/depart/), and from Python use import share_pb2 and import detect_pb2.
  2. Or are they com.abc.<something> packages? In that case they are com/abc/protobuf/share.proto and com/abc/depart/detect.proto (as when using protoc -I/pathToSrcDir and in "com/abc/protobuf/share.proto"), and from Python use import com.abc.protobuf.share_pb2 and import com.abc.depart.detect_pb2.

I suspect you probably want the second one, so I'll just cover that. Then the import in detect.proto is fine, but you need to change your protoc commands like this:

cd /pathToSrcDir/
protoc -I. --python_out=. com/abc/protobuf/share.proto 
protoc -I. --python_out=. com/abc/depart/detect.proto

You could even merge the two commands:

cd /pathToSrcDir/
protoc -I. --python_out=. com/abc/protobuf/share.proto com/abc/depart/detect.proto

Make sure you have added /pathToSrc/ to your PYTHONPATH, NOT /pathToSrc/com/abc/protobuf.

By the way, you can always use a different directory for your protos and Python files. For example, if you have a proto/ directory and a src/ directory, use this command (and add /pathToProjectDir/src to your PYTHONPATH):

cd /pathToProjectDir/
protoc -Iproto --python_out=src proto/com/abc/protobuf/share.proto proto/com/abc/depart/detect.proto

Your solution works,thanks very much

Was this page helpful?
0 / 5 - 0 ratings