Protobuf: protoc does not find plugins within PATH

[gegenschall]

I'm using protobuf 3.0.0_beta_1 in combination with grpc 0.10.2. grpc installs an executable named grpc_cpp_plugin within /usr/bin which should be used with protoc's --plugin switch. It seems that even though the documentation for the --plugin switch tells me that protoc is looking for plugins in $PATH, it does not. Prepending the full path to the binary works, however.

C.f. the following output:

$ protoc --grpc_out=/tmp --plugin=protoc-gen-grpc=grpc_cpp_plugin something.proto
grpc_cpp_plugin: program not found or is not executable
--grpc_out: protoc-gen-grpc: Plugin failed with status code 1.

$ which grpc_cpp_plugin
/usr/bin/grpc_cpp_plugin

$ protoc --grpc_out=/tmp --plugin=protoc-gen-grpc=/usr/bin/grpc_cpp_plugin something.proto
$ echo $?
0

$ protoc --grpc_out=/tmp --plugin=protoc-gen-grpc=true something.proto
true: program not found or is not executable
--grpc_out: protoc-gen-grpc: Plugin failed with status code 1.

$ protoc --grpc_out=/tmp --plugin=protoc-gen-grpc=/bin/true something.proto
$ echo $?
0

[zj8487]

any idea?
update the following is working

"plugin=protoc-gen-grpc=/usr/local/bin/grpc_python_plugin

[mfycheng]

I was running into this issue, and did some digging. The core issue seems to be that it attempts to execute the plugin using execv instead of execvp.

The subprocess is called from: https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/command_line_interface.cc#L1553

if (plugins_.count(plugin_name) > 0) {
    subprocess.Start(plugins_[plugin_name], Subprocess::EXACT_NAME);
} else {
    subprocess.Start(plugin_name, Subprocess::SEARCH_PATH);
}

which calls down to execv:
https://github.com/google/protobuf/blob/master/src/google/protobuf/compiler/subprocess.cc#L323

switch (search_mode) {
    case SEARCH_PATH:
        execvp(argv[0], argv);
        break;
    case EXACT_NAME:
        execv(argv[0], argv);
}

[xfxyjwf]

Hi,

I think the documentation may be a little misleading here. What the doc meant to say is, if you don't specify the --plugin flag, protocol compiler will try to find the plugin in $PATH with the given output name.

For example:

$ protoc --grpc_out=/tmp foo.proto

With the above command, protocol compiler will try to find protoc-gen-grpc in $PATH. However, if you do:

$ protoc --grpc_out=/tmp --plugin=protoc-gen-grpc=grpc_cpp_plugin foo.proto

Protocol compiler will treat grpc_cpp_plugin as a relative file path.

Could you guys point me to the documentation you are looking at?

[haberman]

If you can point us to some documentation that is misleading about this, please re-open and we'll fix it to be more clear.

[dontsovcmc]

I also got error in Windows 10:
--grpc_out: protoc-gen-grpc: Не удается найти указанный файл. (program not found or is not executable)

fullpath of grpc_cpp_plugin helps me:
protoc.exe --plugin="protoc-gen-grpc=D://grpc_cpp_plugin.exe" --grpc_out=src helloworld.proto

[TomK]

@haberman
This is still a problem. The error message suggests the plugin binary can exist in PATH, but it does not appear to check for it. This (windows) issue also appears to be related.

The snip below (from OSX) shows that grpc_php_plugin can be found in PATH, but also that protoc exits non-zero stating the plugin cannot be found.

$ command -V grpc_php_plugin
grpc_php_plugin is /usr/local/bin/grpc_php_plugin

$ protoc -I ./tmp_proto -I $GOPATH/src --proto_path=tmp_proto --php_out=src --grpc_out=src --plugin=protoc-gen-grpc=grpc_php_plugin ./tmp_proto/assemble/*.proto
grpc_php_plugin: program not found or is not executable
Please specify a program using absolute path or make sure the program is available in your PATH system variable
--grpc_out: protoc-gen-grpc: Plugin failed with status code 1.

Specifying the full path to the plugin does work.

If you want to clean up the documentation I'd suggest removing the suggestion that binaries will be found in PATH, but it would be nicer if it was supported.

Hope this helps.

EDIT: For reference, the issue probably stems from here. I'm happy to make a PR, but i'd like to discuss the desired behaviour first.

[haberman]

I think I understand the confusion. The error message is saying: "Please either specify an absolute path or don't specify a program name at all and make sure the protoc-gen-grpc program is in your PATH".

I agree the error message you mentioned could be more clear.

Yes, it does seem possible to allow grpc_php_plugin to be searched from $PATH also. I'm not sure what the rationale was for not doing this.

I'm guessing the intention was that the plugin would be always be named protoc-gen-grpc, and that you wouldn't need to specify a filename unless the program was in a different directory.

I wonder why the gRPC plugin is not just called protoc-gen-grpc_php. That would be a better experience all around, because then you wouldn't need to specify --plugin at all.