Protobuf: Lost import and Cannot batch generated

Created on 28 Dec 2018 · 1Comment · Source: golang/protobuf

What version of protobuf and what language are you using?
Version: libprotoc 3.6.1

What did you do?
Generate gRPC code.

What did you expect to see?

Code work;
Auto create package directory;
Auto generate lot of proto.

What did you see instead?

Lost import, code not work;
Need to create package directory first;
Cannot batch generated.

Anything else we should know about your project / environment?
Ubuntu Desktop 18.04, GoLand 2018.2, Go 1.11.1

Project File: https://github.com/alx696/share/tree/master/grpc

Grpc server is Java based. The proto files work fine both Java and .Net. But with go. Oh, not work. I need to create package directory myself, generate grpc code one by one, then fix the lost import...

father.proto

syntax = "proto3";

package auto.grpc.father;

message Info {
  string name = 1;
}

son.proto

syntax = "proto3";

package auto.grpc.son;

import "father.proto";

service SonService {
  rpc Insert (Info) returns (Info) {
  }
}

message Info {
  string id = 1; //auto set
  string name = 2;
  father.Info father = 3;
}

First, i create the base package auto/grpc, then create auto/grpc/fater, auto/grpc/son. Yes, with option go_package directory will create automatic. But why not if no option go_package?

After that:

x@x-ThinkPad-T460:~/go/src/lilu.red/grpc$ protoc -I proto/ proto/father.proto --go_out=plugins=grpc:auto/grpc/father/
x@x-ThinkPad-T460:~/go/src/lilu.red/grpc$ protoc -I proto/ proto/son.proto --go_out=plugins=grpc:auto/grpc/son/

Then i fix the lost import (one import, two auto_grpc_father.):

...
import (
  ...
  "lilu.red/grpc/auto/grpc/father"
  ...
)
...
type Info struct {
    ...
    Father               *auto_grpc_father.Info `protobuf:"bytes,3,opt,name=father,proto3" json:"father,omitempty"`
    ...
}
...
func (m *Info) GetFather() *auto_grpc_father.Info {
...

Works, but not good anymore. If lot of proto ...

Try

So, how about set option go_package?

father.proto

...
option go_package = "auto/grpc/father";
...

son.proto

...
option go_package = "auto/grpc/son";
...

protoc --go_out=plugins=grpc:. proto/*.proto not work:

father.proto: File not found.
proto/son.proto: Import "father.proto" was not found or had errors.
proto/son.proto:16:3: "auto.grpc.father" seems to be defined in "proto/father.proto", which is not imported by "proto/son.proto".  To use it here, please add the necessary import.

protoc -I proto/ --go_out=plugins=grpc:. proto/*.proto not work:

2018/12/28 14:28:04 protoc-gen-go: error:inconsistent package import paths: "auto/rpc/father", "auto/rpc/son"
--go_out: protoc-gen-go: Plugin failed with status code 1.

Must generate one by one:

x@x-ThinkPad-T460:~/go/src/lilu.red/grpc$ protoc -I proto/ --go_out=plugins=grpc:. proto/father.proto 
x@x-ThinkPad-T460:~/go/src/lilu.red/grpc$ protoc -I proto/ --go_out=plugins=grpc:. proto/son.proto

Looks ok, but import lost project base path lilu.red/grpc/:

auto/grpc/son/son.pb.go:7:2: cannot find package "auto/grpc/father" in any of:
    /home/x/app/go/src/auto/grpc/father (from $GOROOT)
    /home/x/go/src/auto/grpc/father (from $GOPATH)

Just not work... I must change father "auto/grpc/father" to father "lilu.red/grpc/auto/grpc/father".

Suggest

Make generation easier.

Use `package` instead of `option go_package`

Create package directory like Java, and keep the go package name with full path. father.proto will create ./auto/grpc/father directory, and package name of father.pb.go is auto_grpc_father. option go_package need to be set in every proto file. It`s different from other languages. We can keep it, but just option.

Add parameter to set base import path

If no option go_package, import lost project base path lilu.red/grpc/. So we need it to set project base path. Just like protoc --go_out=plugins=grpc,project_prefix=lilu.red/grpc:. proto/*.proto. Now, if add lilu.red/grpc/ to option go_package, the directory ./lilu.red/grpc will be created. The code is under ./lilu.red/grpc/auto/grpc/father, its wrong! If set it with import_prefix, all import path wrong! So, we need a parameter just set the proto import base path.

Support lot of proto generation And import automatic

At least ensure that the protocols in same directory are imported correctly. No -I proto/ needed, without import the Generated code can not work. This should be the default behavior.

With Java, we just need to conf gradle:

plugins {
  id "com.google.protobuf" version "0.8.5"
}

protobuf {
  protoc {
    artifact = "com.google.protobuf:protoc:3.5.1-1"
  }
  plugins {
    grpc {
      artifact = 'io.grpc:protoc-gen-grpc-java:1.17.1'
    }
  }
  //生成文件路径
  generatedFilesBaseDir = "$projectDir/src"
  generateProtoTasks {
    all()*.plugins {
      grpc {
        outputSubDir = 'java'//设为跟Proto目录一致.
      }
    }
  }
}

then run generateProto, works fine.

Source