Protobuf: Incorrect imports for generated files

[pdoherty]

I am unable to build a package when a proto file is included from a separate package.

src/foo/foo.proto

package foo;

message Foo
{
  optional string foo = 1;
}

protoc --go_out=src/ --proto_path=src/ src/foo/foo.proto
src/foo/foo.pb.go

// Code generated by protoc-gen-go.
// source: foo/foo.proto
// DO NOT EDIT!

/*
Package foo is a generated protocol buffer package.

It is generated from these files:
    foo/foo.proto

It has these top-level messages:
    Foo
*/
package foo

import proto "github.com/golang/protobuf/proto"
import math "math"

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = math.Inf

type Foo struct {
    Foo              *string `protobuf:"bytes,1,opt,name=foo" json:"foo,omitempty"`
    XXX_unrecognized []byte  `json:"-"`
}

func (m *Foo) Reset()         { *m = Foo{} }
func (m *Foo) String() string { return proto.CompactTextString(m) }
func (*Foo) ProtoMessage()    {}

func (m *Foo) GetFoo() string {
    if m != nil && m.Foo != nil {
        return *m.Foo
    }
    return ""
}

func init() {
}

src/bar/bar.proto

package bar;

import "foo/foo.proto";

message Bar
{
  optional foo.Foo foo = 1;
  optional string bar = 2;
}

protoc --go_out=src/ --proto_path=src/ src/bar/bar.proto
src/bar/bar.pb.go

// Code generated by protoc-gen-go.
// source: bar/bar.proto
// DO NOT EDIT!

/*
Package bar is a generated protocol buffer package.

It is generated from these files:
    bar/bar.proto

It has these top-level messages:
    Bar
*/
package bar

import proto "github.com/golang/protobuf/proto"
import math "math"
import foo "foo/foo.pb"

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = math.Inf

type Bar struct {
    Foo              *foo.Foo `protobuf:"bytes,1,opt,name=foo" json:"foo,omitempty"`
    Bar              *string  `protobuf:"bytes,2,opt,name=bar" json:"bar,omitempty"`
    XXX_unrecognized []byte   `json:"-"`
}

func (m *Bar) Reset()         { *m = Bar{} }
func (m *Bar) String() string { return proto.CompactTextString(m) }
func (*Bar) ProtoMessage()    {}

func (m *Bar) GetFoo() *foo.Foo {
    if m != nil {
        return m.Foo
    }
    return nil
}

func (m *Bar) GetBar() string {
    if m != nil && m.Bar != nil {
        return *m.Bar
    }
    return ""
}

func init() {
}

go build foo
go build bar
src/bar/bar.pb.go:18:8: cannot find package "foo/foo.pb" in any of:
/usr/local/Cellar/go/1.3/libexec/src/pkg/foo/foo.pb (from $GOROOT)
/Users/patrick/Code/workspace/src/foo/foo.pb (from $GOPATH)

When I change import foo "foo/foo.pb" to import foo "foo" the package will build correctly. Is this an issue with my use of the generated files or an issue with code generation? Thanks.

[tv42]

Any update? I'm still running a forked copy just to get code that compiles.

[tv42]

Ping? As is, this library is unusable.

[jianyuan]

Hi @tv42, I had this issue today. I find that using https://github.com/gogo/protobuf addresses this issue for me.

protoc --proto_path=${GOPATH}/src:. --gogo_out=. **/*.proto

[YijinLiu]

This exact issue was firstly posted at least 3 years ago. Cannot believe that Google haven't solved it yet!!

[bradfitz]

@dsymonds, can you look at this?

[freedom1989]

how to solve this problem?

[dkolbly]

For those stumbling across this issue while trying to figure out how to get protoc to generate useful import paths for Go (as I did)... while this issue as originally reported may have been solved, this is still a very confusing area of protoc/Go, and the docs at https://developers.google.com/protocol-buffers/docs/reference/go-generated describe a solution in the section on Packages that worked for me:

You can override the default generated package for a particular .proto using the option go_package in your .proto file.

tl;dr, I basically did this in my .proto file:

package foo;
option go_package = "bitbucket.org/dkolbly/lib/foo";

and then other .proto files importing that one wound up with the right import statement.

[mimoo]

It does not work great btw. If you use go modules then protoc will produce the file in the bitbucket.org/dkolbly/lib/foo folder (in the previous comment example).

I'm currently using the protobuf files from a different rust project, and I have to manually modify them to add the go_package option every time there is a change.

This is quite hacky, but here is what I do to solve both of these problems if you have this setup:

• you use go modules with go mod init github.com/mimoo/X
• you have your .proto files in proto/types

then this bash script works:

# add `go_package` to all proto files
for file in proto/types/*
do 
    sed -i '' 's#syntax = "proto3";#syntax = "proto3"; option go_package = "github.com/mimoo/X/proto/types";#' $file
done

# generate protobuf files
protoc -I proto/types --go_out=proto/types proto/types/*.proto

# move generated files to correct place, remove artifacts
mv proto/types/github.com/mimoo/X/proto/types/* proto/types
rm -rf proto/types/github.com

but that is super hacky... how come Golang doesn't have first class support for imports?

@dsymonds I think you should re-open this issue.

[dcow]

I have this problem because I am outputting my generated protobufs in a different directory than the the one in which the proto source files live, so solutions that depend on the output path matching the import path don't work for me.

The correct solution is, at protobuf generate/compile time, to specify a the package that the compiler should use for a given input proto file. This way you can have shared protobuf files and downstream consumers organize the artifacts however they want without polluting the source files with a go_package option (which as people have pointed out is specific to the downstream use case).

It turns out there is support for such an option! It looks like:

--go_out=Mfoo/foo.proto=bitbucket.org/dkolbly/lib/foo

You can specify any number of these and they're seperated from other options with a comma, like normal.

[Gilthoniel]

Small precision to @dcow 's reply: --go_out=Mfoo/foo.proto=bitbucket.org/dkolbly/lib/foo:..

Notice the semicolon and point at the end to generate the file in the current folder.

[lolin9]

Golang: because hacks with a zillion fanboys on Twitter can too impose an incredibly badly designed language on an entire industry.