Envoy: Envoy API proto build rules via Bazel native proto support

[htuch]

Bazel has _some_ support for generating protos via its inbuilt proto_library and cc_proto_library rules today (see https://docs.bazel.build/versions/master/be/protocol-buffer.html). We've so far been using the imported rules for cc_proto_library etc from https://github.com/google/protobuf/blob/master/protobuf.bzl, because we wanted a high degree of control over the protobuf compiler and runtime (associated largely with our former approach to external dep build for protobuf prior to #1682).

We may now be able to switch to native support, which has the advantage that the proto_library rule provides language neutral support for management of transitive proto dependencies. This is useful when doing things like adding additional proto rules to generate constraint validation code (https://github.com/lyft/protoc-gen-validate) and documentation (#1866), since you can avoid horrible hacks like https://github.com/lyft/protoc-gen-validate/blob/master/bazel/go_proto_library.bzl.diff. This might also make it simpler to add support for additional languages (e.g. Go), since the transitivity management is factored out.

It's a bit unclear to me how mature the support is in native Bazel for the latest protobuf toolchain, so creating this tracking issue to investigate. https://blog.bazel.build/2017/02/27/protocol-buffers.html describes how to get started. One thing we'll need to do is rename our protobuf repo to @com_google_protobuf.

@jmillikin-stripe @lizan for comment. @rodaine for PGV applications.

[htuch]

This is already suggested in #1847 as option 1, will close that out and continue here, as I have a bit more Envoy specific context in the issue description.

[htuch]

I have a WiP moving data-plane-api to native Bazel and proto_library for C++. However, there is no Python native support, i.e. native.py_proto_library doesn't exit. We don't really care that much about Python, but it would be good to work through support for this to see how it might work for Go and also for plugin customization.

The native Bazel cc_proto_library rule is baked in via Java, see https://github.com/bazelbuild/bazel/tree/master/src/main/java/com/google/devtools/build/lib/rules/cpp/proto. We need to do some Skylark rule for py_proto_library that basically mirrors this (hopefully simplified). Will experiment further today.

[jmillikin-stripe]

The native proto_library rule is still primitive, and language-specific Protobuf libraries don't really support it well. go_rules HEAD can handle proto_library deps, I think. It might be worth offering multiple targets (//api/filter:mongo_proxy as proto_library, //api/filter:mongo_proxy_cc as cc_proto_library, etc) until the upstream rules have consolidated.

[htuch]

Which languages do we want to support? If we can provide rules for C++, Java (both of these using the native Bazel), Python (custom Skylark), Go (https://github.com/bazelbuild/rules_go) for data-plane-api, can we call it good enough for now? I think if we can provide an example of how to do this for Python, we're basically there for anyone who wants to extend.

[rshriram]

cc @kyessenov FYI

[kyessenov]

I can help out with go_rules once you move to native proto libraries.

[htuch]

Was busy this week, but I did play around a bit with Python generation a bit. This seems almost possible but hard to do with existing Bazel tech, see https://github.com/bazelbuild/bazel/issues/3935.

I think a good compromise would be to proto_library support, switch to this for C++ and keep the existing use of py_proto_library from @com_google_protobuf.