Protobuf: No Protoc binaries for support Arm64 on Maven Central

[guyuqi]

Hi,
We want to use 'protoc' to compile the '.proto' file in the java project with a maven plugin for Arm64 platform.
Unfortunately, there are no Arm64 binaries on https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/.

It is noted that the protoc binaries are built and deployed to Maven Central by the kind folks at Google.
Could you kindly please include Arm64 binaries in the next release?

Thanks a lot!

[liujisi]

Unfortunately we don't have the environment to test the protoc build. Would you be able to modify the /protoc-artifact and enable cross-compiling from a x86 linux environment?

[guyuqi]

Hi @pherl ，

Thank you for your attention on this issue.
We'd like to find a way to offer you Arm64 environment to verify the protoc build.

Could you please give contact info( email ) to us for more further details?
My Email: yuqi.[email protected]

BRs
Yuqi

[johnmark]

@pherl - we can arrange for hardware to be sent to you, or we can have Packet provision resources for you as part of the WorksOnArm initiative - see https://github.com/WorksOnArm/cluster/issues

If you open a new issue there, you'll see a form to fill out to get access.

[liujisi]

Thanks for the offer, however, that probably requires a series of legal/reviews and would take a long time. I played with the cross-compiling and it seems to be working, would you be able to try out the binary and see whether it actually works:
protoc.zip

Note this binary requires the following dynamic libraries:

$ objdump -p ../src/protoc
Dynamic Section:
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6
  NEEDED               ld-linux-aarch64.so.1

Are those available in your platform?

Binary info:

$ file ../src/protoc
../src/protoc: ELF 64-bit LSB  executable, ARM aarch64, version 1 (GNU/Linux), dynamically linked (uses shared libs), for GNU/Linux 3.7.0, BuildID[sha1]=01c97bde3c5788cc4c9c7c90658e6f4cb5a392d1, stripped

[vielmetti]

@pherl - at Packet we currently have the Golang project generating binaries on a system using Works on Arm resources, so I am confident that your legal review will be modest.

I have linked a related issue in linkerd caused by the lack of this binary.

[liujisi]

@vielmetti Even if we get legal approval, there's unfortunately not enough the man power to build and test protoc on different platforms manually. Current released protoc are all cross-compiled using a single script in a linux machine.

Would you please check the package I posted in the previous comment and see if that works?

[vielmetti]

With that protoc, I am able to compile linkerd and run a sample test:

I 1109 23:09:52.565 UTC THREAD333: linkerd 1.3.1-SNAPSHOT (rev=718514fb1d4b86153820880162d3c9559e115725) built at 20171109-230935           

Short of full-scale integration testing - which is on the roadmap - I would say this file "works".

[liujisi]

Cool! I will include aarch64-linux in the 3.5.0 mvn protoc artifacts then.

[vielmetti]

Thanks @pherl !

[guyuqi]

@pherl ,
For Apache-Hbase takes 2.5.0 protoc to compile the 'proto2' file, could you please also include 2.5.0 protoc artifacts for aarch64-linux in maven central?

[liujisi]

I'm not able to modify the already closed/published maven repos. To make that work, there needs to be a 2.5.1 minor release (after 2.5.0 was released for 4 years..) Then ask HBase to depend on the new version. I remembered they were very cautious about upgrading protobuf dependencies. Would you be able to get their buy-in first?

[vielmetti]

@guyuqi is there an open issue somewhere tracking HBase issues? happy to track that.

[guyuqi]

@vielmetti , Hbase ticket: HBASE-19146

[guyuqi]

@pherl, thanks for your quick response. It is definitely appreciated that you would like to make the 2.5.1 minor release. I'll discuss it with hbase guys in HBASE-19146. Thanks!

[liujisi]

FYI 3.5.0 is now released with aarch_64 protoc in both maven repo and download page.

[guyuqi]

@pherl Could you also please make the 2.5.1 release which includes the aarch64 protoc?
Thanks!

[os72]

Folks, I got a few binary contributions here, including aarch_64 for older versions, but no 2.5.0 yet unfortunately, there was some difficulty with the build (https://github.com/os72/protoc-jar/issues/33)

https://github.com/os72/protoc-jar
http://central.maven.org/maven2/com/github/os72/protoc/

Ping @garciagorka

[garciagorka]

As @os72 said, v2.5.0 had a dependency with the architecture and would not build fro aarch64. I did not dig into the issue, just tried to build and failed. Both earlier and later versions seemed to build OK and my builds are available in https://github.com/os72/protoc-jar
I have no plans on investigating this issue further, sorry.

[acozzette]

Let me close this issue because in the past few releases we have been publishing aarch64 binaries.