Protobuf: Does not work on ruby:2.3-alpine docker image

[sullerandras]

It throws __memcpy_chk: symbol not found error when trying to require 'google/protobuf'.
As i understand the issue is that Apline is using musl instead of glibc. Would be nice if google-protobuf could work on Alpine ruby since other ruby images are significantly bigger.

Environment: ruby:2.3-alpine docker image
To reproduce it:

$ docker run -it --rm ruby:2.3-alpine sh
/ # gem install google-protobuf
Fetching: google-protobuf-3.1.0-x86_64-linux.gem (100%)
Successfully installed google-protobuf-3.1.0-x86_64-linux
1 gem installed
/ # irb
irb(main):001:0> require 'google/protobuf'
LoadError: Error relocating /usr/local/bundle/gems/google-protobuf-3.1.0-x86_64-linux/lib/google/protobuf_c.so: __memcpy_chk: symbol not found - /usr/local/bundle/gems/google-protobuf-3.1.0-x86_64-linux/lib/google/protobuf_c.so
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
    from /usr/local/bundle/gems/google-protobuf-3.1.0-x86_64-linux/lib/google/protobuf.rb:50:in `rescue in <top (required)>'
    from /usr/local/bundle/gems/google-protobuf-3.1.0-x86_64-linux/lib/google/protobuf.rb:47:in `<top (required)>'
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:133:in `require'
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:133:in `rescue in require'
    from /usr/local/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:40:in `require'
    from (irb):1
    from /usr/local/bin/irb:11:in `<main>'
irb(main):002:0>

Workaround is to change the docker image to ruby:2.3-slim, which is about 150 megabytes bigger.

[apstndb]

I am also faced with a problem. I have tried to solve it by some workaround.

1. Force to build by source using :git in Gemfile. (See also https://github.com/bundler/bundler/issues/1679)
2. __va_copy is glibc's internal function. Use standard va_copy ( https://github.com/google/protobuf/blob/master/ruby/ext/google/protobuf_c/upb.h#L129 )
3. There are some auto-generated files in ./lib directories but don't exist in repos and source gem. Copy from precompiled gems.

After doing it, the error in protobuf seems to be resolved.
( If you plan to use google-cloud-*, see also https://github.com/grpc/grpc/issues/6850 )

[ernie]

For those following along here, it's worth noting that as of Bundler 1.14, you can set either BUNDLE_FORCE_RUBY_PLATFORM=1 in your environment or use the corresponding force_ruby_platform config entry to get Bundler to build gems from source and not try to use the precompiled version. I thought that would solve my issues building on Alpine, but alas:

/ # vi Gemfile
/ # export BUNDLE_FORCE_RUBY_PLATFORM=1
/ # bundle
Fetching gem metadata from http://rubygems.org/.
Fetching version metadata from http://rubygems.org/
Resolving dependencies...
Installing google-protobuf 3.2.0.1 with native extensions
Using bundler 1.14.6
Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
current directory:
/usr/local/bundle/gems/google-protobuf-3.2.0.1/ext/google/protobuf_c
/usr/local/bin/ruby -r ./siteconf20170313-16-b4ycm5.rb extconf.rb
creating Makefile
current directory:
/usr/local/bundle/gems/google-protobuf-3.2.0.1/ext/google/protobuf_c
make "DESTDIR=" clean
current directory:
/usr/local/bundle/gems/google-protobuf-3.2.0.1/ext/google/protobuf_c
make "DESTDIR="
compiling protobuf.c
compiling defs.c
compiling storage.c
compiling message.c
compiling repeated_field.c
compiling map.c
compiling encode_decode.c
compiling upb.c
compiling wrap_memcpy.c
linking shared-object google/protobuf_c.so
/usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
protobuf_c.so: No symbol version section for versioned symbol
`memcpy@GLIBC_2.2.5'
/usr/lib/gcc/x86_64-alpine-linux-musl/5.3.0/../../../../x86_64-alpine-linux-musl/bin/ld:
final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Makefile:254: recipe for target 'protobuf_c.so' failed
make: *** [protobuf_c.so] Error 1
make failed, exit code 2
Gem files will remain installed in
/usr/local/bundle/gems/google-protobuf-3.2.0.1 for inspection.
Results logged to
/usr/local/bundle/extensions/x86_64-linux/2.3.0/google-protobuf-3.2.0.1/gem_make.out
An error occurred while installing google-protobuf (3.2.0.1), and
Bundler cannot continue.
Make sure that `gem install google-protobuf -v '3.2.0.1'` succeeds before
bundling.

Looks like wrap_memcpy.c tries to help us out here, to ill effect, in line 42:

__asm__(".symver memcpy,memcpy@GLIBC_2.2.5");

[EDIT: FWIW, just altering extconf.rb to not build wrap_memcpy.o and remove the $LDFLAGS seems to have yielded a working gem on this container. Hoping a tweak to detect this case and avoid the issue isn't too hard to sort out.]

[ernie]

Looks like the fix for #2783 released in 3.2.0.1 was the culprit. So, for those who are willing to build from source to work around this issue, with BUNDLE_FORCE_RUBY_PLATFORM=1 in your environment:

gem 'google-protobuf', '3.2.0'

That's it!

[acozzette]

@ernie I wrote a possible fix on this branch, and essentially just added an #ifdef to ensure that my previous fix only applies when building for glibc. Do you want to try that out and let me know if it works for your case? This should make it possible to build from source again for musl, but I am still not sure of how to solve the broader problem of being able to use our published gems directly without building from source.

[ernie]

@acozzette happy to. Will report back here.

[ernie]

@acozzette yep, the gem built fine. Including output here for documentation purposes:

/opt/app/vendor/protobuf/ruby # rake
/opt/app/vendor/protobuf/ruby # bundle
Using rake 12.0.0
Using google-protobuf 3.2.0.1 from source at `.`
Using power_assert 1.0.1
Using rake-compiler-dock 0.6.0
Using rubygems-tasks 0.2.4
Using bundler 1.14.6
Using rake-compiler 0.9.9
Using test-unit 3.2.3
Bundle complete! 5 Gemfile dependencies, 8 gems now installed.
Bundled gems are installed into /opt/app/vendor/bundle-development.
/opt/app/vendor/protobuf/ruby # bundle exec rake
rm -f lib/google/protobuf/any_pb.rb lib/google/protobuf/api_pb.rb lib/google/protobuf/duration_pb.rb lib/google/protobuf/empty_pb.rb lib/google/protobuf/field_mask_pb.rb lib/google/protobuf/source_context_pb.rb lib/google/protobuf/struct_pb.rb lib/google/protobuf/timestamp_pb.rb lib/google/protobuf/type_pb.rb lib/google/protobuf/wrappers_pb.rb tests/generated_code.rb tests/test_import.rb
mkdir -p tmp/x86_64-linux/protobuf_c/2.3.3
cd tmp/x86_64-linux/protobuf_c/2.3.3
/usr/local/bin/ruby -I. ../../../../ext/google/protobuf_c/extconf.rb
creating Makefile
cd -
cd tmp/x86_64-linux/protobuf_c/2.3.3
/usr/bin/make
compiling ../../../../ext/google/protobuf_c/protobuf.c
compiling ../../../../ext/google/protobuf_c/defs.c
compiling ../../../../ext/google/protobuf_c/storage.c
compiling ../../../../ext/google/protobuf_c/message.c
compiling ../../../../ext/google/protobuf_c/repeated_field.c
compiling ../../../../ext/google/protobuf_c/map.c
compiling ../../../../ext/google/protobuf_c/encode_decode.c
compiling ../../../../ext/google/protobuf_c/upb.c
compiling ../../../../ext/google/protobuf_c/wrap_memcpy.c
linking shared-object google/protobuf_c.so
cd -
mkdir -p tmp/x86_64-linux/stage/lib/google
mkdir -p tmp/x86_64-linux/stage/ext/google/protobuf_c
cp ext/google/protobuf_c/defs.c tmp/x86_64-linux/stage/ext/google/protobuf_c/defs.c
cp ext/google/protobuf_c/encode_decode.c tmp/x86_64-linux/stage/ext/google/protobuf_c/encode_decode.c
cp ext/google/protobuf_c/extconf.rb tmp/x86_64-linux/stage/ext/google/protobuf_c/extconf.rb
cp ext/google/protobuf_c/map.c tmp/x86_64-linux/stage/ext/google/protobuf_c/map.c
cp ext/google/protobuf_c/message.c tmp/x86_64-linux/stage/ext/google/protobuf_c/message.c
cp ext/google/protobuf_c/protobuf.c tmp/x86_64-linux/stage/ext/google/protobuf_c/protobuf.c
cp ext/google/protobuf_c/protobuf.h tmp/x86_64-linux/stage/ext/google/protobuf_c/protobuf.h
cp ext/google/protobuf_c/repeated_field.c tmp/x86_64-linux/stage/ext/google/protobuf_c/repeated_field.c
cp ext/google/protobuf_c/storage.c tmp/x86_64-linux/stage/ext/google/protobuf_c/storage.c
cp ext/google/protobuf_c/upb.c tmp/x86_64-linux/stage/ext/google/protobuf_c/upb.c
cp ext/google/protobuf_c/upb.h tmp/x86_64-linux/stage/ext/google/protobuf_c/upb.h
cp ext/google/protobuf_c/wrap_memcpy.c tmp/x86_64-linux/stage/ext/google/protobuf_c/wrap_memcpy.c
cp lib/google/protobuf.rb tmp/x86_64-linux/stage/lib/google/protobuf.rb
mkdir -p tmp/x86_64-linux/stage/lib/google/protobuf
cp lib/google/protobuf/message_exts.rb tmp/x86_64-linux/stage/lib/google/protobuf/message_exts.rb
cp lib/google/protobuf/repeated_field.rb tmp/x86_64-linux/stage/lib/google/protobuf/repeated_field.rb
cp lib/google/protobuf/well_known_types.rb tmp/x86_64-linux/stage/lib/google/protobuf/well_known_types.rb
mkdir -p tmp/x86_64-linux/stage/tests
cp tests/basic.rb tmp/x86_64-linux/stage/tests/basic.rb
cp tests/generated_code_test.rb tmp/x86_64-linux/stage/tests/generated_code_test.rb
cp tests/stress.rb tmp/x86_64-linux/stage/tests/stress.rb
install -c tmp/x86_64-linux/protobuf_c/2.3.3/protobuf_c.so lib/google/protobuf_c.so
cp tmp/x86_64-linux/protobuf_c/2.3.3/protobuf_c.so tmp/x86_64-linux/stage/lib/google/protobuf_c.so
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/any.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/api.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/duration.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/empty.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/field_mask.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/source_context.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/struct.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/timestamp.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/type.proto
../src/protoc -I../src --ruby_out=lib ../src/google/protobuf/wrappers.proto
../src/protoc --ruby_out=. tests/generated_code.proto
../src/protoc --ruby_out=. tests/test_import.proto

[acozzette]

Thanks @ernie! I just sent out that fix as pull request #2870.

[acozzette]

Actually let me reopen this; the issue about building from source is now resolved, but I'm not sure if we still want to look into ways of providing a published gem that works out of the box.

[lifeiscontent]

@acozzette how do I force a gem to build? I tried setting BUNDLE_FORCE_RUBY_PLATFORM in my docker-compose file but it doesn't seem to build.

[OpakAlex]

Same issue for me. why this issue closed without any solutions?

gem "grpc", "1.25.0", platforms: ["ruby"]
gem "google-protobuf", "3.11.2", platforms: ["ruby"]

FROM ruby:2.6.5-alpine3.10

LoadError: Error relocating /bundle/ruby/2.6.0/gems/google-protobuf-3.11.2/lib/google/protobuf_c.so: __va_copy: symbol not found - /bundle/ruby/2.6.0/gems/google-protobuf-3.11.2/lib/google/protobuf_c.so

Can somebody look?

[archonic]

I've got this in my Gemfile:

gem 'google-protobuf', platforms: [:ruby]
gem 'grpc', platforms: [:ruby]

Which is building correctly:

Fetching anyway_config 1.4.4
Installing anyway_config 1.4.4
Fetching google-protobuf 3.11.2
Installing google-protobuf 3.11.2 with native extensions
Fetching googleapis-common-protos-types 1.0.4
Installing googleapis-common-protos-types 1.0.4
Fetching grpc 1.26.0
Installing grpc 1.26.0 with native extensions

But when I run the ruby app (rails, to precompile assets), I get this:

 ---> Running in 8aca182b74c7
/usr/local/lib/ruby/2.6.0/bundler/spec_set.rb:91:in `block in materialize': Could not find google-protobuf-3.11.2 in any of the sources (Bundler::GemNotFound)
    from /usr/local/lib/ruby/2.6.0/bundler/spec_set.rb:85:in `map!'
    from /usr/local/lib/ruby/2.6.0/bundler/spec_set.rb:85:in `materialize'
    from /usr/local/lib/ruby/2.6.0/bundler/definition.rb:170:in `specs'
    from /usr/local/lib/ruby/2.6.0/bundler/definition.rb:237:in `specs_for'
    from /usr/local/lib/ruby/2.6.0/bundler/definition.rb:226:in `requested_specs'
    from /usr/local/lib/ruby/2.6.0/bundler/runtime.rb:108:in `block in definition_method'
    from /usr/local/lib/ruby/2.6.0/bundler/runtime.rb:20:in `setup'
    from /usr/local/lib/ruby/2.6.0/bundler.rb:107:in `setup'
    from /usr/local/lib/ruby/2.6.0/bundler/setup.rb:20:in `<top (required)>'
    from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /usr/local/lib/ruby/2.6.0/rubygems/core_ext/kernel_require.rb:54:in `require'
    from /app/config/boot.rb:5:in `<top (required)>'
    from bin/rails:3:in `require_relative'
    from bin/rails:3:in `<main>'

[stenlarsson]

We also had problems running the google-protobuf gem on Alpine Linux. Compiling worked fine, but the gem could not be loaded:

LoadError:
  Error relocating [...]/ruby/2.6.0/gems/google-protobuf-3.11.2/lib/google/protobuf_c.so: __va_copy: symbol not found - [...]/ruby/2.6.0/gems/google-protobuf-3.11.2/lib/google/protobuf_c.so

I think the problem is that code checks if we are compiling with C99 here: https://github.com/protocolbuffers/protobuf/blob/a62ee83401cb28052342c3833dcdb552e9c5a8ca/ruby/ext/google/protobuf_c/upb.h#L81

While at the same time it is specified that we compile with gnu90 here:
https://github.com/protocolbuffers/protobuf/blob/a62ee83401cb28052342c3833dcdb552e9c5a8ca/ruby/ext/google/protobuf_c/extconf.rb#L6

It will instead enter the elif case and use the symbol __va_copy: https://github.com/protocolbuffers/protobuf/blob/a62ee83401cb28052342c3833dcdb552e9c5a8ca/ruby/ext/google/protobuf_c/upb.h#L110

Not sure why it compiles, but musl, which is used on Alpine Linux instead of glibc, does not provide this symbol.

A workaround we found is to define __va_copy to va_copy

bundle config build.google-protobuf --with-cflags=-D__va_copy=va_copy

[MatayoshiMariano]

This is suppose to be fixed in Alpine 3.11.5, check main/protobuf: fix ruby gem - missing symbol __va_copy line, but i'm still getting it

[tzusman]

@MatayoshiMariano I'm running into the same issue. Have you found a solution?

[MatayoshiMariano]

@tzusman unfortunately not, right now i'm using

gem "grpc", "~> 1.23.0", platforms: ["ruby"]
gem "google-protobuf", "~> 3.9.0", platforms: ["ruby"]

And ruby 2.6.6

Can't upgrade to 2.7.1 until this and this issue is fixed

[stanhu]

The workaround in https://github.com/protocolbuffers/protobuf/issues/2335#issuecomment-579913357 worked for me. https://github.com/protocolbuffers/protobuf/pull/7773 should fix this problem.