Rubygems: `bundler install` should not fail when a native platform gem's required_ruby_version is not met, and a suitable `ruby` platform gem is available

[flavorjones]

Describe the problem as clearly as you can

Given:

• I am running a specific version of Ruby
• a Gem is available as both a ruby platform gem and a native (e.g., x86_64-linux) platform gem
  
  
  
  • where my specific version of Ruby does not meet the requirements set by the native platform gem via spec.required_ruby_version
  
  
  • where my specific version of Ruby does meet the requirements set by the ruby platform gem via spec.required_ruby_version
  
  

I expect bundler install (and/or gem install?) to install the ruby platform gem.

More specifically, I expect bundler install to iterate through the list of Rubygems Platforms for my system, from more-specific to more-general, until it finds a gem that meets all the requirements. It should not give up and raise an error until the list of platforms has been exhausted.

Instead, I see bundler install select the native gem, and then fail while resolving dependencies with an error message like:

nokogiri-1.11.0.rc3-x86_64-linux requires ruby version < 2.8.dev, >= 2.4, which is incompatible with the current version, ruby 3.0.0p-1

Additional context

Given the work done at https://github.com/rubygems/bundler/pull/7522 to improve platform-specific gem resolution, I expected this to eventually install the matching ruby gem rather than give up after failing to install the non-matching native gem. Perhaps this is my misunderstanding of that issue.

As a maintainer of Nokogiri, about to release native gems (see https://github.com/sparklemotion/nokogiri/issues/2075), I'd like very much for this to work as described. I believe the current behavior will cause confusion for Ruby users whenever a new version of Ruby is made available, and that will create additional support work for native gem maintainers.

I can imagine this unfortunate scenario:

1. ruby 3.0.0 final is released in December 2020
2. nokogiri v1.11.0 is simultaneously released with native gems that support ruby ~> 3.0.0
3. time passes
4. ruby 3.1 final is released in December 2021 (or, more realistically, ruby 3.1.0preview1 is released in September 2021 and people start trying it out)
5. developers everywhere try to upgrade and see the above error message
6. developers open multiple issues on bundler and nokogiri, and post on stackoverflow, and complain on twitter ...
7. adoption of ruby 3.1.0 is slowed or inhibited

Further confounding the problem is that Ruby users have no way to pin a specific gem to the ruby platform, instead they have to set the bundler config force_ruby_platform and avoid native gems entirely, until each and every gem they use supports the new version of Ruby.

I hope I've explained this scenario adequately -- it's a ticking time bomb once a commonly-used Gem like nokogiri ships native platform gems. Please ask me questions if I'm not being clear, or correct me if my understanding is incorrect.

(I'm tagging @tenderlove, @larskanis, and @casperisfine for visibility because I've discussed this scenario with each of them, and they have context they may wish to share.)

Post steps to reproduce the problem

Brief

1. on an x86_64-linux machine
2. install ruby 3.0.0preview1, rubygems 3.2.0.rc1, and bundler 2.2.0.rc1
3. attempt to install nokogiri =1.11.0.rc3 from rubygems.org using bundler install
4. see the above error message

Longer

On a linux machine:

Linux juno 5.4.0-48-lowlatency #52-Ubuntu SMP PREEMPT Thu Sep 10 11:51:50 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

Running ruby installed via rvm:

$ ruby -v
ruby 3.0.0preview1 (2020-09-25 master 0096d2b895) [x86_64-linux]

$ gem -v
3.2.0.rc.1

$ bundler -v
Bundler version 2.2.0.rc.1

With these configurations:

$ gem env
RubyGems Environment:
  - RUBYGEMS VERSION: 3.2.0.rc.1
  - RUBY VERSION: 3.0.0 (2020-09-25 patchlevel -1) [x86_64-linux]
  - INSTALLATION DIRECTORY: /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1
  - USER INSTALLATION DIRECTORY: /home/flavorjones/.gem/ruby/3.0.0
  - RUBY EXECUTABLE: /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/bin/ruby
  - GIT EXECUTABLE: /usr/bin/git
  - EXECUTABLE DIRECTORY: /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1/bin
  - SPEC CACHE DIRECTORY: /home/flavorjones/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/etc
  - RUBYGEMS PLATFORMS:
     - ruby
     - x86_64-linux
  - GEM PATHS:
     - /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1
     - /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/lib/ruby/gems/3.0.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
     - "gem" => "--no-ri --no-rdoc --no-document"
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1/bin
     - /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1@global/bin
     - /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/bin
     - /home/flavorjones/.rvm/bin
     - <other entries redacted>

$ bundler env
## Environment

...
Bundler       2.2.0.rc.1
  Platforms   ruby, x86_64-linux
Ruby          3.0.0p-1 (2020-09-25 revision 0096d2b895395df5ab8696d3b6d444dc1b7730b6) [x86_64-linux]
  Full Path   /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/bin/ruby
  Config Dir  /home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/etc
RubyGems      3.2.0.rc.1
  Gem Home    /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1
  Gem Path    /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1:/home/flavorjones/.rvm/rubies/ruby-3.0.0-preview1/lib/ruby/gems/3.0.0
  User Home   /home/flavorjones
  User Path   /home/flavorjones/.gem/ruby/3.0.0
  Bin Dir     /home/flavorjones/.rvm/gems/ruby-3.0.0-preview1/bin
Tools         
  Git         2.25.1
  RVM         1.29.10-next (master)
  rbenv       not installed
  chruby      not installed
...

## Bundler Build Metadata

...
Built At          2020-10-09
Git SHA           unknown
Released Version  false
...

## Bundler settings

...
jobs
  Set for the current user (/home/flavorjones/.bundle/config): 2
gem.test
  Set for the current user (/home/flavorjones/.bundle/config): "rspec"
gem.mit
  Set for the current user (/home/flavorjones/.bundle/config): true
gem.coc
  Set for the current user (/home/flavorjones/.bundle/config): true
force_ruby_platform
  Set for the current user (/home/flavorjones/.bundle/config): false
...

## Gemfile

### Gemfile

...ruby
source "https://rubygems.org"
gem "nokogiri", "=1.11.0.rc3"
...

### Gemfile.lock

...
<No /home/flavorjones/tmp/foobar/Gemfile.lock found>
...

In a project directory with the following Gemfile:

# Gemfile
source "https://rubygems.org"
gem "nokogiri", "=1.11.0.rc3"

Run:

$ bundle install
Fetching gem metadata from https://rubygems.org/.......
Resolving dependencies...
nokogiri-1.11.0.rc3-x86_64-linux requires ruby version < 2.8.dev, >= 2.4, which is incompatible with the current version, ruby 3.0.0p-1

Which command did you run?

bundle install

What were you expecting to happen?

I expected bundler to choose the ruby platform gem rather than the native platform gem.

More specifically, I expect bundler install to iterate through the list of Rubygems Platforms for my system, from more-specific to more-general, until it finds a gem that meets all the requirements. It should not give up and raise an error until the list of platforms has been exhausted.

What actually happened?

Error message:

nokogiri-1.11.0.rc3-x86_64-linux requires ruby version < 2.8.dev, >= 2.4, which is incompatible with the current version, ruby 3.0.0p-1

If not included with the output of your command, run bundle env and paste the output below

See above.

[deivid-rodriguez]

Hi @flavorjones!

This is definitely how I expected this to work too after rubygems/bundler#7522 :grimacing:.

However, it turns out that this behaviour is only enabled with the specific_platform setting, which will get enabled by default in bundler 3.

Since bundler 2.2.0 is supposed to be the release finally providing a nicer platform support, my proposal would be to enable this setting by default in bundler 2.2.0 instead.

Thoughts?

EDIT: Thanks for the nicely written report by the way, it's really great when people spend time on writing good issues :heart:.

[flavorjones]

Somehow I missed the specfic_platform bundler config setting. :exploding_head:

Yes, I personally think we should turn that on in Bundler 2.2 in time for a Ruby 3 release. But my opinion is obviously non-binding, as I'm not a maintainer.

EDIT: I've gone through the installation experience with specific_platform set to true and it worked _exactly_ as I'd hoped.

[deivid-rodriguez]

The main inconvenience of this setting is that it will generate lockfile changes by adding the current "specific platform" (x86_64-linux or whatever) to the lockfile. I guess some users could find this annoying, but it seems necessary for proper multiplatform support and other changes we are shipping already go in this direction, like https://github.com/rubygems/bundler/pull/7580, where bundler will automatically add all platforms specified in the Gemfile to the lockfile.

So I'd say let's do this now, but I'd like to hear other thoughts.

If I don't get feedback in a few days I'll go ahead and merge #4015.

[deivid-rodriguez]

@flavorjones I didn't forget about this, I'm just fixing some issues with specific_platform before enabling it by default :+1:.

Also, regarding

Further confounding the problem is that Ruby users have no way to pin a specific gem to the ruby platform, instead they have to set the bundler config force_ruby_platform and avoid native gems entirely, until each and every gem they use supports the new version of Ruby.

I added https://github.com/rubygems/rubygems/pull/4049 to implement this.

[deivid-rodriguez]

With the fixed platform resolution logic https://github.com/rubygems/rubygems/pull/4049 is not as necessary, but I found some cases where it could still be handy.